Spam Fighting Software Installed!
Over the last few months we have been testing and refining a new spam
fighting system for all servers that is designed to drastically reduce
the amount of spam that makes it to your inboxes. Throughout this, we
have added many layers of spam protection which targets different types
of spam while providing protection against many future methods of
bypassing spam filtering. Many of you have reported drastically reduced amounts of spam being received, and here is why..
SMTP Time Filtering
When email is sent from server to server it is done via an SMTP exchange
during which a remote email server will connect to the mail server here
to deliver an email. The unfortunate side effect of this process is that
it was designed many years ago without a form of identity confirmation
so any email server can claim to be any other email server. With this in
mind we have installed the following tests to attempt to combat this
problem.
HELO Testing
This method of testing is designed to block servers that attempt to forge their HELO string to bypass less intelligent spam filtering systems. These filters are capable of blocking tens of thousands of emails per hour with a near 0% false positive rate due to the way the test was designed.
DNSBL Testing
This method of testing can mean many different things depending on which blacklists are used. We rotate blacklists frequently and keep up with the latest happenings so we know exactly what we are blocking by using the lists. Our choices in targets are known confirmed spam sources, known bad IP space (stolen IP space), and IP ranges where email should never come from. These are excellent methods of blocking spam with a few low false positive rate due to the specific targeting of spam only IP space.
SURBL/URIBL Testing
This method of testing is mostly unknown to the vast majority of the internet community and is one of the more promising new methods of blocking spam. Instead of targeting where spam comes fromspecifically as DNSBLs do, this method of testing is designed to block the websites spammers are attempting to send you to. This is also known as blocking spamvertised websites. This filtering reads inside the body of the email for the URLs inside the email and checks them against the
SURBL and URIBL blacklists for any mention of spamvertised websites. An
additional reason for its effectiveness is it will block emails that contain these websites no matter what IP address they are sent from so newly compromised servers cannot be used to spam when this method is used. The response time of Spamcop without the false positives.
With all of this in mind we must come to terms with the thought that
spammers will occasionally be able to bypass these blocking methods with
tricks such as enclosing their message in an image or putting a *
character in the URL in the email to prevent SURBL/URIBL blocking. This
is where Spam Assassin is more effective due to its ability to parse and
scan all parts of the email with a variety of tests before scoring the
email.
Spam Assassin Filtering:
Image Only spam - This form of spam is typically when an image is included with a large block of random text from a book or website which
attempts to bypass spam filtering by including no filterable parts of
the email. Some methods for stopping this include OCR which allows the
software to parse the image to find certain strings within the image,
but this method is very resource intensive. We have found that by
allowing Spam Assassin to look at characteristics of the Image Only spam
such as a single large image with test below it we have been able to
block these with a high degree of success without any false positives to
note. This form of testing is not 100%, but combined with other testing
it will block up to 99% of image only spam.
Collaborative network testing
This form of testing is much like spam
cop in that it accepts reports from many different servers and uses its
framework to process if an email is spam or not. Its effectiveness
depends largely on the age of the email with its strongest point being
is relatively short time between new spam being send out to servers
around the world and the system marking those emails as likely bulk spam email. When used with other testing within Spam Assassin this proves to be a very effective method of blocking in addition to bayesian filters.
Bayesian filtering
Up until now the bayesian engine has been used mostly as a secondary method of filtering, but recent improvements allow for much more efficient updates of the bayesian databases. After a few days or weeks (depending on the level of spam you receive) the bayesian engine will begin to help Spam Assassin block spam much more effectively for your accounts. Training is automatic and is setup to work without any additional setup beyond choosing the proper spam score.
SPF Record testing
As many of you might be aware a very common practice in spamming is to spoof the from address of the email. As this happens some websites will publish SPF records to help others block the
spam that contains their spoofed from address. Spam Assassin at Inet
will now make use of these records to confirm if the SPF records match.
No points are added or deducted for properly matching records, but if
the records do not match according to tests then the message is scored
higher as it will be more likely to be a joe job.
As you can see we have made massive improvements to the overall effectiveness of spam blocking at Inet. We are always trying improve on an already excellent system, and allow our clients to have a trouble free online business.