Townsville Linux Users Group

SpamAssassin is an open source application that examines incoming email and categorises it as spam (or not).  The project run by the Apache foundation.  Their home page can be found at: http://spamassassin.apache.org/  It can integrate with a wide variety of MTAs (mail servers). 

SpamAssassin works by performing a number of tests on the email and assigning points based on their outcome.  If an email accrues more than a certain number of points, it is classed as Spam.

SpamAssassin is highly configurable.  Each user may have their own spam threshold.  They may even have their own scoring system.

There are a wide variety of tests that it performs (see:  http://spamassassin.apache.org/tests_3_2_x.html for the full list) 

• Tests to find words that look like viagra etc
• Is the sender in a RBL
• Does the sender match the SPF record
• Does the body look like spam
• The ratio of text to images
• Bayesian analysis of the content

You can also set up folders containing Spam and Ham (non Spam) for SpamAssassin to learn from.  As a large proportion of email is actually spam, doing this may not be a good idea, as eventually the Bayesian filter gets poisoned and everything ends up looking like spam.

Greylisting is a Spam reduction technique that relies on the fact that *most* spammers aren't using a real MTA (mail server) to send out the Spam.  When greylisting, the receiving mail server gives a temporary failure code to any "suspect" emails.  A valid mail server will take that in it's stride and resend the message again later, whilst a spam bot will not.

The greylisting server determines which emails to challenge by checking the combination of:

• Email from address
• Email to address
• IP address of sending mail server

If that combination hasn't been seen before, the server records that information and the date / time.  The email is temporarily rejected.

 There is a time to go live that is typically a server parameter.  The next time that combination is seen, the time is checked again.  If it is past the time to go live, the email is accepted, otherwise it is rejected again.  The date / time of this subsequent combination is also recorded.  Many people recommend 60 minutes at the time to go live, but I use 60 seconds.  This appears to work quite satisfactorily.

There may also be a parameter to tell the server when to make each combination go "stale", so that if it hasn't been seen the the correct timeframe the server is challenged again.

Obviously, one of the down sides of greylisting is that some emails will be delayed.  There is also a small chance that some emails may never be delivered.

Some / many greylisting servers can have various whitelists that can reduce any delays for legitimate emails.  For example, you could automatically whitelist any emails from any know real mail server that is likely to email your domain.  As they are real mail servers they *will* pass the greylisting test, so why bother them. NB. This white listing can be done by exact name match, IP match, or regex of either.

One way to obtain the list of mail servers you have contact with is to grep through your mail logs. The following code will search your logs for any mail servers that appear to be legitimate. (ie they have mx, mail, or smtp in their domain name).

	egrep "client=.*mail.*|client=.*mx.*|client=.*smtp.*" /var/log/maillog* \
		| awk '{print $7}' \
		| awk -F = '{print $2}' | awk -F [ '{print $1}' \
		| sort |  uniq -u

If you want to see the frequency of access of each of these servers, you could change the line to:

	egrep "client=.*mail.*|client=.*mx.*|client=.*smtp.*" /var/log/maillog* \
		| awk '{print $7}' \
		| awk -F = '{print $2}' | awk -F [ '{print $1}' \
		| sort |  uniq -c | sort -n

NB. This code has been tested against postfix mail logs.

Obviously, you would make some intelligent choices of which mail servers to white-list.

Once you have your greylisting bedded down,  all of your regular contacts will have their email delivered immediately, so it will be fairly un-noticable to the end user.

There are implementations of greylisting for virtually all popular email servers (including MS Exchange). 

 For further information see http://www.greylisting.org

It has be hypothesized that if too many people start using greylisting, the spammers will adapt.  This may be the case, but at this time, they haven't.  Even if they do, it will mean that the spammers will have to work considerably harder to deliver their spam.