Postfix, dovecot, dspam and dovecot antispam

Notes on implementation for Ubuntu

Background

When I started using an Android phone to check my email I discovered that I needed to change from filtering spam on my desktop client, to doing it on the server.  It would also be very convenient if messages I read on the phone were still marked read when checking email on my desktop, and even better if email could be pushed to my phone instead of it having to poll.  A little bit of searching identified the magic combination that would allow this to happen: K-9 Mail as my Android client, dspam for handling server side spam filtering, dovecot for IMAP IDLE (push email) support and a dovecot plugin called antispam for marking email as spam.

It took a full day of googling and messing around with many different configurations to finally get this working.  Here are my notes on the configuration that eventually worked.  Hopefully they will be useful to someone else out there.  I've not tired re-doing this configuration from scratch with these instructions, so please email me with corrections and edits if you find anything incorrect or missing!

The four step process

Step 1: Get postfix and dovecot working

There are lots of good tutorials on the web on how to do this part, so I'll not attempt to replicate them here.  Do make sure that inbound and outbound email are fully working before attempting the rest of the steps, and make sure you have a backup of your working configuration!

Step 2: Configure postfix to identify spam with dspam

Install dspam and postfix-pcre.

Edit /etc/postfix/master.cf and add a new configuration line for dspam:

dspam     unix  -       n       n       -       10      pipe
  flags=Ru user=vmail argv=/usr/bin/dspam --deliver=innocent,spam --user $recipient -i -f $sender -- $recipient

Some explanation:

• user=vmail means that dspam will run as the vmail user.  In my setup this is the unix user handling all mail and that owns the mailboxes.

Setup a filter that will trigger dspam.  To do this create a new file /etc/postfix/dspam_filter_access:

/./   FILTER dspam:dspam

Now configure postfix to use this filter and so pass mail to dspam.  Edit /etc/postfix/main.cf:

dspam_destination_recipient_limit = 1
smtpd_client_restrictions = permit_sasl_authenticated reject_rbl_client zen.spamhaus.org check_client_access pcre:/etc/postfix/dspam_filter_access

Some explanation:

• dspam_destination_recipient_limit = 1 passes emails one at a time to dspam.
• permit_sasl_authenticated means that any authenticated user will be allowed to send email through this server, and will not pass outbound email through the spam filter.  You must have some mechanism that means outbound email is not filtered for spam.
• pcre:/etc/postfix/dspam_filter_access should be last so that dspam gets a look at the mail before delivering it.

Finally configure dspam by editing /etc/dspam/dspam.conf.  Here are the values that I changed:

Home /var/spool/dspam
TrustedDeliveryAgent "/usr/sbin/sendmail"
UntrustedDeliveryAgent "/usr/lib/dovecot/deliver -d %u"
OnFail error
Trust root
Trust dspam
Trust mail
Trust postfix
Trust vmail

Preference "spamAction=deliver"
Preference "showFactors=off"

TrainPristine off
Opt in

ParseToHeaders on
ChangeModeOnParse off
ChangeUserOnParse full

Some explanation:

• Home - where dspam will keep it's files.
• TrustedDeliveryAgent - how dspam will deliver email it has processed.
• UntrustedDeliveryAgent - probably not needed, but it got in here at some point during the testing.
• Trust vmail - Ensure we can do everything as the vmail user.
• Preference "spamAction=deliver" - I want spam to be marked, but still delivered.
• TrainPristine off - Email is marked up with spam headers.
• Opt in - I wanted to enable this for specific users one at a time.
• ParseToHeaders on, ChangeModeOnParse off, ChangeUserOnParse full - get the user info from the email itself rather than the arguments passed on the command line.

I also edited the /etc/dspam/default.prefs to match the conf file.  I don't know if this is required.

To switch this on for a particular user you can use the dspam_admin command.  This needs to be run as the vmail user to avoid permission issues with the resulting generated files:

dspam_admin add preference user@domain optIn on

Now test.  If everything is working correctly you should now have a series of X-dspam-* headers in incoming email.

Step 3: Configure postfix and dovecot to send spam to the Spam folder

Now that postfix and dspam can identify and flag an email as spam, I want to move it to a dedicated Spam folder per user.  To do this I can use the sieve functionality in dovecot's LDA (Local Delivery Agent).

To kick things off we need to tell postfix how to deliver email through dovecot.  Add the following to master.cf:

# Dovecot local delivery agent - allows us to use sieve filters for spam
dovecot   unix  -       n       n       -       -       pipe
  flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -f ${sender} -d ${recipient}

Note that I'm using the vmail user again so that all mail files are kept under a common ownership.

To make postfix use the dovecot delivery agent add the following to main.cf:

# Dovecot for local delivery
dovecot_destination_recipient_limit = 1
virtual_transport = dovecot

These two changes gets email as far as dovecot, now we need to configure dovecot to look for the spam headers and redirect it to the Spam folder if present.  A bit of googling turned up the required steps:

In /etc/dovecot/dovecot.conf enable the sieve plugin for the LDA:

protocol lda {
  ...
  mail_plugins = sieve
  ...

In the same file configure a location for storing the global sieve filter files:

plugin {
  ...
  sieve_before = /etc/sieve/conf.d/before
  sieve_after = /etc/sieve/conf.d/after
  ...

Now we need to create an appropriate filter that will check for the spam headers.  I did this in /etc/sieve/conf.d/before/spam-folder.sieve (found here):

require ["regex", "fileinto", "imap4flags"];

# Catch mail tagged as Spam, except Spam retrained and delivered to the mailbox
if allof (header :regex "X-DSPAM-Result" "^(Spam|Virus|Bl[ao]cklisted)$",
          not header :contains "X-DSPAM-Reclassified" "Innocent") {

  # Mark as read
  setflag "\\Seen";

  # Move into the Junk folder
  fileinto "Spam";

  # Stop processing here
  stop;
}

The final step is to compile the script ready for dovecot to use (discovered here):

sievec /etc/sieve/conf.d/before/

To test this part of the configuration out ensure that the user has an IMAP folder called "Spam" at the top level of their account.  I'm not sure what happens if this doesn't exist, but I doubt it works.

Step 4: Configure dovecot_antispam to manage learning spam / ham

This step should be really straight forward.  Unfortunately, at least on Ubuntu 10.04 LTS, the dovecot_antispam package is badly maintained, being both compiled against the wrong version of dovecot and failing to support dspam.

Solving these problems however turned out to be mostly painless:

• apt-get install build-essential to get the basic development tools.
• In /usr/src do: apt-get source dovecot-antispam and apt-get build-dep dovecot-antispam
• In the source directory: cp debian/dovecot-antispam-config .config
• Edit the .config file and comment out the default backend and enable dspam-exec
• Build and install: make install

Having solved this packaging issue the rest of the antispam configuration is fairly easy.  Edit /etc/dovecot/dovecot.conf...

Enable the antispam plugin:

protocol imap {
  ...
  mail_plugins = antispam
  ...

And then configure it:

plugin {
  ...
  antispam_backend = dspam-exec
  antispam_signature = X-DSPAM-Signature
  antispam_signature_missing = error
  antispam_trash = trash;Trash;Deleted Items; Deleted Messages
  antispam_spam = Spam
  antispam_dspam_binary = /usr/bin/dspam
  antispam_dspam_args = --deliver;--user;%u
  ...

Now moving an email from the user's inbox to the Spam folder will trigger dspam to learn this email as spam, and vice versa.

Conclusion

So, was it worth it?  It's early days yet, but so far I'd say yes it was.  Being able to move an email to the Spam folder on my phone and knowing that the server will update it's filter to catch it in future makes going through spam much easier.  Logging in to the desktop email and only seeing messages that I've not already dealt with is great.

Please let me know if this page is of any help - and do send corrections and edits!