Docs/SysAdmin/Server/Mail/FetchmailSA

From Mandriva Community Wiki

Main Page < Docs/SysAdmin/Server/Mail/FetchmailSA

Setting up Fetchmail, Procmail & Spamassassin

What I wanted to do was retrieve my mail from two different pop accounts, pass it all through a spam filter and have the Subjects re-written, store it locally in a single file, then have Evolution check the local mail file and use its filters to sort into Evolutions mail folders.

There are of course many configurations you could have using these 3 programs but this will concentrate specifically on accomplishing the task outlined above.

This all assumes you have Fetchmail, Procmail and Spamassassin already installed.

1 Fetchmail
2 Procmail
3 Spamassassin
4 Evolution
5 References

[edit] Fetchmail

Set up a config file, either $HOME/.fetchmailrc or /etc/fetchmailrc

Here's my config file for fetchmail:

# Set the last resort mail recipient
set postmaster "postmaster"
# Send mail to postmaster rather than sender
set no bouncemail
# Send spam bounces
set no spambounce
# How often to run the daemon (seconds).
set daemon 60 
# Set the mail server, IP (via 'IP' is optional) and protocol
poll pop.domain1.net via 10.x.x.x
  with proto POP3
# user information.
        user 'remoteUserName' there with password 'xxxxxx'
        is 'localUserName' here options fetchall warnings 3600
# start over for next mail account
poll mail.domain2.com via 10.10.x.x
  with proto POP3
        user 'remoteUserName' there with password 'xxxxxx'
        is 'localUserName' here options fetchall warnings 3600

This is all well documented in the the fetchmail man page (man fetchmail then /Keyword/Option Summary).

Fetchmail has a unique way of handling the polling section with syntax that almost reads like a full sentence. Personally I'm not fond of it and find it more tedious to deal with and would rather just have something like:

REMOTE_USER=
LOCAL_USER=
REMOTE_PASSWORD=

etc...

But perhaps someone not used to dealing with config files will appreciate it.

The fetchmail config stores and sends a plain text password in it. YUK! So what I did was utilize my remote servers APOP abilities and use the 'secret value' in place of the password. This is documented at the fetchmail site in the FAQ: http://www.catb.org/~esr/fetchmail/fetchmail-FAQ.html#G10

There are of course several security models you can implement but this was a quick, dirty method that my remote mail server already supported.

So, there's a TON of stuff fetchmail can do and it's well documented in the man page but this will cover the basics of this specific config.

When it's time to run the daemon just use the service command as root to start it up:

$ service fetchmail start

[edit] Procmail

$HOME/.procmailrc or /etc/procmailrc

SHELL=/bin/sh
PATH=/usr/local/bin:/usr/bin:/bin
# uncomment to have appended to each log entry,
# can contain any string you want
## LOG="
## "
# turn off verbose logging, set to yes when trying new
# recipies or debugging
VERBOSE=no
MAILDIR=/var/spool/mail
LOGDIR=/var/log/mail
# By default procmail logs each transaction on exit,
# turn that off here if you do not want your log to fill up
# with this data
LOGABSTRACT=no
LOGFILE=$LOGDIR/procmail.log

# If message is larger then 256kb re-write Subject
# since it's not being run through SA
:0fhw
* > 256000
*^Subject:\/.*
| formail -I "Subject: {* -BIG- *} $MATCH"

:0fw: /var/run/spam.lock
* < 256000
| spamc -f -u $LOGNAME

# Work around procmail bug: any output on stderr will cause the "F" in "From"
# to be dropped.  This will re-add it.
:0 H
* ! ^From[ ]
* ^rom[ ]
{
  LOG="*** Dropped F off From_ header! Fixing up. "

  :0 fhw
  | sed -e 's/^rom /From /'
}

Procmail is of course capable of very extensive mail filtering none of which we are using here. Only thing the procmail recipes above are doing is re-writing the subject line of messages bigger than 256kb so they are "marked" with "-BIG-" and we know they did not get run through spamassasin then the next recipe sends everything under 256kb to spamassassin.

I use spamc -u otherwise you have to use DROPPRIV=yes before the spamassassin recipe and will not be able to write a lock file in /var/run as a normal user. (Just my own personal pref. you could always write the lock file elsewhere like $HOME/tmp).

[edit] Spamassassin

$HOME/.spamassassin/user_prefs

# How many hits before a message is considered spam.
required_hits           5.0

# Whether to change the subject of suspected spam
rewrite_subject         1

# Text to prepend to subject if rewrite_subject is used
subject_tag             *****SPAM*****

# Encapsulate spam in an attachment
report_safe             0

# Use terse version of the spam report
use_terse_report        0

# Enable the Bayes system
use_bayes               1

# Enable Bayes auto-learning
auto_learn              1

# Enable or disable network checks
skip_rbl_checks         0
use_razor2              1
use_dcc                 1
use_pyzor               1

# Mail using languages used in these country codes will not be marked
# as being possibly spam in a foreign language.
# - english 
ok_languages            en 

# Mail using locales used in these country codes will not be marked
# as being possibly spam in a foreign language.
ok_locales              en

There is a nice config file generator at http://www.yrex.com/spam/spamconfig.php that will help you generate this.

In addition I set up a crontab to run sa-learn on the spam files to keep the bayesian system up-to-date.

[edit] Evolution

Under tools > settings > mail accounts add or edit an existing account, on the receiving mail tab choose Local Delivery from the Server Type drop down then specify the path to your mail (Mbox) file.

Then under tools > filters add a new filter that moves all mail with a subject line of ****Spam**** to a folder.

Of course, spamassassin writes all sorts of additional headers and you can do much more sophisticated filtering based on the scores and different headers but this was sufficient for my needs. You could additionally have procmail do all the filtering and use Evolution as an IMAP client as well, but, again, I'm just outlining what I did here..

While it was a bit more work to get up and running, I found this to be a much better solution then setting up an Evolution mail filter that piped everything to spamc -c which created a TON of overhead and made mail downloading feel like I was on a 2400 baud modem again. The 600+ spam emails I woke up to everyday required about a 10 minute processing time, no thanks!