Bits and thoughts

#!/bin/bash is not rude


Fail2ban Analysis

Written by ⓘⓓⓔⓝⓣⓛⓤⓓ - -

Having restarted my fail2ban daemon on 2014-12-17 I was curious about the time distribution of bans and their original location.I crafted a small bash one-liner that gives me raw data : 
for ipadd in $(zgrep Ban fail2ban.log.* | awk '{print $NF}' | sort -u); \
do \
zgrep $ipadd fail2ban.log.* | awk -F':' '{print $2 $NF}' ; \
done | awk '{print $1" " $NF}' | \
grep -v '2014-12-16\|2014-12-15\|2014-12-14\|2014-12-13\|2014-12-12\|2014-12-11\|2014-12-10\|2014-12-09\|2014-12-08\|2014-12-07' \
| sort | while read data; \
do \
ipaddr2=$(echo $data | awk '{print $2}'); \
country=$(whois $ipaddr2 | egrep -i "^country:"| awk '{print $NF}' | sort -u); \
echo $data $country; \
It represents a total of 270 bans in almost three weeks.The ip adresses for which there is no country code is because the whois returned a Korean UTF-8 content that I couldn't parse. But they are all Koreans (I checked them manually).

IP address bans by day China comes first with 97 bans. It is followed by the United states of america (50 bans) and then Germany (30 bans).

Filtering SPAM on server with postfix, dovecot, and sieve

Written by ⓘⓓⓔⓝⓣⓛⓤⓓ - -

Identify spams is one step to fight against them. The next step is to be able to move them around independently of the device you are using to view your inbox.

Filtering spams by configuring the email client is sometimes possible (and sometimes not) but it requires to implement filtering rules on every client. Be it evolution on your computer or some other email client on your smartphone.

The solution is to filter them on the server when they are locally dispatched to recipients by the local delivery agent. (LDA)

In my basic setup of my email server I had postfix that was used as a mail transfer agent (MTA) and LDA. It happens that sieve is a mail filtering solution that comes bundled with dovecot. The setup is then made to :

  • use dovecot as a local delivery agent
  • configure sieve to move spam emails to 'Junk' folder

Of course the pre-required step is that spamassassin has already been set up to identify what emails are spams and what emails are not.

Base configuration

  • Debian Wheezy : 7.4
  • postfix : 2.9.6
  • dovecot with sieve embedded : 2.1.7

Use dovecot as LDA

Modify /etc/postfix/ file to change mailbox_command to dovecot deliver


Configure dovecot to enable sieve

The first step is to configure where sieve should be reading the rules configuration files. This is done by modifying /etc/dovecot/conf.d/90-sieve.conf and adapting sieve_default location. The result is that the behavior will be the same for all users for which a local mail delivery is made. If your MTA is configured to redirect emails to external mailboxes then the spams emails won't be moved to a junk folder.

sieve_default = /etc/dovecot/default.sieve

Enable dovecot user to read the file :

chgrp dovecot /etc/dovecot/conf.d/90-sieve.conf

Next step is to enable sieve plugin for LDA (local delivery agent) in /etc/dovecot/conf.d/15-lda.conf declare sieve as a plugin :

mail_plugins = sieve

Now how can sieve move spams to 'Junk' folders ? This is done by configuring the /etc/dovecot/default.sieve file with this content :

require ["fileinto"];
# Move spam to Junk
folderif header :contains "X-spam-flag" ["YES"] {
  fileinto "Spam";

This file must be binary compiled by sieve compiler

cd /etc/dovecot
sievec default.sieve

A new file default.svbin is created and it must be readable by dovecot user

chgrp dovecot /etc/dovecot/default.svbin

There are some permission problems with /var/mail/<<user>> INBOX with dovecot on Debian. The email can be delivered directly to ~/Maildir with a configuration of /etc/dovecot/10-mail.conf. 

#mail_location = mbox:~/mail:INBOX=/var/mail/%u
mail_location = maildir:~/Maildir

You can now restart dovecot and test a specially crafted spam email ;)

service dovecot restart
mail -s "Product for you" an-account-existing@your-server.tld

The following content was flagged as spam by spamassassin :

Online Drugstore can have your order of discounted Viagra shipped to you for only 5 minutes of your time!!!
No Prior Prescriptions Needed  
-Licensed U.S. Physicians are ready to fill your order  
-Guaranteed Lowest Prices Available  
-Discreet Mailing directly to your home or office

Just visit and enjoy the good life today!!!

Have a watch on your logs

Written by ⓘⓓⓔⓝⓣⓛⓤⓓ - -

Want to know what happened on your server ? Logwatch is for you.It sums up and aggregates some information from your log files and you can have it sent by email to you.

Context and installation

The server I'm running is a Debian Wheezy.
apt-get install logwatch
I wanted it to send an email to a specific local user and I wanted the email to contains as much detail as possible. I wrote to etc/logwatch/conf/override.conf
# cat >/etc/logwatch/conf/override.conf <<EOF
logwatch: MailTo =
logwatch: Output = mail
logwatch: Detail = high

The result is an email sent with every important details.

Filtering spam emails with Spamassassin

Written by ⓘⓓⓔⓝⓣⓛⓤⓓ - -

Last time I explained how I setup my own email server . One of the possible improvement was to be able to filter spam on the server-side rather than relying on the client-side configuration. I configured this using spamassassin.The technical background is Debian Wheezy (testing version as it is not yet stable as for now).You will see in another article that we can go further by adding some filtering rules on the server as well ...But for now let's see how to set this up.

Installing spamassassin

Where talking about Debian here ... :
apt-get install spamassassin
One configuration step is to enable spamassassin in its configuration file /etc/default/spamassassin
# sed -i "s/ENABLED=0/ENABLED=1/g" /etc/default/spamassassin
And then the spamd service that must be launched with :
# service spamassassin start
You can check that the spamd daemon is listening to inputs on loopback address :
# netstat -ntpl | grep spamdtcp  
0  0*       LISTEN      20892/
The version installed is :
# spamassassin --version
SpamAssassin version 3.3.2  running on Perl version 5.14.2

Filtering SMTP content through spamassassin

I'll have to configure a service for smtp in /etc/postfix/ by adding a -o option :
smtp inet  n       -       -       -       -            smtpd
  -o content_filter=spamassassin
submission inet n       -       -       -       -       smtpd
   -o smtpd_tls_security_level=encrypt
   -o smtpd_sasl_auth_enable=yes
   -o smtpd_client_restrictions=permit_sasl_authenticated,reject
   -o content_filter=spamassassin
smtps     inet  n       -       -       -       -       smtpd
   -o smtpd_tls_wrappermode=yes
   -o smtpd_sasl_auth_enable=yes
   -o smtpd_client_restrictions=permit_sasl_authenticated,reject
   -o content_filter=spamassassin
And configure what spamassassin stands for in /etc/postfix/ by adding it at the end of the file :
spamassassin unix -     n       n       -       -       pipe  user=debian-spamd argv=/usr/bin/spamc -f -e /usr/sbin/sendmail \
-oi -f ${sender} ${recipient}
NB : I edited the line to have it fit in the <pre> section but I guess that it is better if the line starting with "user=" and ending with "${recipient}" is a one-liner.debian-spamd is the user created by the apt-get install.
# getent passwd | grep debian-spamd


You may find some interesting logs in /var/log/mail.log
Nov 17 19:30:44  postfix/pipe[32112]: 596F361DA4: to=<>, relay=spamassassin, delay=1.1, delays=0.76/0.02/0/0.29, dsn=2.0.0, status=sent (delivered via spamassassin service)
Nov 17 19:35:14  postfix/pipe[5098]: C8A6761D6F: to=<>, relay=spamassassin, delay=519, delays=518/0.01/0/0.63, dsn=2.0.0, status=sent (delivered via spamassassin service)

Updating filters

Once in a while, or through a cron entry, you can update filters with this command  (man pages are well written) :
sa-update && service spamassassin reload

Learning behaviour

Look at man pages for sa-learn to improve spamassassin bayesian filters efficiency.