Bits and thoughts

#!/bin/bash is not rude

Self-hosting emails

Written by ⓘⓓⓔⓝⓣⓛⓤⓓ - -

Of the few services that where left in the hands of Google some were finally migrated to my own server :  
  • E mails
  • Calendars
  • Contacts
I had in mind to host them for autonomy and privacy reasons (and also because I find it fun to tinker with this ...).

One of my goal was to be able to access emails, contacts and calendar through my home computer (running Debian) and my smartphone (running Android).

The background OS is Debian Wheezy (testing .. not yet a 7.0 stable) as of November 2012.

Hosting emails

Somehow it's a tricky job to implement your own email server !! There are many steps, many tools to integrate and many configurations to tweak. The functionalities that I wanted to provide are :
  • Store my emails on my own server
  • Being able to securely read them from my home computer and my smart-phone
  • Avoid being a spam relay to the nasty bots while alowing to post emails without using my ISP gateway
  • Forward emails to my familly when they want to receive their emails with my domain name :
Of course I wanted to be able to migrate all my emails from Google to this new infrastructure. Over the course of the years I had stored something like 1.7 Gb of email

In a more technical fashion I took these decisions :

Store my emails on my own server

Store my emails in an easily readable format without a database back-end. Maildir format is good for this. I can even read my emails with the simple mail command through an SSH connection ... postfix is my tool of the trade.
  • Incoming messages are delivered to /home/<<user>>/Maildir. in a plain raw format. Each <<user>> has its own file
  • Creating email folders to organize them is made by using linux user's home directory : /home/<<user>>/Maildir/<<subdirectories>>

Being able to securely read them from my home computer and my smart-phone

Reading from different devices requires that the messages are downloaded on said devices but must still be available on the server. Two protocols are available IMAP and POP3. I chose IMAP (mainly because it first transfers email header so that you can sort them out before actually downloading the whole message when you want to read it)  and implemented it using dovecot.
  • Access to the mail boxes are secured with the unix user account's credentials
  • The authentication is made under TLS so that the credentials are encrypted before being sent over the network. I had to uncomment disable_plaintext_auth = yes to force TLS authentication in /etc/dovecot.conf.d/10-auth.conf
#### Authentication processes
### Disable LOGIN command and all other plaintext authentications unless
# SSL/TLS is used (LOGINDISABLED capability). Note that if the remote IP
# matches the local IP (ie. you're connecting from the same computer), the
# connection is considered secure and plaintext authentication is allowed.
disable_plaintext_auth = yes
I also added login authentication mode to /etc/dovecot.conf.d/10-auth.conf :
# Space separated list of wanted authentication mechanisms:
#   plain login digest-md5 cram-md5 ntlm rpa apop anonymous gssapi
#   otp skey gss-spnego# NOTE: See also disable_plaintext_auth
setting.auth_mechanisms = plain login

Avoid being a spam relay to the nasty bots while allowing to post emails without using my ISP gateway

This relies on :
  • Allowing any SMTP incoming emails that must be delivered to my domain. Delivering to is ok. Delivering to is forbidden.
  • Allowing any SMTP incoming emails to be forwarded to another domain ONLY IF the sender is authenticated.
  • Denying access when the message is received from an unqualified domain name server, unknown domain name server or of the sender's email address is not resolvable to an existing one. Examples :
    • if the server is instead of agoodserver.fromdomain.xx it is rejected.
    • if the domain agoodserver.fromdomain.xx does not exists it is rejected
    • if the sender's email is unknown it's rejected
    I have changed this rule a bit for I could not receive or forward messages coming from addresses like noreply@xxxx.xx. It was the case for my ISP invoices notification that where rejected.
  • Denying access to well known spammers through RBL lists
Authentication is made by SMTP protocol and it relies on dovecot :in /etc/postfix/
smtp      inet  n       -       -       -       -       smtpd
#smtp inet n - - - 1 postscreen
#smtpd pass - - - - - smtpd
#dnsblog unix - - - - 0 dnsblog
#tlsproxy unix - - - - 0 tlsproxy
submission inet n - - - - smtpd
-o syslog_name=postfix/submission
-o smtpd_tls_security_level=encrypt
-o smtpd_sasl_auth_enable=yes
-o smtpd_client_restrictions=permit_sasl_authenticated,reject
# -o milter_macro_daemon_name=ORIGINATING
smtps inet n - - - - smtpd
-o syslog_name=postfix/smtps
-o smtpd_tls_wrappermode=yes
-o smtpd_sasl_auth_enable=yes
-o smtpd_client_restrictions=permit_sasl_authenticated,reject

in /etc/postfix/

In bold are the modifications I made after realizing that some legitimate emails where rejected.
#smtpd_sender_restrictions = reject_unknown_sender_domain, 
# reject_unverified_sender
smtpd_recipient_restrictions = reject_invalid_hostname,
smtpd_helo_restrictions = reject_invalid_helo_hostname,
smtpd_client_restrictions = reject_rbl_client,
Delegating authentication to dovecot is made by configuring some other entries in /etc/postfix/
#Activate SASLsmtpd_sasl_auth_enable = yes
#Use Dovecotsmtpd_sasl_type = dovecotsmtpd_sasl_path = private/auth
# Add in header SASL authentication information.
smtpd_sasl_authenticated_header = yes
smtpd_sasl_path = private/auth means that postfix is going to rely on informations provided by dovecot in a file located in /var/spool/postfix/private/auth and configured in /etc/dovecot/conf.d/10-master.conf
service auth {  
# auth_socket_path points to this userdb socket by default. It's typically 
# used by dovecot-lda, doveadm, possibly imap process, etc. Users that have 
# full permissions to this socket are able to get a list of all usernames and 
# get the results of everyone's userdb lookups. 

# The default 0666 mode allows anyone to connect to the socket, but the 
# userdb lookups will succeed only if the userdb returns an "uid" field that 
# matches the caller process's UID. Also if caller's uid or gid matches the 
# socket's uid or gid the lookup succeeds. Anything else causes a failure. 

# To give the caller full permissions to lookup all users, set the mode to 
# something else than 0666 and Dovecot lets the kernel enforce the 
# permissions (e.g. 0777 allows everyone full permissions). 
unix_listener auth-userdb {
    mode = 0666
    user = postfix
    group = postfix
# Postfix smtp-auth 
# 3 following lines uncommented on 11/11/2012 
unix_listener /var/spool/postfix/private/auth
    mode = 0666

Forward emails to my familly when they want to receive their emails with my domain name :

This is configured through /etc/aliases file. If I want my sister's email to be delivered to her Gmail account : goes to then I configure this entry :
This goes live only when using this command line
postalias /etc/aliases

Results and improvements

As for now I have configured Evolution email client and my Android phone with the default Android "E-mail" application.What I'd like to improve is filtering of incoming messages. There are still some spam coming in. I'd like to get rid of them on the server side to avoid to configure my Evolution client and my Android client with the same filters (though I haven't found yet how to writer filter rules on the default Android app ...)

Next step is also to move my calendars and contacts out of Google grasp ... Ok It's already done in fact ... but I'll explain it in an other blog entry ...

Comments are closed.