Mail

Design decisions

Black box mail system, no user shell logins 

Few users would make good use of this anyway. Greatly simplifies network and host security, allows the use of some (non-critical) non-standardized extensions between software components for greater performance, interoperability and features because it doesn't have to support whatever shell users might install to access things directly.

IMAP only, no POP3 

IMAP has good client support nowadays, and for a large part solves the problem of having multiple clients. Also backups can be done centrally on the server side, and multiple folders with server side mail filtering might be supported.

Support for mail submission 

Through SMTP authentication we can allow our users to submit mails through the mail server, without them having to configure an outgoing mail server for whatever network they reside on. Can support multiple ports/protocols to evade firewalls.

SSL/TLS access only, no plain-text 

Although client support for this is not 100% yet, especially on mobile devices, the risks of using plain-text protocols is too high, especially with users visiting conferences and other locations with insecure wireless networks.

Quota support 

Although we can set quotas widely especially for those who need it, quotas should be implemented to protect the system.

Spam and virus filtering 

Is unfortunately necessary. Whether this should be global or per-user is to be determined.

Multi-domain support 

We have many domains, and the mail setup should be able to distinguish between domains where necessary.

Web access 

Some form of web-mail would be nice, although not critical at first and can be implemented at later stages.

Backups 

At least daily, with snapshots.

Cold failover 

Setting up a completely redundant system is probably a bit overkill at this stage, but we should make it easy and quick to set up a new mail system on other hardware in case of major breakage.

Documentation 

Although not all aspects of the involved software can be described of course, the specifics of the Wikimedia setup should be properly documented and HOWTOs for commonly needed tasks should be provided.

Software

MTA

Exim : Great flexibility, very configurable, reliable, secure.

IMAP server

Dovecot : Fast, secure, flexible.

Formats used

Maildir 

Safe, convenient format, moderately good performance, good software support.

Password and user databases 

To be determined. Important aspects: easy maintenance, good software support, replication support. Possible options:

• passwd-file - Simple field-separated text file, non-indexed. Supported by both Exim and Dovecot.
• sqlite - Indexed file format, powerful SQL queries, no full-blown RDBMS needed. Also easy to change to MySQL/PostgreSQL should that ever be necessary. Supported by both Exim and Dovecot.

Other data lookups 

either flat-file for small lists, or cdb for larger, indexed lookups.

Mailbox storage and mail delivery

Ext3 as file system 

ReiserFS may be a bit faster, but Ext3 is more reliable. Make sure directory indexes are enabled.

LVM 

For easy resizing, moving of data to other disks, and snapshots for backups.

RAID-1 

The new mail servers have hardware RAID controllers, we'll probably use them.

Dovecot's "deliver" as LDA 

Though Exim has a good internal Maildir "transport", the use of Dovecot's LDA allows it to use and update the Dovecot specific indexing for greater performance.

fcntl() and dot-file locking 

Greatest common divisors.

Maildir++ quotas 

Standard, reasonably fast.

Authentication

PLAIN authentication 

Universally supported for both IMAP and SMTP. Encrypted connections are used exclusively, so no elaborate hashing schemes needed.

SMD5 or SSHA password scheme 

Salted hashing.

SMTP authentication through either Exim's Dovecot authenticator, or using direct lookups 

Exim 4.64 has support for directly authenticating against Dovecot's authenticator processes, though this version is not in Ubuntu Feisty yet, so needs backporting. If direct lookups from Exim's authenticators are easy enough, use that. Also depends on the security model.

Layout

The mail setup consists of 2 general mail servers, plus a mailing lists server (lily) and an OTRS server. The two general mail servers are mchenry and sanger.

Wikimedia mail setup

One server (mchenry) acts as relay; it accepts mail connections from outside, checks them for spam, viruses and other policy checks, and then queues and/or forwards to the appropriate internal mail server. It also accepts mail destined for outside domains from internal servers, including the application servers.

The other server, sanger, is the IMAP server. It accepts mail from mchenry and delivers it to local user mailboxes. Outgoing mail from SMTP authenticated accounts are also accepted on this server, and forwarded to mchenry, where it's queued and sent out. Web mail and other supportive applications related to user mail accounts and their administration will also run on sanger.

Lily, the mailing lists server, also acts as a secondary MX and forwards non-mailing list mail to mchenry. In case of downtime of mchenry, it might be able to send partial (IMAP account) mail to sanger directly, depending on the added complexity of the configuration. During major hardware failure of sanger, mchenry (with identical hardware) should be able to be setup as IMAP server.

Configuration details

Mail relay

The current mail relay is mchenry.

OTRS

For OTRS, the mail relay queries the OTRS MySQL servers directly to check the existence of an OTRS mail address. This implies that newly created OTRS queues / mail addresses will start to work immediately and does not involve Wikimedia admins.

The MySQL servers are specified near the top of the Exim configuration file:

# MySQL lookups (OTRS)
hide mysql_servers = srv7.wikimedia.org/otrs/exim/password : \
                     srv8.wikimedia.org/otrs/exim/password

These servers will be queried in turn. If neither of these servers respond, or respond with an error, the mail will be deferred. A MySQL user account "exim" with (just) SELECT privileges on the system_address table of the otrs database needs to exist, which is accessible from the mail relay (mchenry.wikimedia.org).

The following router does the actual aliasing of the OTRS address to otrs@ticket.wikimedia.org, if the OTRS queue address exists in the database:

# Query the OTRS MySQL server(s) for the existence of the queue address
# $local_part@$domain, and alias to otrs@ticket.wikimedia.org if
# successful.

otrs:
       driver = redirect
       domains = +local_domains
       condition = ${lookup mysql{SELECT value0 FROM system_address WHERE value0='${quote_mysql:$local_part@$domain}'}{true}fail}
       data = otrs@ticket.wikimedia.org

Old Wikimedia email setup

• postfix, mailman, OTRS, spamassassin on goeje
  • mail.wikimedia.org <- service ip/name for mail
  • secure.wikimedia.org <- service ip/name for OTRS
  • ticket.wikimedia.org <- service name for OTRS, redirects to secure.wikimedia.org currently
• sendmail on most other machines (apaches)
  • internally in pmtpa they forward to smtp.pmtpa.wmnet, which relays to the external world.

How to make an email alias

On goeje:

vi /etc/postfix/aliases
newaliases
postfix reload

It takes up to 15 minutes until the secondary MXes accept mail for new aliases.

See also

• Mailing lists for the setup of the mailing lists server.

External documentation

• The Exim website and wiki
• The Dovecot website and wiki