OTRS

Revision as of 22:38, 29 January 2009

OTRS is installed on bart.wikimedia.org.

• Source is in /opt/otrs
• Config file is /opt/otrs/Kernel/Config.pm
• URL is https://secure.wikimedia.org/otrs/index.pl
• The root user/pass is in /home/wikipedia/doc/otrs
• Primary db is on db9, replica on db10. OTRS uses only the primary DB, they apparently don't support slaves.
• The codebase is patched, e.g. for supporting one-click-spam. The patches can be found at http://svn.wikimedia.org/svnroot/mediawiki/trunk/otrs

You no longer need to invoke a specific script or update config files to add email addresses to OTRS; mchenry will automatically see that the queue exists or has disappeared.

It is possible (due to negative caching at the secondary mail exchangers) that new addresses will take up to two hours to begin working.

To upgrade

1. Stop postfix. You don't want to have mail coming in while OTRS is broken.
2. Fetch new OTRS code
3. Decompress into /opt/otrs-X.Y.Z
4. Get the patches with svn export --force http://svn.wikimedia.org/svnroot/mediawiki/trunk/otrs /opt/otrs-cvs
5. Apply them with quilt push -a
6. Copy in Kernel/Config/Files/*, home, var/log, var/stats, var/sessions
7. Symlink the templates...
   • cd Kernel/Output/HTML && ln -s Standard OTRS
8. Set permissions on the dir, eg:
   • bin/SetPermissions.pl --secure --otrs-user=otrs --web-user=apache --otrs-group=otrs --web-group=apache /opt/otrs-X.Y.Z
9. change the /opt/otrs symlink to the new version
10. start postfix
11. send a mail to e.g. info-en and check that it shows up in OTRS

Mail setup

In the new OTRS setup, OTRS is installed on williams, and e-mail is sent and received through a special Exim instance on this server. Its configuration follows the lines of the setup described in Mail, but OTRS specific configuration is listed below.

Exim

The Exim setup for OTRS on williams is somewhat comparable to the one on sanger. It does not follow the concept of serving some specific domains, but rather accepts any mail from a select sets of hosts only (mchenry). It will then check the OTRS database for validity of the recipient addresses, and pipe the message to an OTRS command in case of success.

Locally submitted mail (e.g. by OTRS) is forwarded to mchenry.

The top of /etc/exim4/exim4.conf]] contains some OTRS specific macros:

OTRS_POSTMASTER=/opt/otrs/bin/PostMaster.pl
OTRS_HOME=/opt/otrs-home
OTRS_USER=otrs
OTRS_GROUP=otrs

# MySQL lookups (OTRS)
hide mysql_servers = srv7.wikimedia.org/otrs/exim/password : \
                     srv8.wikimedia.org/otrs/exim/password

The hostlist <tt>otrs_mail_from_hosts lists the MXes from which williams accepts mail:

hostlist otrs_mail_from_hosts = mchenry.wikimedia.org

OTRS must be able to set any sender address:

# Allow OTRS to use any sender address
untrusted_set_sender = *

ACLs

The mail server on williams only needs to accept mail from and send mail to the Wikimedia mail relay(s), so any other connections can be denied in the acl_smtp_connect acl:

acl_smtp_connect = acl_check_connect

acl_check_connect:
        # We only accept mail from our own mail relays
        require message = This server does not accept external mail
                hosts = <; 127.0.0.0/8 ; ::1 ; +wikimedia_nets

        accept

In the acl_smtp_rcpt ACL, mail from the hosts listed in otrs_mail_from_hosts is explicitly accepted without any further checks:

accept hosts = +otrs_mail_from_hosts : +relay_from_hosts

SpamAssassin scanning

Spam filtering is done using SpamAssassin and an Exim ACL which is run at the DATA phase during the SMTP connection. Should SpamAssassin fail for some reason, mail is let through.

acl_smtp_data = acl_check_data

acl_check_data:
        # Run a SpamAssassin check on any non-locally submitted messages.
        # If the message is estimated to be spam, we discard it, since
        # rejecting only moves the problem to mchenry.
        
        accept hosts = :

        discard message = This message scored $spam_score spam points.
                spam = otrs/defer_ok

        accept

OTRS mail routing

Mail destined for OTRS is served by a simple accept router otrs, which does a MySQL database query to determine the validity of the recipient address being routed, similar to the check done earlier by mchenry.

# Mail destined for OTRS

otrs:
        driver = accept
        condition = ${lookup mysql{SELECT value0 FROM system_address WHERE value0='${quote_mysql:$local_part@$domain}'}{true}fail}
        transport = otrs

On success, the message is handed over to the otrs pipe transport:

# OTRS pipe transport

otrs:
        driver = pipe
        command = OTRS_POSTMASTER
        current_directory = OTRS_HOME
        home_directory = OTRS_HOME
        user = OTRS_USER
        group = OTRS_GROUP
        freeze_exec_fail
        log_fail_output
        timeout = 1m
        timeout_defer

This transport pipes the full contents of the message to the command/path specified in the macro OTRS_POSTMASTER (defined at the top of the file). A current and home directory will be set as specified, and the command will be run as the otrs user and group. If the actual execution/invocation fails for some reason, the message will be frozen on the queue with a warning message sent to root. If the command invocation succeeds, but the return code is EX_TEMPFAIL (e.g. when OTRS cannot access the database), the message is deferred/queued, and will be retried later. Any output will be logged.

Outbound mail

Any mail destined for an address that is not an OTRS address, e.g. mail submitted by OTRS itself, will be forwarded to mchenry using the manualroute router:

# Send all mail not destined for the local machine via a set of
# mail relays ("smart hosts")

smart_route:
        driver = manualroute
        transport = remote_smtp
        route_list = *  mchenry.wikimedia.org:lists.wikimedia.org

SpamAssassin

williams runs its own SpamAssassin instance, so sa-learn can be used to train it from the OTRS Junk queue. The normal Ubuntu spamassassin package is used, with a few configuration modifications as listed below.

Multiple user profiles are not really used, SpamAssassin only reads global configuration settings. Training data is stored under the otrs home directory, /opt/otrs-home/.spamassassin/

/etc/default/spamassassin

Make sure spamd is enabled:

# Change to one to enable spamd
ENABLED=1

By default it runs as root, which is unnecessary. Since it's only used by OTRS, we can run it as the OTRS user. User preferences are disabled, spamd listens on the loopback interface only.

OPTIONS="--max-children 5 --nouser-config --listen-ip=127.0.0.1 -u otrs -g otrs"

Don't let spam filtering eat all resources:

# Set nice level of spamd
NICE="--nicelevel 10"

Automatically update SpamAssassin rules:

# Cronjob
# Set to anything but 0 to enable the cron job to automatically update
# spamassassin's rules on a nightly basis
CRON=1

/etc/spamassassin/local.cf

Allow SpamAssassin to trust these IP ranges:

trusted_networks 208.80.152.0/22 91.198.174.0/24 203.212.189.192/26