ExtensionDistributor

Revision as of 18:23, 20 August 2012

The ExtensionDistributor extension for MediaWiki is configured as follows. This is also managed by Puppet, using class misc::extension-distributor.

Apache for www.mediawiki.org invokes svn-invoker.php remotely using xinetd on fenari. The configuration file is in puppet, and is currently:

service svn_invoker
{
        type = UNLISTED
        disable = no
        socket_type = stream
        protocol = tcp
        port = 8430
        wait = no
        user = extdist
        instances = 1
        server = /usr/bin/php
        server_args = /home/wikipedia/common/wmf-config/extdist/svn-invoker.php
        only_from = 127.0.0.1 10.0.0.0/16 208.80.152.0/22
        log_type = FILE /var/log/svn-invoker.log
}

The extdist user has write access to a special MediaWiki working copy checked out in /mnt/upload6/private/ExtensionDistributor/mw-snapshot. This snapshot is created with svn-setup.sh. The remote service will update the svn working copy and return the last changed revision.

After apache retrieves the response from fenari, it creates a tarball of the extension using its NFS access to /mnt/upload5/private/ExtensionDistributor/mw-snapshot. It puts the result up for download on ms1, in /mnt/upload5/ext-dist.

svn-invoker.php will update existing extensions, but it won't add or delete extensions. So there is also an hourly cron job set up on fenari, in extdist's user crontab, which runs the equivalent of the following:

#!/bin/bash
PATH=/usr/bin
cd /home/wikipedia/ExtensionDistributor/mw-snapshot
for branch in trunk branches/*;do
	svn up $branch/extensions
done