Wikimedia binaries

Revision as of 23:55, 9 August 2012

The following sync scripts are maintained in /home/wikipedia/bin on fenari. See also Heterogeneous deployment.

Operate on a specific server

sync-common

Performs the action only on the server from which this command is executed

• syncs /home/wikipedia/common -> /apache/common

Operating on mediawiki-installation boxes via dsh

These scripts sync to all:

• apaches
• image scalers
• misc other hosts running apache:
  • search servers
  • live html snapshot browsers
  • the secure server
  • etc.

sync-common-all

• syncs /home/wikipedia/common -> /apache/common

sync-common-file

• syncs /home/wikipedia/common/ (some file) -> /apache/common

sync-dblist

• syncs /home/wikipedia/common/*.dblist -> /apache/common

sync-docroot

• syncs /home/wikipedia/common/docroot -> /apache/common/docroot

sync-file

• checks PHP syntax
• syncs /home/wikipedia/common/wmf-deployment/ (some file) -> /apache/common/wmf-deployment

scap

• checks PHP syntax
• rebuilds localization caches and texvc
• syncs /home/wikipedia/common/wmf-deployment -> /apache/common/wmf-deployment

Operating on apaches and image_scalers dsh groups

sync-apache

Note: these files are outside of /home/wikipedia/common, and as such are not synced by the sync-common-like scripts.

• syncs /home/wikipedia/conf/httpd to /apache/conf

Mysterious or obsolete things?

• scap14, scap14-1, scap14-2
  • obsolete from MediaWiki 1.4 days
• scap15, scap15-1, scap15-1skins
  • 'scap15' is actual script for 'scap' currently; probably could use cleanup
• sync-file13, sync-file14
• sync-file-log14 ?
• sync-to-seoul
• sync-common-allphp
• sync-common-clean
• sync-common-it
• sync-common-php
• sync-common-php-x

No idea what these are

• sync-mirror-dag.sh
• sync-mirror-fedora.sh
• sync-users-and-groups.sh
• syncout
• sync-cgibin

Examples

sync-apache example

1. On fenari:
   cd /home/wikipedia/bin
2. (optionally) run:
   ./sync-apache-simulated
   This just has an "-n" added to the rsync commands. (--dry-run | show what would have been transferred)
3. If that looks fine, actually sync:
   ./sync-apache
4. Connect to a single random "srv" server, confirm the expected change has arrived here, and restart Apache to check your change did not break things.
   1. (how to restart that apache?)
5. If that also looks fine, restart all Apaches:
   ./apache-graceful-all

After this you might want to purge a URL from Squid cache. Also see Squids#Purge a given external (non WMF) url.

In most cases this is enough. However if the change involves more than a simple change to redirects, depooling/ repooling may be necessary. Make sure you find out whether this is needed ahead of time (while at it, start the depooling page and link to it, we need documentation!).