Wikimedia binaries

Revision as of 23:15, 5 September 2012

The following executable binaries are maintained in version control (operations/puppet : files/misc/scripts) and copied/synced to servers (such as fenari) by puppet into /usr/local/bin.

bin

apache-graceful

Performs the action only on the one server from which the command is ran

apache-graceful-all

This script operates on all servers in the following dsh groups: apaches

• Runs apache-graceful locally on all apaches.

sync-apache

This script operates on all servers in the following dsh groups: apaches, image_scalers, snapshot, searchidx

Note: The conf files are outside of /home/wikipedia/common, and as such are not synced by the sync-common-like scripts.

• syncs /home/wikipedia/conf/httpd to /apache/conf

sync-common

Performs the action only on the one server from which the command is ran

• syncs /home/wikipedia/common -> /apache/common

sync-common-all

This is an all-script!

• syncs /home/wikipedia/common -> /apache/common

sync-common-file

This is an all-script!

• for a single directory or file
• syncs /home/wikipedia/common/ (some path) -> /apache/common

sync-dblist

This is an all-script!

• syncs /home/wikipedia/common/*.dblist -> /apache/common

sync-docroot

This is an all-script!

• syncs /home/wikipedia/common/docroot -> /apache/common/docroot

sync-dir

This is an all-script!

• for a single directory
• checks PHP syntax
• syncs /home/wikipedia/common/ (some dir) -> /apache/common with sync-common-file

sync-file

This is an all-script!

• for a single file
• checks PHP syntax
• syncs /home/wikipedia/common/ (some file) -> /apache/common with sync-common-file

sync-wikiversions

This script operates on all servers in the following dsh groups: mediawiki-installation

• syncs common/wikiversions.dat -> /apache/common-local/wikiversions.dat
• syncs common/wikiversions.cdb -> /apache/common-local/wikiversions.cdb

scap

This is an all-script!

• checks PHP syntax
• rebuilds localization caches and texvc
• syncs /home/wikipedia/common/wmf-deployment -> /apache/common/wmf-deployment

Unsorted

These are outdated, redundant, broken or otherwise undocumented:

• scap14, scap14-1, scap14-2
  • obsolete from MediaWiki 1.4 days
• scap15, scap15-1, scap15-1skins
  • 'scap15' is actual script for 'scap' currently; probably could use cleanup
• sync-file13, sync-file14
• sync-file-log14 ?
• sync-to-seoul
• sync-common-allphp
• sync-common-clean
• sync-common-it
• sync-common-php
• sync-common-php-x
• sync-mirror-dag.sh
• sync-mirror-fedora.sh
• sync-users-and-groups.sh
• syncout
• sync-cgibin

All-script

An all-script is a script that operates on mediawiki-installation boxes via dsh. These scripts performs the action on all servers. These servers to be exact:

• apaches
• image scalers
• misc other hosts running apache:
  • search servers
  • live html snapshot browsers
  • the secure server
  • etc.

Examples

sync-apache example

1. On fenari:
2. (optionally) run:
   sync-apache-simulated
   This just has an "-n" added to the rsync commands. (--dry-run | show what would have been transferred)
3. If that looks fine, actually sync:
   sync-apache
4. Connect to a single random "srv" server, confirm the expected change has arrived here, and restart Apache to check your change did not break things.
   1. (how to restart that apache?)
5. If that also looks fine, restart all Apaches:
   apache-graceful-all

After this you might want to purge a URL from Squid cache. Also see Squids#Purge a given external (non WMF) url.

In most cases this is enough. However if the change involves more than a simple change to redirects, depooling/ repooling may be necessary. Make sure you find out whether this is needed ahead of time (while at it, start the depooling page and link to it, we need documentation!).

See also

• Heterogeneous deployment.