Wikimedia binaries
From Wikitech
(Difference between revisions)
| Line 1: | Line 1: | ||
| − | '''See [[Heterogeneous deployment]] | + | The following '''sync scripts''' are maintained in <code>/home/wikipedia/bin</code> on [[fenari]]. See also [[Heterogeneous deployment]]. |
| − | == | + | == Operate on a specific server == |
| − | + | === <code>sync-common</code> === | |
| − | + | {{shortcut|sync-common}} | |
| + | : ''Performs the action only on the server from which this command is executed'' | ||
| + | * syncs <code>/home/wikipedia/common</code> -> <code>/apache/common</code> | ||
== Operating on mediawiki-installation boxes via dsh == | == Operating on mediawiki-installation boxes via dsh == | ||
| − | These scripts all sync to: | + | These scripts all sync to: |
| − | * sync-common-all | + | * [[apache]]s |
| − | + | * [[image scalers]] | |
| − | + | * misc other hosts running apache: | |
| − | + | ** [[search]] servers | |
| − | + | ** live html snapshot browsers | |
| − | + | ** the [[secure]] server | |
| − | + | ** ''etc.'' | |
| − | + | ||
| − | + | === <code>sync-common-all</code> === | |
| − | * | + | {{shortcut|sync-common-all}} |
| − | + | * syncs <code>/home/wikipedia/common<code> -> <code>/apache/common</code> | |
| − | * | + | |
| + | === <code>sync-common-file</code> === | ||
| + | * syncs <code>/home/wikipedia/common/ (some file)</code> -> <code>/apache/common</code> | ||
| + | |||
| + | === <code>sync-dblist</code> === | ||
| + | * syncs <code>/home/wikipedia/common/*.dblist</code> -> <code>/apache/common</code> | ||
| + | |||
| + | === <code>sync-docroot</code> === | ||
| + | * syncs <code>/home/wikipedia/common/docroot</code> -> <code>/apache/common/docroot</code> | ||
| + | |||
| + | === <code>sync-file</code> === | ||
| + | * checks PHP syntax | ||
| + | * syncs <code>/home/wikipedia/common/wmf-deployment/ (some file)</code> -> <code>/apache/common/wmf-deployment</code> | ||
| + | |||
| + | === <code>scap</code> === | ||
| + | {{shortcut|scap}} | ||
| + | * checks PHP syntax | ||
| + | * rebuilds localization caches and texvc | ||
| + | * syncs <code>/home/wikipedia/common/wmf-deployment</code> -> <code>/apache/common/wmf-deployment</code> | ||
== Operating on apaches and image_scalers dsh groups == | == Operating on apaches and image_scalers dsh groups == | ||
| − | |||
| − | |||
| − | |||
| + | === <code>sync-apache</code> === | ||
| + | : '''Note:''' these files are outside of <code>/home/wikipedia/''common''</code>, and as such are '''not''' synced by the <code>sync-common</code>-like scripts. | ||
| + | * syncs <code>/home/wikipedia/conf/httpd</code> to <code>/apache/conf</code> | ||
== Mysterious or obsolete things? == | == Mysterious or obsolete things? == | ||
| Line 40: | Line 60: | ||
* sync-common-php-x | * sync-common-php-x | ||
| − | == No idea what these are == | + | === No idea what these are === |
* sync-mirror-dag.sh | * sync-mirror-dag.sh | ||
* sync-mirror-fedora.sh | * sync-mirror-fedora.sh | ||
| Line 47: | Line 67: | ||
* sync-cgibin | * sync-cgibin | ||
| − | == | + | == Examples == |
| − | # | + | === <code>sync-apache</code> example === |
| − | #(optionally) run <code>./sync-apache-simulated</code> | + | # On [[fenari]]:<br/><code>cd /home/wikipedia/bin</code> |
| − | # | + | # (optionally) run:<br/><code>./sync-apache-simulated</code><br/>This just has an "-n" added to the [[rsync]] commands. (<code>--dry-run | </code> show what would have been transferred) |
| − | # | + | # If that looks fine, actually sync:<br/><code>./sync-apache</code> |
| − | # | + | # Connect to a single random "srv" server, confirm the expected change has arrived here, and restart [[Apache]] to check your change did not break things. |
| − | + | ## (how to restart that apache?) | |
| − | + | # If that also looks fine, restart all Apaches:<br/><code>./apache-graceful-all</code> | |
| + | |||
| + | After this you might want to [[Purge URL|purge a URL]] from [[Squid]] cache. Also see [[Squids#Purge a given external (non WMF) url]]. | ||
| + | |||
| + | In most cases this is enough. However if the change involves more than a simple change to redirects, [[depooling]]/ repooling may be necessary. Make sure you find out whether this is needed ahead of time{{#ifexist: depooling || (while at it, start the [[depooling]] page and link to it, we need documentation!)}}. | ||
[[Category:Software]] | [[Category:Software]] | ||
[[Category:How-To]] | [[Category:How-To]] | ||
Revision as of 23:50, 9 August 2012
The following sync scripts are maintained in /home/wikipedia/bin on fenari. See also Heterogeneous deployment.
Contents |
Operate on a specific server
sync-common
shortcut: sync-common
- Performs the action only on the server from which this command is executed
- syncs
/home/wikipedia/common->/apache/common
Operating on mediawiki-installation boxes via dsh
These scripts all sync to:
- apaches
- image scalers
- misc other hosts running apache:
sync-common-all
shortcut: sync-common-all
- syncs
/home/wikipedia/common<code> -> <code>/apache/common
sync-common-file
- syncs
/home/wikipedia/common/ (some file)->/apache/common
sync-dblist
- syncs
/home/wikipedia/common/*.dblist->/apache/common
sync-docroot
- syncs
/home/wikipedia/common/docroot->/apache/common/docroot
sync-file
- checks PHP syntax
- syncs
/home/wikipedia/common/wmf-deployment/ (some file)->/apache/common/wmf-deployment
scap
shortcut: scap
- checks PHP syntax
- rebuilds localization caches and texvc
- syncs
/home/wikipedia/common/wmf-deployment->/apache/common/wmf-deployment
Operating on apaches and image_scalers dsh groups
sync-apache
- Note: these files are outside of
/home/wikipedia/common, and as such are not synced by thesync-common-like scripts.
- syncs
/home/wikipedia/conf/httpdto/apache/conf
Mysterious or obsolete things?
- scap14, scap14-1, scap14-2
- obsolete from MediaWiki 1.4 days
- scap15, scap15-1, scap15-1skins
- 'scap15' is actual script for 'scap' currently; probably could use cleanup
- sync-file13, sync-file14
- sync-file-log14 ?
- sync-to-seoul
- sync-common-allphp
- sync-common-clean
- sync-common-it
- sync-common-php
- sync-common-php-x
No idea what these are
- sync-mirror-dag.sh
- sync-mirror-fedora.sh
- sync-users-and-groups.sh
- syncout
- sync-cgibin
Examples
sync-apache example
- On fenari:
cd /home/wikipedia/bin - (optionally) run:
./sync-apache-simulated
This just has an "-n" added to the rsync commands. (--dry-run |show what would have been transferred) - If that looks fine, actually sync:
./sync-apache - Connect to a single random "srv" server, confirm the expected change has arrived here, and restart Apache to check your change did not break things.
- (how to restart that apache?)
- If that also looks fine, restart all Apaches:
./apache-graceful-all
After this you might want to purge a URL from Squid cache. Also see Squids#Purge a given external (non WMF) url.
In most cases this is enough. However if the change involves more than a simple change to redirects, depooling/ repooling may be necessary. Make sure you find out whether this is needed ahead of time (while at it, start the depooling page and link to it, we need documentation!).
