Wikimedia binaries

From Wikitech
(Difference between revisions)
Jump to: navigation, search
m (<1movedto2>)
(update based on source code)
Line 1: Line 1:
−
The following '''sync scripts''' are maintained in <code>/home/wikipedia/bin</code> on [[fenari]]. See also [[Heterogeneous deployment]].
+
{{TOC|align=right}}
 +
The following executable '''binaries''' are maintained in <code>/home/wikipedia/bin</code> on [[fenari]].
 +
 
 +
== bin ==
 +
=== <code>apache-graceful</code> ===
 +
: ''Performs the action '''only on the one server''' from which the command is ran''
 +
 
 +
=== <code>apache-graceful-all</code> ===
 +
: ''This script operates on all servers in the following dsh groups: '''apaches'''
 +
* Runs <code>apache-graceful</code> locally on all apaches.
  
−
== Operate on a specific server ==
 
 
=== <code>sync-common</code> ===
 
=== <code>sync-common</code> ===
−
{{shortcut|sync-common}}
+
: ''Performs the action '''only on the one server''' from which the command is ran''
−
: ''Performs the action only on the server from which this command is executed''
+
 
* syncs <code>/home/wikipedia/common</code> -> <code>/apache/common</code>
 
* syncs <code>/home/wikipedia/common</code> -> <code>/apache/common</code>
−
 
−
== Operating on mediawiki-installation boxes via dsh ==
 
−
''These scripts performs the action on '''all''' servers.''
 
−
These servers to be exact:
 
−
* [[apache]]s
 
−
* [[image scalers]]
 
−
* misc other hosts running apache:
 
−
** [[search]] servers
 
−
** live html snapshot browsers
 
−
** the [[secure]] server
 
−
** ''etc.''
 
  
 
=== <code>sync-common-all</code> ===
 
=== <code>sync-common-all</code> ===
−
{{shortcut|sync-common-all}}
+
: ''This is an [[#All-script|all-script]]!''
 
* syncs <code>/home/wikipedia/common</code> -> <code>/apache/common</code>
 
* syncs <code>/home/wikipedia/common</code> -> <code>/apache/common</code>
  
 
=== <code>sync-common-file</code> ===
 
=== <code>sync-common-file</code> ===
 +
: ''This is an [[#All-script|all-script]]!''
 
* syncs <code>/home/wikipedia/common/ (some file)</code> -> <code>/apache/common</code>
 
* syncs <code>/home/wikipedia/common/ (some file)</code> -> <code>/apache/common</code>
  
 
=== <code>sync-dblist</code> ===
 
=== <code>sync-dblist</code> ===
 +
: ''This is an [[#All-script|all-script]]!''
 
* syncs <code>/home/wikipedia/common/*.dblist</code> -> <code>/apache/common</code>
 
* syncs <code>/home/wikipedia/common/*.dblist</code> -> <code>/apache/common</code>
  
 
=== <code>sync-docroot</code> ===
 
=== <code>sync-docroot</code> ===
 +
: ''This is an [[#All-script|all-script]]!''
 
* syncs <code>/home/wikipedia/common/docroot</code> -> <code>/apache/common/docroot</code>
 
* syncs <code>/home/wikipedia/common/docroot</code> -> <code>/apache/common/docroot</code>
  
 
=== <code>sync-file</code> ===
 
=== <code>sync-file</code> ===
 +
: ''This is an [[#All-script|all-script]]!''
 
* checks PHP syntax
 
* checks PHP syntax
 
* syncs <code>/home/wikipedia/common/wmf-deployment/ (some file)</code> -> <code>/apache/common/wmf-deployment</code>
 
* syncs <code>/home/wikipedia/common/wmf-deployment/ (some file)</code> -> <code>/apache/common/wmf-deployment</code>
  
 
=== <code>scap</code> ===
 
=== <code>scap</code> ===
 +
: ''This is an [[#All-script|all-script]]!''
 
{{shortcut|scap}}
 
{{shortcut|scap}}
 
* checks PHP syntax
 
* checks PHP syntax
 
* rebuilds localization caches and texvc
 
* rebuilds localization caches and texvc
 
* syncs <code>/home/wikipedia/common/wmf-deployment</code> -> <code>/apache/common/wmf-deployment</code>
 
* syncs <code>/home/wikipedia/common/wmf-deployment</code> -> <code>/apache/common/wmf-deployment</code>
−
 
−
== Operating on apaches and image_scalers dsh groups ==
 
  
 
=== <code>sync-apache</code> ===
 
=== <code>sync-apache</code> ===
−
: '''Note:''' these files are outside of <code>/home/wikipedia/''common''</code>, and as such are '''not''' synced by the <code>sync-common</code>-like scripts.
+
: ''This script operates on all servers in the following dsh groups: '''apaches, image_scalers, snapshot, searchidx'''
 +
: '''Note:''' The conf files are outside of <code>/home/wikipedia/''common''</code>, and as such are '''not''' synced by the <code>sync-common</code>-like scripts.
 
* syncs <code>/home/wikipedia/conf/httpd</code> to <code>/apache/conf</code>
 
* syncs <code>/home/wikipedia/conf/httpd</code> to <code>/apache/conf</code>
  
−
== Mysterious or obsolete things? ==
+
== Unsorted ==
 +
: These are outdated, redundant, broken or otherwise undocumented:
 +
 
* scap14, scap14-1, scap14-2
 
* scap14, scap14-1, scap14-2
 
** obsolete from MediaWiki 1.4 days
 
** obsolete from MediaWiki 1.4 days
Line 60: Line 62:
 
* sync-common-php
 
* sync-common-php
 
* sync-common-php-x
 
* sync-common-php-x
−
 
−
=== No idea what these are ===
 
 
* sync-mirror-dag.sh
 
* sync-mirror-dag.sh
 
* sync-mirror-fedora.sh
 
* sync-mirror-fedora.sh
Line 67: Line 67:
 
* syncout
 
* syncout
 
* sync-cgibin
 
* sync-cgibin
 +
 +
== All-script ==
 +
An '''all-script''' is a script that operates on mediawiki-installation boxes via dsh. ''These scripts performs the action on '''all''' servers.''
 +
These servers to be exact:
 +
* [[apache]]s
 +
* [[image scalers]]
 +
* misc other hosts running apache:
 +
** [[search]] servers
 +
** live html snapshot browsers
 +
** the [[secure]] server
 +
** ''etc.''
  
 
== Examples ==
 
== Examples ==
Line 81: Line 92:
 
In most cases this is enough. However if the change involves more than a simple change to redirects, [[depooling]]/ repooling may be necessary. Make sure you find out whether this is needed ahead of time{{#ifexist: depooling ||&nbsp;(while at it, start the [[depooling]] page and link to it, we need documentation!)}}.
 
In most cases this is enough. However if the change involves more than a simple change to redirects, [[depooling]]/ repooling may be necessary. Make sure you find out whether this is needed ahead of time{{#ifexist: depooling ||&nbsp;(while at it, start the [[depooling]] page and link to it, we need documentation!)}}.
  
 +
== See also ==
 +
* [[Heterogeneous deployment]].
 
[[Category:Software]]
 
[[Category:Software]]
 
[[Category:How-To]]
 
[[Category:How-To]]

Revision as of 01:47, 10 August 2012

Contents

The following executable binaries are maintained in /home/wikipedia/bin on fenari.

bin

apache-graceful

Performs the action only on the one server from which the command is ran

apache-graceful-all

This script operates on all servers in the following dsh groups: apaches
  • Runs apache-graceful locally on all apaches.

sync-common

Performs the action only on the one server from which the command is ran
  • syncs /home/wikipedia/common -> /apache/common

sync-common-all

This is an all-script!
  • syncs /home/wikipedia/common -> /apache/common

sync-common-file

This is an all-script!
  • syncs /home/wikipedia/common/ (some file) -> /apache/common

sync-dblist

This is an all-script!
  • syncs /home/wikipedia/common/*.dblist -> /apache/common

sync-docroot

This is an all-script!
  • syncs /home/wikipedia/common/docroot -> /apache/common/docroot

sync-file

This is an all-script!
  • checks PHP syntax
  • syncs /home/wikipedia/common/wmf-deployment/ (some file) -> /apache/common/wmf-deployment

scap

This is an all-script!
shortcut: scap
  • checks PHP syntax
  • rebuilds localization caches and texvc
  • syncs /home/wikipedia/common/wmf-deployment -> /apache/common/wmf-deployment

sync-apache

This script operates on all servers in the following dsh groups: apaches, image_scalers, snapshot, searchidx
Note: The conf files are outside of /home/wikipedia/common, and as such are not synced by the sync-common-like scripts.
  • syncs /home/wikipedia/conf/httpd to /apache/conf

Unsorted

These are outdated, redundant, broken or otherwise undocumented:
  • scap14, scap14-1, scap14-2
    • obsolete from MediaWiki 1.4 days
  • scap15, scap15-1, scap15-1skins
    • 'scap15' is actual script for 'scap' currently; probably could use cleanup
  • sync-file13, sync-file14
  • sync-file-log14 ?
  • sync-to-seoul
  • sync-common-allphp
  • sync-common-clean
  • sync-common-it
  • sync-common-php
  • sync-common-php-x
  • sync-mirror-dag.sh
  • sync-mirror-fedora.sh
  • sync-users-and-groups.sh
  • syncout
  • sync-cgibin

All-script

An all-script is a script that operates on mediawiki-installation boxes via dsh. These scripts performs the action on all servers. These servers to be exact:

Examples

sync-apache example

  1. On fenari:
    cd /home/wikipedia/bin
  2. (optionally) run:
    ./sync-apache-simulated
    This just has an "-n" added to the rsync commands. (--dry-run | show what would have been transferred)
  3. If that looks fine, actually sync:
    ./sync-apache
  4. Connect to a single random "srv" server, confirm the expected change has arrived here, and restart Apache to check your change did not break things.
    1. (how to restart that apache?)
  5. If that also looks fine, restart all Apaches:
    ./apache-graceful-all

After this you might want to purge a URL from Squid cache. Also see Squids#Purge a given external (non WMF) url.

In most cases this is enough. However if the change involves more than a simple change to redirects, depooling/ repooling may be necessary. Make sure you find out whether this is needed ahead of time (while at it, start the depooling page and link to it, we need documentation!).

See also

Retrieved from "http://wikitech-old.wikimedia.org/index.php?title=Wikimedia_binaries&oldid=50091"
Personal tools
Namespaces

Variants
Actions
Navigation
Ops documentation
Wiki
Toolbox