Svn.wikimedia.org

Revision as of 18:50, 29 December 2010

Hardware

• formey.esams.wikimedia.org
  • SVN+SSH developer access
  • SVN over HTTP public read-only access
  • ViewVC

Related services

• svn-private - houses internal scripts and configurations that can't be publicly viewable

Dependencies:

• lists.wikimedia.org - MediaWiki-CVS commit notification list
• Offsite -- CIA update notifications

Used by:

• codereview-proxy.wikimedia.org - indirection tool to access SVN logs from MediaWiki for CodeReview extension

Server setup

Packages

apt-get install some packages:

 subversion          # SVN core bits
 libsvn-notify-perl  # For commit notification mails
 viewvc              # For repository view
 python-subversion   # Needed for viewvc
 python-pygments     # Used by viewvc for syntax highlighting
 php5-cli            # Used by svnusers.php
 doxygen             # For doc generation
 apache2             # Web server!
 libapache2-svn      # HTTP SVN module for apache
 libapache2-mod-php5 # For user.php

Custom software

 /usr/local/bin/sillyshell     # restricted shell; in tools/sillyshell in SVN
 /usr/local/bin/ciabot_svn.py  # IRC notifications for SVN, available at cia.cx
 /usr/local/viewvc/*           # ViewVC - viewvc.org

Doc generation

Handled by puppet.

# Set up an account to run the docs under
adduser --system mwdocs
su -s /bin/bash mwdocs
cd && svn co file:///svnroot/mediawiki/trunk/phase3

# make a log file for the cronjob
cd /var/log
touch mwdocs.log
chown mwdocs mwdocs.log 

# fix path in apache config
# add a cron job:
crontab -e
30 0 * * * (cd /home/mwdocs/phase3 && svn up && php maintenance/mwdocgen.php --all) >> /var/log/mwdocs.log 2>&1

SVN users setup

Note: By default most LDAP commands try to run as the "scriptuser", which requires root privileges. In general most of these should run as root anyway. You'll need to use sudo to run these commands if you are not root.

Add users

Run add-ldap-user on formey:

Usage: add-ldap-user [options] <username> <key(s)>
example: add-ldap-user laner http://ryandlane.com/static/pubkey.key

Options:
  -h, --help            show this help message and exit
  -s, --self            Use your credentials, rather than the proxyagent's
  -D BINDAS, --bindas=BINDAS
                        Specify user to bind as
  -m, --directorymanager
                        Use the Directory Manager's credentials, rather than
                        your own
  --shell=LOGINSHELL    The user's shell (default: /usr/bin/sillyshell)
  --gid=GIDNUMBER       The user's gid (default: 500)
  --uid=UIDNUMBER       The user's uid (default: next available uid)
  --home=HOMEDIRECTORY  The user's home directory (default /home/username)
  --cn=CN               The user's CN (default: firstname + lastname, or
                        username, if first/last not set)
  --firstname=GIVENNAME
                        The user's first name (default: username)
  --lastname=SN         The user's last name (default: username)

Modify users

Run modify-ldap-user on formey:

Usage: modify-ldap-user [options] <username> [--rename <newusername>]
example: modify-ldap-user --replacekeys=http://ryandlane.com/static/pubkey.key laner

Options:
  -h, --help            show this help message and exit
  -s, --self            Use your credentials, rather than the proxyagent's
  -D BINDAS, --bindas=BINDAS
                        Specify user to bind as
  -m, --directorymanager
                        Use the Directory Manager's credentials, rather than
                        your own
  --shell=LOGINSHELL    Set the user's shell
  --gid=GIDNUMBER       Set the user's gid
  --uid=UIDNUMBER       Set the user's uid
  --cn=CN               Set the user's CN
  --firstname=GIVENNAME
                        Set the user's first name
  --lastname=SN         Set the user's last name
  --replacekeys=REPLACEKEYLOCATION
                        Replaces all of the user's keys
  --addkeys=ADDKEYLOCATION
                        Adds keys to the user's entry
  --rename              Rename the user

Dumps

Dumps are handled daily at 18:00 UTC and offloaded to Tridge via Amanda. Cron runs daily /usr/local/bin/svndump.php. Sunday performs a full dump, with daily incrementals. The previous day's dump is cleared before starting each day's dump.

Post-commit hooks

SVN Post commit hooks are used for commit e-mail notification, CIA and CodeReview, and can be found in /svnroot/svn tree/hooks/

CodeReview

# Hit CodeReview
wget -q -O /dev/null --post-data="" \
  "http://www.mediawiki.org/w/api.php?action=codeupdate&format=xml&repo=MediaWiki&rev=$REV" &