Svn.wikimedia.org

Revision as of 18:23, 23 October 2010

Hardware

• formey.esams.wikimedia.org
  • SVN+SSH developer access
  • SVN over HTTP public read-only access
  • ViewVC

Related services

• svn-private - houses internal scripts and configurations that can't be publicly viewable

Dependencies:

• lists.wikimedia.org - MediaWiki-CVS commit notification list
• Offsite -- CIA update notifications

Used by:

• codereview-proxy.wikimedia.org - indirection tool to access SVN logs from MediaWiki for CodeReview extension

Server setup

Packages

apt-get install some packages:

 subversion          # SVN core bits
 libsvn-notify-perl  # For commit notification mails
 viewvc              # For repository view
 python-subversion   # Needed for viewvc
 python-pygments     # Used by viewvc for syntax highlighting
 php5-cli            # Used by svnusers.php
 doxygen             # For doc generation
 apache2             # Web server!
 libapache2-svn      # HTTP SVN module for apache
 libapache2-mod-php5 # For user.php

Custom software

 /usr/local/bin/sillyshell     # restricted shell; in tools/sillyshell in SVN
 /usr/local/bin/ciabot_svn.py  # IRC notifications for SVN, available at cia.cx
 /usr/local/viewvc/*           # ViewVC - viewvc.org

Doc generation

Handled by puppet.

# Set up an account to run the docs under
adduser --system mwdocs
su -s /bin/bash mwdocs
cd && svn co file:///svnroot/mediawiki/trunk/phase3

# make a log file for the cronjob
cd /var/log
touch mwdocs.log
chown mwdocs mwdocs.log 

# fix path in apache config
# add a cron job:
crontab -e
30 0 * * * (cd /home/mwdocs/phase3 && svn up && php maintenance/mwdocgen.php --all) >> /var/log/mwdocs.log 2>&1

SVN users setup

Add users

Run add-ldap-user on formey:

Usage: add-ldap-user [options] <username> <filename>
example: add-ldap-user laner http://ryandlane.com/static/pubkey.key

Options:
  -h, --help            show this help message and exit
  -s, --self            Use your credentials, rather than the proxyagent's
  -D BINDAS, --bindas=BINDAS
                        Specify user to bind as
  -m, --directorymanager
                        Use the Directory Manager's credentials, rather than
                        your own
  --shell=LOGINSHELL    The user's shell (default: /usr/bin/sillyshell)
  --gid=GIDNUMBER       The user's gid (default: 500)
  --uid=UIDNUMBER       The user's uid (default: next available uid)
  --home=HOMEDIRECTORY  The user's home directory (default /home/username)
  --cn=CN               The user's CN (default: firstname + lastname, or
                        username, if first/last not set)
  --firstname=GIVENNAME
                        The user's first name (default: username)
  --lastname=SN         The user's last name (default: username)

Modify users

Run modify-ldap-user on formey:

Usage: modify-ldap-user [options] <username> [--rename <newusername>]
example: modify-ldap-user --key=http://ryandlane.com/static/pubkey.key laner

Options:
  -h, --help            show this help message and exit
  -s, --self            Use your credentials, rather than the proxyagent's
  -D BINDAS, --bindas=BINDAS
                        Specify user to bind as
  -m, --directorymanager
                        Use the Directory Manager's credentials, rather than
                        your own
  --shell=LOGINSHELL    Set the user's shell
  --gid=GIDNUMBER       Set the user's gid
  --uid=UIDNUMBER       Set the user's uid
  --cn=CN               Set the user's CN
  --firstname=GIVENNAME
                        Set the user's first name
  --lastname=SN         Set the user's last name
  --replacekeys=REPLACEKEYLOCATION
                        Replaces all of the user's keys
  --addkeys=ADDKEYLOCATION
                        Adds keys to the user's entry
  --rename              Rename the user

Dumps

Currently handled by demon, runs as a cron on Saturdays at 1AM. Stored at /home/demon/svn/* (mounted from storage2 by Tomasz). Can do next incremental dump by running /home/demon/svn-dump-update. If you need to do a full dump (we do incremental by default) this takes about an hour or so to dump ~65000 revs. Available on the web at svn.wikimedia.org/dumps

• Needs more permanent home

Post-commit hooks

SVN Post commit hooks are used for commit e-mail notification, CIA and CodeReview, and can be found in /svnroot/svn tree/hooks/

CodeReview

# Hit CodeReview
wget -q -O /dev/null --post-data="" \
  "http://www.mediawiki.org/w/api.php?action=codeupdate&format=xml&repo=MediaWiki&rev=$REV" &