Svn.wikimedia.org

From Wikitech
(Difference between revisions)
Jump to: navigation, search
(Packages)
Line 40: Line 40:
 
   /usr/local/viewvc/*          # ViewVC - viewvc.org
 
   /usr/local/viewvc/*          # ViewVC - viewvc.org
  
=== Viewvc ===
+
=== Doc generation ===
  
Using 1.0.x branch from their SVN; install it and set up the .conf file.
+
Handled by puppet.
 
+
TODO: clean up the conf file and put it in SVN or this wiki.
+
 
+
# Install viewvc!
+
svn co http://viewvc.tigris.org/svn/viewvc/branches/1.0.x/ viewvc-1.0
+
./viewvc-install
+
# Installation path [/usr/local/viewvc]:
+
# update viewvc path in apache config file
+
 
+
==== Patch for PHP view ====
+
 
+
To get view links instead of download links for PHP files in the repo:
+
 
+
<pre>
+
Index: lib/viewvc.py
+
===================================================================
+
--- lib/viewvc.py      (revision 1698)
+
+++ lib/viewvc.py      (working copy)
+
@@ -904,7 +904,7 @@
+
  return mime_type and mime_type in ('image/gif', 'image/jpeg', 'image/png')
+
+
def is_text(mime_type):
+
-  return not mime_type or mime_type[:5] == 'text/'
+
+  return not mime_type or mime_type[:5] == 'text/' or mime_type in ('application/x-httpd-php')
+
+
def is_plain_text(mime_type):
+
  return not mime_type or mime_type == 'text/plain'
+
</pre>
+
 
+
=== Apache ===
+
 
+
Stick conf file in /etc/apache2/sites-available and link it into sites-enabled, replacing the default config.
+
 
+
TODO: clean up the conf file and put it in SVN or this wiki.
+
 
+
 
+
=== Doc generation ===
+
  
 
  # Set up an account to run the docs under
 
  # Set up an account to run the docs under
Line 99: Line 62:
 
=== SVN users setup ===
 
=== SVN users setup ===
  
# The Easy way:
+
Run add-ldap-user on formey:
run add-svn-user on mayflower:  
+
   
  root@mayflower:~# add-svn-user
+
  Usage: add-ldap-user [options] <username> <filename>
  Usage: add-svn-user <username> <keyfile>
+
  example: add-ldap-user laner http://ryandlane.com/static/pubkey.key
  The keyfile should contain the user's SSH public key, and will be copied
+
to their ~/.ssh/authorized_keys.
+
 
+
# Setup...
+
addgroup svn
+
 
+
# When adding users, put in restricted setup:
+
/usr/sbin/useradd -G svn -s /usr/local/bin/sillyshell "$name"
+
 
+
# Make cache directory for users.php
+
mkdir /var/cache/svnusers
+
chown www-data /var/cache/svnusers
+
  
  # Change path in users.php from /usr/local/bin/svn to /usr/bin/svn
+
  Options:
# Check out thingy
+
  -h, --help            show this help message and exit
su www-data
+
  -s, --self            Use your credentials, rather than the proxyagent's
  svn co file:///svnroot/mediawiki/USERINFO /var/cache/svnusers
+
  -D BINDAS, --bindas=BINDAS
 +
                        Specify user to bind as
 +
  -m, --directorymanager
 +
                        Use the Directory Manager's credentials, rather than
 +
                        your own
 +
  --shell=LOGINSHELL    The user's shell (default: /usr/bin/sillyshell)
 +
  --gid=GIDNUMBER      The user's gid (default: 500)
 +
  --uid=UIDNUMBER      The user's uid (default: next available uid)
 +
  --home=HOMEDIRECTORY The user's home directory (default /home/username)
 +
  --cn=CN              The user's CN (default: firstname + lastname, or
 +
                        username, if first/last not set)
 +
  --firstname=GIVENNAME
 +
                        The user's first name (default: username)
 +
  --lastname=SN        The user's last name (default: username)
  
 
=== Dumps ===
 
=== Dumps ===

Revision as of 18:15, 23 October 2010

Contents

Hardware

  • formey.esams.wikimedia.org
    • SVN+SSH developer access
    • SVN over HTTP public read-only access
    • ViewVC

Related services

  • svn-private - houses internal scripts and configurations that can't be publicly viewable

Dependencies:

Used by:

Server setup

Packages

apt-get install some packages:

 subversion          # SVN core bits
 libsvn-notify-perl  # For commit notification mails
 viewvc              # For repository view
 python-subversion   # Needed for viewvc
 python-pygments     # Used by viewvc for syntax highlighting
 php5-cli            # Used by svnusers.php
 doxygen             # For doc generation
 apache2             # Web server!
 libapache2-svn      # HTTP SVN module for apache
 libapache2-mod-php5 # For user.php

Custom software

 /usr/local/bin/sillyshell     # restricted shell; in tools/sillyshell in SVN
 /usr/local/bin/ciabot_svn.py  # IRC notifications for SVN, available at cia.cx
 /usr/local/viewvc/*           # ViewVC - viewvc.org

Doc generation

Handled by puppet.

# Set up an account to run the docs under
adduser --system mwdocs
su -s /bin/bash mwdocs
cd && svn co file:///svnroot/mediawiki/trunk/phase3
# make a log file for the cronjob
cd /var/log
touch mwdocs.log
chown mwdocs mwdocs.log 
# fix path in apache config
# add a cron job:
crontab -e
30 0 * * * (cd /home/mwdocs/phase3 && svn up && php maintenance/mwdocgen.php --all) >> /var/log/mwdocs.log 2>&1


SVN users setup

Run add-ldap-user on formey:

Usage: add-ldap-user [options] <username> <filename>
example: add-ldap-user laner http://ryandlane.com/static/pubkey.key
Options:
  -h, --help            show this help message and exit
  -s, --self            Use your credentials, rather than the proxyagent's
  -D BINDAS, --bindas=BINDAS
                        Specify user to bind as
  -m, --directorymanager
                        Use the Directory Manager's credentials, rather than
                        your own
  --shell=LOGINSHELL    The user's shell (default: /usr/bin/sillyshell)
  --gid=GIDNUMBER       The user's gid (default: 500)
  --uid=UIDNUMBER       The user's uid (default: next available uid)
  --home=HOMEDIRECTORY  The user's home directory (default /home/username)
  --cn=CN               The user's CN (default: firstname + lastname, or
                        username, if first/last not set)
  --firstname=GIVENNAME
                        The user's first name (default: username)
  --lastname=SN         The user's last name (default: username)

Dumps

Currently handled by demon, runs as a cron on Saturdays at 1AM. Stored at /home/demon/svn/* (mounted from storage2 by Tomasz). Can do next incremental dump by running /home/demon/svn-dump-update. If you need to do a full dump (we do incremental by default) this takes about an hour or so to dump ~65000 revs. Available on the web at svn.wikimedia.org/dumps

  • Needs more permanent home

Post-commit hooks

SVN Post commit hooks are used for commit e-mail notification, CIA and CodeReview, and can be found in /svnroot/svn tree/hooks/

CodeReview

# Hit CodeReview
wget -q -O /dev/null --post-data="" \
  "http://www.mediawiki.org/w/api.php?action=codeupdate&format=xml&repo=MediaWiki&rev=$REV" &
Personal tools
Namespaces

Variants
Actions
Navigation
Ops documentation
Wiki
Toolbox