Cron jobs

Note: the job queue runs continuously on many servers and is not a cron job.

QueryPage update

hume:/etc/cron.d/mw-update-special-pages: updates the special pages derived from QueryPage

PATH=/usr/local/bin:/bin:/usr/bin
00 4 */3 * * apache flock -n /var/lock/update-special-pages-small \
  /home/wikipedia/bin/update-special-pages-small > \
  /home/wikipedia/logs/norotate/updateSpecialPages-small.log 2>&1
00 5 */3 * * apache flock -n /var/lock/update-special-pages \
   /home/wikipedia/bin/update-special-pages > \
   /home/wikipedia/logs/norotate/updateSpecialPages.log       2>&1

update-special-pages-small

#!/bin/bash

cd /home/wikipedia/common/php-1.5/maintenance
for db in `</home/wikipedia/common/small.dblist`; do
	echo $db
	php updateSpecialPages.php $db
	echo
	echo
done

update-special-pages

#!/bin/bash

cd /home/wikipedia/common/php-1.5/maintenance
for db in `</home/wikipedia/common/all.dblist`; do
	echo $db
	php updateSpecialPages.php $db
	echo
	echo
done

Tor exit list update

hume:/etc/cron.d/mw-tor-list: Loads the tor exit list from check.torproject.org and saves it into memcached for later use by the TorBlock extension.

PATH=/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/wikipedia/bin:/home/tstarling/bin:/usr/local/bin:/usr/local/mysql/bin:/home/tstarling/bin
*/20 * * * * php -n /home/wikipedia/common/php/extensions/TorBlock/loadExitNodes.php 2>&1

FlaggedRevs stats update

hume:/etc/cron.d/mw-flagged-revs: Updates the flaggedrevs_stats table

0 */2 * * * /home/wikipedia/common/php/extensions/FlaggedRevs/maintenance/wikimedia-periodic-update.sh 2>&1

wikimedia-periodic-update.sh

#!/bin/bash
for db in `</home/wikipedia/common/flaggedrevs.dblist`;do
	echo $db
	php -n /home/wikipedia/common/php/extensions/FlaggedRevs/maintenance/updateStats.php $db
done

Ganglia RRD commit

zwinger:/etc/cron.hourly/save-gmetad-rrds: The live RRD files for ganglia are kept in a tmpfs, for performance reasons. This script copies them back to disk in case of server restart

#!/bin/sh
/usr/local/bin/save-gmetad-rrds >> /var/log/save-gmetad-rrds.log 2>&1

save-gmetad-rrds

#!/bin/bash
service gmetad_pmtpa stop
echo "Saving RRDs..."
time rsync -a /mnt/ganglia_tmp/rrds.pmtpa/ /var/lib/ganglia/rrds.pmtpa
echo "Done"
service gmetad_pmtpa start

LDAP server backups

nfs1/2:/usr/local/sbin/opendj-backup.sh: Runs OpenDJ backups and stores them in /var/opendj/backup for pickup by amanda; cleans up backups older than three days.

0 18 * * * /usr/local/sbin/opendj-backup.sh > /dev/null 2>&1

SVN crons

formey:/usr/local/bin/svndump.php: Runs SVN dumps and stores them in /svnroot/bak for pickup by amanda; cleans up previous dump.

0 18 * * * /usr/local/bin/svndump.php > /dev/null 2>&1

formey:(mwdocs)/home/mwdocs/phase3/maintenance/mwdocgen.php: Updates the doxygen documentation for svn.

0 0 * * * (cd /home/mwdocs/phase3 && svn up && php maintenance/mwdocgen.php --all) >> /var/log/mwdocs.log 2>&1

formey:(www-data)svn up: Updates the userinfo file

0 0 * * * (cd /var/cache/svnusers && svn up) > /dev/null 2>&1