Job queue

From Wikitech
(Difference between revisions)
Jump to: navigation, search
(how to stop)
(Emergency kill)
 
(7 intermediate revisions by 3 users not shown)
Line 1: Line 1:
Job queue runners run on srv151-180 (the job-runners node group). Start them with
+
__TOC__
  
dsh -N job-runners -f /home/wikipedia/bin/jobs-daemon.
+
==Overview==
 +
Job queue runners run on the job-runners node group. Install a new job runner using the application::jobrunner class in puppet.
  
No installation or setup is required, just MediaWiki. The script contains:
+
The daemon is controlled via <tt>/etc/init.d/mw-job-runner</tt> and will start on boot by default.
  
:<tt>start-stop-daemon --start --background --chuid apache --nicelevel 20 --make-pidfile --pidfile /var/run/mw-jobs.pid --startas /home/wikipedia/common/php-1.5/maintenance/jobs-loop.sh </tt>
+
See also http://www.mediawiki.org/wiki/Manual:Job_queue
  
To stop the job runners on a given server, use:
+
==Emergency kill==
 
+
If there's an urgent call to kill job queues everywhere for example because they're causing clusterwide swapdeath:<br />
start-stop-daemon --stop --pidfile /var/run/mw-jobs.pid
+
- on the job-runner group (see /home/config/others/usr/local/dsh/node_groups) . . .<br />
 
+
- as root . . .<br />
It should trap SIGTERM and shut down the child processes.
+
- dsh -g job-runners pkill -9 -f obs (this matches jobs-loop and RunJobs and few others, we don't trap sinals in our php scripts so there's no need to kill gracefully)<br />
 
+
- if you end up having to power cycle hosts that are nonresponsive, remember to rerun the dsh afterward to make sure job-runners is stopped there as well until someone can investigate the memory issue
See also http://www.mediawiki.org/wiki/Manual:Job_queue
+

Latest revision as of 15:55, 19 January 2012

Contents


[edit] Overview

Job queue runners run on the job-runners node group. Install a new job runner using the application::jobrunner class in puppet.

The daemon is controlled via /etc/init.d/mw-job-runner and will start on boot by default.

See also http://www.mediawiki.org/wiki/Manual:Job_queue

[edit] Emergency kill

If there's an urgent call to kill job queues everywhere for example because they're causing clusterwide swapdeath:
- on the job-runner group (see /home/config/others/usr/local/dsh/node_groups) . . .
- as root . . .
- dsh -g job-runners pkill -9 -f obs (this matches jobs-loop and RunJobs and few others, we don't trap sinals in our php scripts so there's no need to kill gracefully)
- if you end up having to power cycle hosts that are nonresponsive, remember to rerun the dsh afterward to make sure job-runners is stopped there as well until someone can investigate the memory issue

Personal tools
Namespaces

Variants
Actions
Navigation
Ops documentation
Wiki
Toolbox