PyBal
(backed up) |
|||
| (10 intermediate revisions by 7 users not shown) | |||
| Line 1: | Line 1: | ||
| − | '''PyBal''' is a LVS monitoring script quite similar to [[lvsmon]]. It's written in Python using the [http://www.twistedmatrix.com Twisted framework]. | + | '''PyBal''' is a [[LVS]] monitoring script quite similar to [[lvsmon]]. It's written in Python using the [http://www.twistedmatrix.com Twisted framework]. |
| + | |||
| + | '''For more information about Wikimedia's LVS setup in general, see [[LVS]].''' | ||
At this moment, just a few features distinguish it from lvsmon: | At this moment, just a few features distinguish it from lvsmon: | ||
| Line 8: | Line 10: | ||
...but I intend to polish it more, and extend it with useful things. | ...but I intend to polish it more, and extend it with useful things. | ||
| − | The script is in MediaWiki CVS, | + | The script <s>is</s> was in MediaWiki <s>CVS</s> SVN repo "mediawiki", subdirectory <s>[http://svn.wikimedia.org/viewvc/mediawiki/trunk/pybal/ pybal]</s>. |
-- Mark | -- Mark | ||
== Setup == | == Setup == | ||
| − | PyBal is currently installed on [[ | + | PyBal is currently installed on our [[LVS]] hosts, in directory <tt>/usr/sbin</tt>. Start or stop it by |
| − | + | /etc/init.d/pybal start|stop | |
| − | + | ||
| − | Configuration is in <tt>/etc/pybal/</tt>. <tt>pybal.conf</tt> defines the LVS service parameters | + | '''NOTE that recently pybal seems to not always stop''' when receiving a TERM signal (i.e. via this script). So you need to check the process start time and see if it *really* restarted, if not kill -9 manually, then start via script. |
| + | |||
| + | Configuration is in <tt>/etc/pybal/</tt>. <tt>pybal.conf</tt> defines the LVS service parameters | ||
| + | |||
| + | The list of pooled hosts resides in <tt>/home/w/conf/pybal</tt> (on fenari), with one file per LVS service. Attributes: | ||
| + | |||
| + | *weight: a larger number means that more requests get sent to this server in comparison with others | ||
| + | *enabled: either True or False, depnding on whether you want requests to be sent to this server | ||
| + | |||
| + | The format should be fairly self explanatory; the files more or less use Python assignment / dictionary syntax. | ||
PyBal supports multiple LVS services through a single instance and configuration file <tt>pybal.conf</tt>, e.g.: | PyBal supports multiple LVS services through a single instance and configuration file <tt>pybal.conf</tt>, e.g.: | ||
| Line 33: | Line 43: | ||
port = 80 | port = 80 | ||
scheduler = wlc | scheduler = wlc | ||
| − | config = file:///etc/pybal/ | + | config = file:///etc/pybal/upload-squids |
</pre> | </pre> | ||
'''Beware''', the code as checked out from Subversion has '''<tt>DryRun = True</tt>''' set in <tt>ipvs.py</tt>, meaning that it will not modify any actual IPVS state but only show the commands for debugging. This should be changed to a commandline option, but for now edit that file to <tt>DryRun = False</tt>. | '''Beware''', the code as checked out from Subversion has '''<tt>DryRun = True</tt>''' set in <tt>ipvs.py</tt>, meaning that it will not modify any actual IPVS state but only show the commands for debugging. This should be changed to a commandline option, but for now edit that file to <tt>DryRun = False</tt>. | ||
| − | The configuration | + | The configuration files are (manually) backed up in <tt>/home/wikipedia/conf/pybal</tt>. |
== Howto == | == Howto == | ||
| + | See [[LVS]]. | ||
| − | + | [[Category:Software]] | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
Latest revision as of 09:09, 11 September 2012
PyBal is a LVS monitoring script quite similar to lvsmon. It's written in Python using the Twisted framework.
For more information about Wikimedia's LVS setup in general, see LVS.
At this moment, just a few features distinguish it from lvsmon:
- It's using asynchronous communication, and thus runs all checks in parallel instead of sequentially
- It has an extra monitoring method called IdleConnection, which keeps an idle connection open to all squids, and therefore notices immediately when the Squid processes are shut down / crashing
- It can fetch server lists over HTTP as well as from the local filesystems
...but I intend to polish it more, and extend it with useful things.
The script is was in MediaWiki CVS SVN repo "mediawiki", subdirectory pybal.
-- Mark
[edit] Setup
PyBal is currently installed on our LVS hosts, in directory /usr/sbin. Start or stop it by
/etc/init.d/pybal start|stop
NOTE that recently pybal seems to not always stop when receiving a TERM signal (i.e. via this script). So you need to check the process start time and see if it *really* restarted, if not kill -9 manually, then start via script.
Configuration is in /etc/pybal/. pybal.conf defines the LVS service parameters
The list of pooled hosts resides in /home/w/conf/pybal (on fenari), with one file per LVS service. Attributes:
- weight: a larger number means that more requests get sent to this server in comparison with others
- enabled: either True or False, depnding on whether you want requests to be sent to this server
The format should be fairly self explanatory; the files more or less use Python assignment / dictionary syntax.
PyBal supports multiple LVS services through a single instance and configuration file pybal.conf, e.g.:
[text] protocol = tcp ip = 145.97.39.155 port = 80 scheduler = wlc config = file:///etc/pybal/text-squids [images] protocol = tcp ip = 145.97.39.156 port = 80 scheduler = wlc config = file:///etc/pybal/upload-squids
Beware, the code as checked out from Subversion has DryRun = True set in ipvs.py, meaning that it will not modify any actual IPVS state but only show the commands for debugging. This should be changed to a commandline option, but for now edit that file to DryRun = False.
The configuration files are (manually) backed up in /home/wikipedia/conf/pybal.
[edit] Howto
See LVS.
