Ganglia

unicast vs multicast

• As of version 3.0.0, released 2005-02-07, status messages can be sent over unicast.

components

• gmetad
  • installed on zwinger using ganglia-monitor-core-gmetad-2.5.6-1.i386.rpm
• gmond
  • installed on each machine using ganglia-monitor-core-gmond-2.5.6-1.i386.rpm
  • /etc/gmond.conf on each machine MUST be a symlink to /home/wikipedia/gmond.conf. If the default gmond.conf is used, ganglia stats for the entire cluster will not be recorded in the right place and will be effectively lost. (But see merging RRDs below.)

version mismatches

• things go wrong when gmond and gmetad have different versions. or is it when peer gmonds have different versions? something like that anyway

zwinger

RRDs (ganglia statistics, in this case) are in /home/wikipedia/rrds there is some old data in the default location, /var/lib/ganglia/rrds

cluster-wide ganglia restart

If something is amiss with the state of ganglia, and reconfiguring and restarting gmetad isn't enough, do this on zwinger as root:

#!/bin/bash
/etc/init.d/gmetad stop
dsh -f -a /etc/init.d/gmond stop
sleep 5
dsh -f -a /etc/init.d/gmond start
/etc/init.d/gmetad start

or you can just run /home/wikipedia/bin/ganglia-restart-all.

merging RRDs

It is possible, in principle, to merge RRDs using a perl script which can be found on the net somewhere. Looks like a royal pain though.