Puppet
puppet is the main configuration management tool to be used on the Wikimedia clusters.
puppetd is the client daemon that runs on all servers, and manages machines with configuration information gathered from puppetmasterd, running on machine sockpuppet.pmtpa.wmnet.
Contents |
puppetd
To install puppet on a single machine, simply run
# apt-get install puppet
Communication with the puppetmaster server is over encrypted SSL and with signed certificates. To sign the certificate of the newly installed machine on the puppetmaster server, log in on sockpuppet.pmtpa.wmnet and run:
# puppetca -s clienthostname
To check the list of outstanding, unsigned certificates, use:
# puppetca -l
Puppetmaster
The puppetmaster server in pmtpa is sockpuppet.pmtpa.wmnet.
Installation
Simply use the (backported) puppetmaster Ubuntu package:
# apt-get install puppetmaster
Configuration
The default configuration is very usable, but we've made some tweaks here and there.
See /etc/puppet/site.pp for the basics. Puppet currently pushes out crontabs for the image scalers, ganglia binaries and conf files on on hosts, and syncs user information including ssh keys on all hosts. It will reread its conf instantly. Changes to any given host get pushed out every 30 minutes, but puppet is continually updating some host or other. See syslog on sockpuppet for details.
MD5 is broken, use SHA1 for signing certificates:
ca_md=sha1
Making changes
We have a private svn for changes. Here's what you do (on sockpuppet):
cd /root/puppet ; svn up ; edit ; svn commit ; cd /etc/puppet/manifests; svn up
Or Else!
You can syntax check your changes by
puppet --parseonly filename-here
You can do a dry run of your changes by taking a copy of the /root/puppet directory, putting it in, say, /home/myname/puppet on the node you want to test, and running the following command on that host:
puppet --noop --verbose --debug --modulepath /home/myusername/puppet/puppet/ puppet/manifests/site.pp
This will give you (among other things) a list of all the changes it would make, followed by "(noop)".
Disabled on certain hosts
Puppet is disabled on the gangllia aggregators while folks work out a little issue with puppet not recognizing which hosts are the aggregators and therefore setting them all not to be :-P This caused ganglia to not receive any data after a puppet refresh. In theory this is fixed now, in practice it needs to be tested and rolled out.
Todo
- More secure certificate signing
- Better, more automated version control
- Better tools for adding/maintaining node definitions
