Squids

There are 4 clusters of squid servers, one upload and one text at each of our two locations: esams and pmpta. Each server runs two instances of squid: a frontend squid listening on port 80, and a cache squid listening on port 3128. The purpose of the frontend squid is to distribute load to the cache squids based on URL hash, using the CARP algorithm.

LVS is used to balance incoming requests between the CARP frontends.

Squid cluster distribution

	text	upload
pmtpa	sq16-sq40 208.80.152.2	sq1-sq15, sq41-sq50 208.80.152.3
knams	knsq1-knsq7, knsq23-30 91.198.174.2	knsq8 - knsq22 91.198.174.3
yaseo	yf1000-yf1004 203.212.189.253	yf1005 - yf1009 203.212.189.254

RE-Installation

Please note that NEW squid servers need to be setup by MARK. There are a number of various setttings that have to be configured, which are a bit in flux and also not yet documented to the point of others implementing new squids. Thus the instructions are only for reinstallation

To reinstall a previously existing squid server:

• Save the SSH Hostkeys (if possible).
• Reinstall the server OS.
• After boot, copy the old ssh hostkey back using scp -o StrictHostKeyChecking=no files hostname:/etc/ssh/
• Follow the instructions on Puppet#Reinstalls
• Leave it alone, puppet will configure the packages and settings from there. (Yes, it just got that easy.)

Old reinstallation instructions

To reinstall a previously existing Squid server:

1. Save the SSH hostkeys
2. Reinstall the server using PXE as documented in Automated installation
3. After boot, copy the old SSH hostkey back using scp -o StrictHostKeyChecking=no files hostname:/etc/ssh/
4. Log in, and set the root password
5. # apt-get install wikimedia-task-squid (Answer the question about the LVS service IP very carefully)
6. From zwinger, do: cd /home/w/conf/squid/ && make && ./deploy hostname
7. (upload squids only) Run # setup-aufs-cachedirs to set up the AUFS cache partition. This will wipe any previous AUFS partition/data!
8. If the Squid had not been running for a long time, clean the cache with /etc/init.d/squid clean
   1. Anything under ~2 hours and the cache does not need to be cleaned.
9. # /etc/init.d/squid start
   1. Wait a few minutes after starting the backend here to start the frontend.
   2. You can watch TOP and see when the squid process slows down, or just read /var/log/squid/cache.log
10. # /etc/init.d/squid-frontend start
11. Install ganglia gmond

Configuration

Configuration is done by editing the master files in /home/wikipedia/conf/squid, then running make to rebuild the configuration files, and ./deploy to deploy them to the remote servers. The configuration files are:

squid.conf.php 

Template file for the cache (backend) instances

frontend.conf.php 

Template file for the frontend instances

text-settings.php 

A settings array which applies to text squids. All elements in this array will become available as variables during execution of squid.conf.php and frontend.conf.php. The settings array can be used to give server-specific configuration.

upload-settings.php 

Same as text-settings.php but for upload squids

common-acls.conf 

ACL directives used by both text and upload frontends. Use this to block clients from all access.

upload-acls.conf 

ACL directives used by upload frontends. Use this for e.g. image referrer regex blocks.

text-acls.conf 

ACL directives used by text frontends. Use this for e.g. remote loader IP blocks.

Configuration.php 

Contains most of the generator code

generate.php 

the script that the makefile runs

Feel free to check in your changes to RCS.

The deployment script has lots of options. Run it with no arguments to get a summary.

Current problems

(none)

See also

• MediaWiki caching -- some cache headers explained
• Multicast HTCP purging -- new method of cache purging
• Squid logging
• Squid log format