Squids

There are 6 clusters of squid servers, one upload and one text at each of our three locations: yaseo, knams and pmpta. Each server runs two instances of squid: a frontend squid listening on port 80, and a cache squid listening on port 3128. The purpose of the frontend squid is to distribute load to the cache squids based on URL hash, using the CARP algorithm.

LVS is used to balance incoming requests between the CARP frontends.

Installation

All Wikimedia squids are running ubuntu. All necessary packages for squid can be installed by installing the wikimedia-task-squid package from the Wikimedia APT repository.

Configuration

Configuration is done by editing the master files in /home/wikipedia/conf/squid, then running make to rebuild the configuration files, and ./deploy to deploy them to the remote servers. The configuration files are:

squid.conf.php 

Template file for the cache (backend) instances

frontend.conf.php 

Template file for the frontend instances

text-settings.php 

A settings array which applies to text squids. All elements in this array will become available as variables during execution of squid.conf.php and frontend.conf.php. The settings array can be used to give server-specific configuration.

upload-settings.php 

Same as text-settings.php but for upload squids

common-acls.conf 

ACL directives used by both text and upload frontends. Use this to block clients from all access.

upload-acls.conf 

ACL directives used by upload frontends. Use this for e.g. image referrer regex blocks.

text-acls.conf 

ACL directives used by text frontends. Use this for e.g. remote loader IP blocks.

Configuration.php 

Contains most of the generator code

generate.php 

the script that the makefile runs

Feel free to check in your changes to RCS.

The deployment script has lots of options. Run it with no arguments to get a summary.

See also

• MediaWiki caching -- some cache headers explained
• Multicast HTCP purging -- new method of cache purging
• Squid logging
• Squid log format