Squids
(udpated) |
(AIO livelock) |
||
| Line 24: | Line 24: | ||
The deployment script has lots of options. Run it with no arguments to get a summary. | The deployment script has lots of options. Run it with no arguments to get a summary. | ||
| + | |||
| + | == Current problems == | ||
| + | |||
| + | === AIO livelock === | ||
| + | Occasionally Squid runs into a livelock, doing no system calls but using 100% user CPU. A gdb backtrace showed: | ||
| + | |||
| + | #1 0x000000000048ae99 in squidaio_sync () at aufs/aiops.c:936 | ||
| + | #2 0x000000000048b123 in squidaio_queue_request (request=0x2aab938f4d00) at aufs/aiops.c:570 | ||
| + | #3 0x000000000048b78c in squidaio_read (fd=13, bufp=0x2aab93923000 "", bufs=6736, offset=3744910336, whence=0, resultp=0x2aab938f4d88) | ||
| + | |||
| + | Next time this happens, collect: | ||
| + | |||
| + | print request_queue_len | ||
| + | print request_queue.head | ||
| + | print request_queue2.head | ||
| + | print done_queue.head | ||
| + | print done_requests.head | ||
== See also == | == See also == | ||
Revision as of 23:12, 12 June 2007
There are 6 clusters of squid servers, one upload and one text at each of our three locations: yaseo, knams and pmpta. Each server runs two instances of squid: a frontend squid listening on port 80, and a cache squid listening on port 3128. The purpose of the frontend squid is to distribute load to the cache squids based on URL hash, using the CARP algorithm.
LVS is used to balance incoming requests between the CARP frontends.
Contents |
Installation
All Wikimedia squids are running ubuntu. All necessary packages for squid can be installed by installing the wikimedia-task-squid package from the Wikimedia APT repository.
Configuration
Configuration is done by editing the master files in /home/wikipedia/conf/squid, then running make to rebuild the configuration files, and ./deploy to deploy them to the remote servers. The configuration files are:
- squid.conf.php
- Template file for the cache (backend) instances
- frontend.conf.php
- Template file for the frontend instances
- text-settings.php
- A settings array which applies to text squids. All elements in this array will become available as variables during execution of squid.conf.php and frontend.conf.php. The settings array can be used to give server-specific configuration.
- upload-settings.php
- Same as text-settings.php but for upload squids
- common-acls.conf
- ACL directives used by both text and upload frontends. Use this to block clients from all access.
- upload-acls.conf
- ACL directives used by upload frontends. Use this for e.g. image referrer regex blocks.
- text-acls.conf
- ACL directives used by text frontends. Use this for e.g. remote loader IP blocks.
- Configuration.php
- Contains most of the generator code
- generate.php
- the script that the makefile runs
Feel free to check in your changes to RCS.
The deployment script has lots of options. Run it with no arguments to get a summary.
Current problems
AIO livelock
Occasionally Squid runs into a livelock, doing no system calls but using 100% user CPU. A gdb backtrace showed:
#1 0x000000000048ae99 in squidaio_sync () at aufs/aiops.c:936 #2 0x000000000048b123 in squidaio_queue_request (request=0x2aab938f4d00) at aufs/aiops.c:570 #3 0x000000000048b78c in squidaio_read (fd=13, bufp=0x2aab93923000 "", bufs=6736, offset=3744910336, whence=0, resultp=0x2aab938f4d88)
Next time this happens, collect:
print request_queue_len print request_queue.head print request_queue2.head print done_queue.head print done_requests.head
See also
- MediaWiki caching -- some cache headers explained
- Multicast HTCP purging -- new method of cache purging
- Squid logging
- Squid log format
