Squids

From Wikitech
(Difference between revisions)
Jump to: navigation, search
(→‎Architecture)
(Remove old outdated stuff)
Line 17: Line 17:
 
The xx.wikipedia.org switches don't have a load balancer. IP addresses have to be assigned manually. Check /usr/local/dsh/node_groups/squid for available hosts and use nslookup to identify which IPs to assign to them. See [[#IPs of virtual ethernet interfaces|below for details]].
 
The xx.wikipedia.org switches don't have a load balancer. IP addresses have to be assigned manually. Check /usr/local/dsh/node_groups/squid for available hosts and use nslookup to identify which IPs to assign to them. See [[#IPs of virtual ethernet interfaces|below for details]].
  
−
===To switch away from Paris squids===
+
===To switch away from foreign squids===
 
*Need to be root
 
*Need to be root
 
*On zwinger:
 
*On zwinger:
Line 25: Line 25:
 
** Wait for DNS propagation time (600s, in /usr/local/etc/pdns.conf geo-ttl=600 . Might reduce it to give faster switch back after problem is over, reducing it can't make the emregency fix work faster though - it's cache time for other DNS servers)
 
** Wait for DNS propagation time (600s, in /usr/local/etc/pdns.conf geo-ttl=600 . Might reduce it to give faster switch back after problem is over, reducing it can't make the emregency fix work faster though - it's cache time for other DNS servers)
 
*Reverse these changes to switch back.
 
*Reverse these changes to switch back.
−
 
−
==General==
 
−
 
−
* to add new IPs for squid:
 
−
** add the IPs in /var/named/master/wikipedia.zone in the obvious places (on zwinger)
 
−
** make sure to update the serial number in the zone file!
 
−
** run <tt>rndc reload</tt> to make it update - no need to restart named
 
−
 
−
* some useful commands:
 
−
** dig ANY en.wikipedia.org @zwinger.wikipedia.org
 
−
** dig +short ANY en.wikipedia.org
 
−
 
−
* editing squid.conf for all: /h/w/conf/squid, read the README
 
  
 
=== Squid builds ===
 
=== Squid builds ===
  
 
Currently running:
 
Currently running:
−
* stable9 + all stable 9 patches except broken 2GB patch + gwicke + nortt + [[Multicast HTCP purging|htcpclr]] (identifies as: 2.5.STABLE9.wp20050410.S9plus.no2GB[icpfix,nortt,htcpclr]) builddir: <tt>/home/wikipedia/src/squid/squid-2.5.STABLE9-kate-no2GB</tt>
+
* squid-2.5.STABLE13-7wm.src.rpm - Multiple binary builds exist for different arches and/or distributions.
−
* squid-2.5.STABLE12RC1-1wm.src.rpm - Multiple binary builds exist for different arches and/or distributions.
+
  
−
==IPs of virtual ethernet interfaces==
+
==IPs of virtual ethernet interfaces on old PMTPA text Squids==
 
* to find out the IPs, use <tt>nslookup rr.pmtpa.wikimedia.org</tt>
 
* to find out the IPs, use <tt>nslookup rr.pmtpa.wikimedia.org</tt>
 
* Used to be assigned at boot time, but this can lead to problems with duplicated IP addresses.
 
* Used to be assigned at boot time, but this can lead to problems with duplicated IP addresses.
Line 52: Line 38:
 
* According to dammit, should use 255.255.255.255 for netmask for the IPs of virtual eth interfaces - otherwise there can be routing confusion. takeip does this automatically
 
* According to dammit, should use 255.255.255.255 for netmask for the IPs of virtual eth interfaces - otherwise there can be routing confusion. takeip does this automatically
  
−
;TODO: set up ''heartbeat'' or wackamole for automatic IP takeover.
+
;TODO: set up LVS
  
−
== Starting ==
+
== Common operations ==
−
=== Squid RPM ===
+
=== Starting ===
 
The Squid RPM has a SysV init script <tt>/etc/init.d/squid</tt> like any proper RPM, start it using
 
The Squid RPM has a SysV init script <tt>/etc/init.d/squid</tt> like any proper RPM, start it using
 
  # /sbin/service squid start
 
  # /sbin/service squid start
Line 61: Line 47:
 
It's automatically started at boot time, alter this using <tt>/sbin/chkconfig</tt>.
 
It's automatically started at boot time, alter this using <tt>/sbin/chkconfig</tt>.
  
−
=== Old squid builds ===
+
=== Reloading ===
−
<tt>/usr/local/bin/</tt> squid is a symlink to /usr/local/squid/RunCache which is a loop that starts squid with the right args and sets the ulimit (squid needs more than 1024 fd's, hence it does ulimit -n 8192).
+
# /sbin/service squid restart
  
−
== Reloading ==
+
===cachemgr.cgi===
−
After config changes, call squidhup (in /usr/local/bin) which is short for /usr/local/squid/sbin/squid -k reconfigure.
+
−
 
+
−
==Logs==
+
−
Logs are rotated every 10 minutes, and deleted shortly after. The logrotate config is in <tt>/etc/logrotate.d/squid</tt>.
+
−
 
+
−
==cachemgr.cgi==
+
 
:''this needs to be updated''
 
:''this needs to be updated''
 
There is a cachemgr.cgi available at http://noc.wikimedia.org/~mark/cgi-bin/cachemgr.cgi. The password (for at least the French squids at this moment) can be found in <tt>/home/wikipedia/doc/fr-cachemgr-pw</tt>.
 
There is a cachemgr.cgi available at http://noc.wikimedia.org/~mark/cgi-bin/cachemgr.cgi. The password (for at least the French squids at this moment) can be found in <tt>/home/wikipedia/doc/fr-cachemgr-pw</tt>.
−
 
−
==to investigate==
 
−
* Selectively remove Squid-2 cache contents: http://www.wa.apana.org.au/~dean/squidpurge/
 
−
 
−
==Diagnostic aid==
 
−
innocence modified the squid error pages to identify which squid was erroring; it was mostly benet on 14 Jan 04; Steps Were Taken
 
−
 
−
==New Donors==
 
−
 
−
As, and even before, we get our protocols entirely worked out for remote squid sites, people are going to [[Volunteer Squid Sites|volunteer]].  That link will provide such sites with a place to put their contact and other information, and for us to ask them questions.
 
  
 
== See also ==
 
== See also ==
Line 90: Line 60:
  
 
== New squid setup ==
 
== New squid setup ==
−
=== memcheck ===
+
* Install the Squid RPM
−
Squid has demanding memory access patterns, these memtest options might help to catch more problems before they are real ones:
+
* Adapt LVS
−
"Also, go to the options, and turn on caching, as well as all memory addresses and tests ... (keys pressed if I can remember, is:
+
−
c->1->2->2->3->3->3
+
−
should turn on above options for memtest)." (from [http://www.uwsg.iu.edu/hypermail/linux/kernel/0401.0/1087.html])
+
−
 
+
−
=== Setup ===
+
−
:''This is out of date information, adapt to the new Squid RPM''
+
−
 
+
−
* Add the machine to /home/wikipedia/conf/squid/Makefile, run make
+
−
* On the new machine run '''/home/wikipedia/bin/squidsetup''' as root after doing a ssh-add. If there are errors fetching the squid stuff, adjust the server to fetch from in the script.
+
−
* follow the instructions re crontab
+
−
* test the server using telnet or nc
+
−
* take ips
+
−
* add /usr/local/bin/squid to /etc/rc.local
+
−
 
+
 
[[Category:How-To]] [[Category:Software]]
 
[[Category:How-To]] [[Category:Software]]

Revision as of 15:13, 22 August 2006

This page desperately needs to be filled out or brought up to date. If you're familiar with the operations of this part of the site, please help!

FIXME: How to update squid error pages

Contents


Architecture

There are 5 squid farms at the moment:

  • PMTPA text squids, serving all text pages. Uses IP/DNS-based round robin, no load balancer.
  • PMTPA image squids, serving upload.wikimedia.org. Uses LVS load balancing on avicenna.
  • YASEO text squids, serving all text pages for Asian users. Uses LVS load balancing.
  • KNAMS text squids, serving all text pages for European users. Uses LVS load balancing on pascal.
  • LOPAR text squids, unused at the moment. decomissioned

Emergency operations

After a PMTPA power outage

The xx.wikipedia.org switches don't have a load balancer. IP addresses have to be assigned manually. Check /usr/local/dsh/node_groups/squid for available hosts and use nslookup to identify which IPs to assign to them. See below for details.

To switch away from foreign squids

  • Need to be root
  • On zwinger:
    • Edit /usr/local/etc/powerdns/geomaps/rr.wikimedia.org. Each non comment line is a country code mapping to a cluster name.(0 is default route, to go to Florida, must be left in).
    • Load root ssh key
    • Run /usr/local/etc/powerdns/update to deploy.
    • Wait for DNS propagation time (600s, in /usr/local/etc/pdns.conf geo-ttl=600 . Might reduce it to give faster switch back after problem is over, reducing it can't make the emregency fix work faster though - it's cache time for other DNS servers)
  • Reverse these changes to switch back.

Squid builds

Currently running:

  • squid-2.5.STABLE13-7wm.src.rpm - Multiple binary builds exist for different arches and/or distributions.

IPs of virtual ethernet interfaces on old PMTPA text Squids

  • to find out the IPs, use nslookup rr.pmtpa.wikimedia.org
  • Used to be assigned at boot time, but this can lead to problems with duplicated IP addresses.
  • Can use script takeip in /home/wikipedia/bin to take over an IP if a squid goes down.
  • To take down a virtual eth interface, /sbin/ip addr del xxx.xxx.xxx.xxx dev eth0
  • According to dammit, should use 255.255.255.255 for netmask for the IPs of virtual eth interfaces - otherwise there can be routing confusion. takeip does this automatically
TODO
set up LVS

Common operations

Starting

The Squid RPM has a SysV init script /etc/init.d/squid like any proper RPM, start it using 

# /sbin/service squid start

It's automatically started at boot time, alter this using /sbin/chkconfig.

Reloading 

# /sbin/service squid restart

cachemgr.cgi

this needs to be updated

There is a cachemgr.cgi available at http://noc.wikimedia.org/~mark/cgi-bin/cachemgr.cgi. The password (for at least the French squids at this moment) can be found in /home/wikipedia/doc/fr-cachemgr-pw.

See also

New squid setup

  • Install the Squid RPM
  • Adapt LVS
Retrieved from "http://wikitech-old.wikimedia.org/index.php?title=Squids&oldid=9429"
Personal tools
Namespaces

Variants
Actions
Navigation
Ops documentation
Wiki
Toolbox