Squids

Revision as of 11:48, 21 October 2004

• 4 squids - using 11 IPs, to allow fairly even load-balancing if 1 or 2 squids go down

• to add new IPs for squid:
  • add the IPs in /var/named/master/wikipedia.zone in the obvious places (on zwinger)
  • make sure to update the serial number in the zone file!
  • run rndc reload to make it update - no need to restart named

• some useful commands:
  • dig ANY en.wikipedia.org @zwinger.wikipedia.org
  • dig +short ANY en.wikipedia.org

IPs of virtual ethernet interfaces

browne:    207.142.131.235, 207.142.131.247
coronelli: 207.142.131.236, 207.142.131.248, 207.142.131.205
maurus:    207.142.131.245, 207.142.131.246, 207.142.131.206
rabanus:   207.142.131.202, 207.142.131.203, 207.142.131.204

• According to damyta, should use 255.255.255.255 for netmask for the IPs of virtual eth interfaces - otherwise there can be routing confusion.
  • We've been using this configuration for months now and it works fine. -- Jeronim 10:41, 19 Sep 2004 (UTC)
• You must ping through the switch via a virtual interface which you have just brought up (using ping -I), to update the switch's idea of who has what IP. When pinging with -I207.142.131.248 or similar, use suda as the ping target; pinging the broadcast address does not always work.
  • Can use script takeip in /home/wikipedia/bin to take over an IP if a squid goes down.
  • To take down a virtual eth interface, /sbin/ifconfig eth0:n down
  • TODO: set up heartbeat for automatic IP takeover.

Things to check when it goes crazy

• check to see if access.log (probably in /var/log/squid) has reached 2GB in size. If it has, logrotate it or at least rename it.

Logs

Currently on yongle in /var/backup/archiv, and/or on zwinger in /home/wikipedia/logs/archiv/

French squids

• 3 600MHz Celeron 1U machines with 20GB HDD and 128MB RAM (now upgraded to 640MB) - HP Web Hosting Server Appliance sa1100 - specs: [1]
• 2 serial ports, 2 NICs, no graphics card
• Debian
• names are chloe, bleuenn, and ennael
• to be located in a colo on Paris - provided free by http://www.lost-oasis.fr/
• serial cables running from ttyS1 on one machine to ttyS0 on another; getty running on ttyS0 on each machine, to allow communication using minicom
• IP range will be 212.85.147.66 - .70

New squid setup

#!/bin/sh
echo 'getting squid from browne'
scp -r  root@browne.wikimedia.org:/usr/local/squid /usr/local
echo 'getting config'
scp -r  root@zwinger.wikimedia.org:/home/gwicke/squid/newsquid.conf /usr/local/squid/etc/squid.conf
echo 'create a shortcut to /usr/local/bin/squid'
ln -s /usr/local/squid/bin/RunCache /usr/local/bin/squid
#squid user/group is already existing on all servers
#echo 'setting up the squid user'
#adduser --system --no-create-home --disabled-login squid
#echo 'setting up the squid group'
#addgroup --system --no-create-home squid

echo 'creating /var/spool/squid and /var/log/squid'
mkdir /var/spool/squid
chown -R squid:squid /var/spool/squid
mkdir /var/log/squid
chown -R squid:squid /var/log/squid
mkdir /var/log/squid/outgoing
chown -R squid:squid /var/log/squid/outgoing
echo 'edit the configuration'
vim /usr/local/squid/etc/squid.conf
echo 'initializing cache dirs'
/usr/local/squid/sbin/squid -z

echo 'getting log rotation scripts'
scp root@zwinger.wikimedia.org:/home/gwicke/squid/crontab /root
scp root@browne.wikimedia.org:/usr/local/bin/preplogs_for_move /usr/local/bin/preplogs_for_move
scp root@browne.wikimedia.org:/etc/logrotate.d/squid /etc/logrotate.d/squid
# don't overwrite root's crontab, as there maybe something important in there
# crontab -u root /root/crontab

• log transfer from /var/log/squid/outgoing/* to /home/wikipedia/logs/incoming/{hostname}/

to investigate

• Selectively remove Squid-2 cache contents: http://www.wa.apana.org.au/~dean/squidpurge/