Squids

Revision as of 16:15, 28 July 2004

DNS and squid

• to add new IPs for squid:
  • add the IPs in /var/named/master/wikipedia.zone in the obvious places (on zwinger)
  • make sure to update the serial number in the zone file!
  • run rndc reload to make it update - no need to restart named

• for 3 squids - use 6 IPs, to allow even load-balancing if 1 or 2 squids go down
• 2 scripts in /home/wikipedia/bin:
  • takeip
  • dropip (vapourware)

• some useful commands:
  • dig ANY en.wikipedia.org @zwinger.wikipedia.org
  • dig +short ANY en.wikipedia.org

Virtual IPs

browne:    207.142.131.235, 207.142.131.247
coronelli: 207.142.131.236, 207.142.131.248
maurus:    207.142.131.245, 207.142.131.246

• According to damyta, should use 255.255.255.255 for netmask for the IPs of virtual eth interfaces - otherwise there can be routing confusion.
• You must ping through the switch via a virtual interface which you have just brought up (using ping -I), to update the switch's idea of who has what IP. When pinging with -I207.142.131.248 or similar, use suda as the ping target; pinging the broadcast address does not always work.
  • Can use script takeip in /h/w/bin to take over an IP if a squid goes down.
  • To take down a virtual eth interface, /sbin/ifconfig eth0:n down
  • TODO: set up heartbeat for automatic IP takeover.

Logs

Currently on yongle in /var/backup/archiv, and/or on zwinger in /home/wikipedia/logs/archiv/

French squids

• 3 600MHz Celeron 1U machines with 20GB HDD and 128MB RAM (to be upgraded to 1GB?) - HP Web Hosting Server Appliance sa1100 - specs: [1]
• 2 serial ports, 2 NICs, no graphics card
• Debian
• names are chloe, bleuenn, and ennael
• to be located in a colo on Paris - provided free by http://www.lost-oasis.fr/
• serial cables running from ttyS1 on one machine to ttyS0 on another; getty running on ttyS0 on each machine, to allow communication using minicom

New squid setup

#!/bin/sh
echo 'getting squid from browne'
scp -r  root@browne.wikimedia.org:/usr/local/squid /usr/local
echo 'getting config'
scp -r  root@zwinger.wikimedia.org:/home/gwicke/squid/newsquid.conf /usr/local/squid/etc/squid.conf
echo 'create a shortcut to /usr/local/bin/squid'
ln -s /usr/local/bin/RunCache /usr/local/bin/squid
echo 'setting up the squid user'
adduser --system --no-create-home --disabled-login squid
echo 'setting up the squid group'
addgroup --system --no-create-home squid

echo 'creating /var/spool/squid and /var/log/squid'
mkdir /var/spool/squid
chown -R squid:squid /var/spool/squid
mkdir /var/log/squid
chown -R squid:squid /var/log/squid
mkdir /var/log/squid/outgoing
chown -R squid:squid /var/log/squid/outgoing
echo 'edit the configuration'
vim /usr/local/squid/etc/squid.conf
echo 'initializing cache dirs'
/usr/local/squid/sbin/squid -z

echo 'getting log rotation scripts'
scp root@zwinger.wikimedia.org:/home/gwicke/squid/crontab /root
scp root@browne.wikimedia.org:/usr/local/bin/preplogs_for_move /usr/local/bin/preplogs_for_move
scp root@browne.wikimedia.org:/etc/logrotate.d/squid /etc/logrotate.d/squid
crontab -u root /root/crontab

• log transfer from /var/log/squid/outgoing/* to /home/wikipedia/logs/incoming/{hostname}/