Squids
From Wikitech
(Difference between revisions)
(log rotation setup, transfer details) |
(updates) |
||
| Line 1: | Line 1: | ||
| − | + | __TOC__ | |
| + | |||
| + | * 4 squids - using 11 IPs, to allow fairly even load-balancing if 1 or 2 squids go down | ||
| + | |||
* to add new IPs for squid: | * to add new IPs for squid: | ||
** add the IPs in /var/named/master/wikipedia.zone in the obvious places (on zwinger) | ** add the IPs in /var/named/master/wikipedia.zone in the obvious places (on zwinger) | ||
** make sure to update the serial number in the zone file! | ** make sure to update the serial number in the zone file! | ||
** run <tt>rndc reload</tt> to make it update - no need to restart named | ** run <tt>rndc reload</tt> to make it update - no need to restart named | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
* some useful commands: | * some useful commands: | ||
** dig ANY en.wikipedia.org @zwinger.wikipedia.org | ** dig ANY en.wikipedia.org @zwinger.wikipedia.org | ||
** dig +short ANY en.wikipedia.org | ** dig +short ANY en.wikipedia.org | ||
| − | |||
| − | |||
| − | |||
| − | |||
| + | ==IPs of virtual ethernet interfaces== | ||
browne: 207.142.131.235, 207.142.131.247 | browne: 207.142.131.235, 207.142.131.247 | ||
| − | coronelli: 207.142.131.236, 207.142.131.248 | + | coronelli: 207.142.131.236, 207.142.131.248, 207.142.131.205 |
| − | maurus: 207.142.131.245, 207.142.131.246 | + | maurus: 207.142.131.245, 207.142.131.246, 207.142.131.206 |
| + | rabanus: 207.142.131.202, 207.142.131.203, 207.142.131.204 | ||
* According to damyta, should use 255.255.255.255 for netmask for the IPs of virtual eth interfaces - otherwise there can be routing confusion. | * According to damyta, should use 255.255.255.255 for netmask for the IPs of virtual eth interfaces - otherwise there can be routing confusion. | ||
| − | * You must ping through the switch via a virtual interface which you have just brought up (using ping -I), to update the switch's idea of who has what IP. When pinging with -I207.142.131.248 or similar, use ''suda'' as the ping target; pinging the broadcast address ''does not always work''. | + | ** We've been using this configuration for months now and it works fine. -- [[User:Jeronim|Jeronim]] 10:41, 19 Sep 2004 (UTC) |
| − | ** Can use script ''takeip'' in / | + | * <strike>You must ping through the switch via a virtual interface which you have just brought up (using ping -I), to update the switch's idea of who has what IP. When pinging with -I207.142.131.248 or similar, use ''suda'' as the ping target; pinging the broadcast address ''does not always work''.</strike> |
| + | ** Can use script ''takeip'' in /home/wikipedia/bin to take over an IP if a squid goes down. | ||
** To take down a virtual eth interface, <tt>/sbin/ifconfig eth0:n down</tt> | ** To take down a virtual eth interface, <tt>/sbin/ifconfig eth0:n down</tt> | ||
** TODO: set up ''heartbeat'' for automatic IP takeover. | ** TODO: set up ''heartbeat'' for automatic IP takeover. | ||
| Line 48: | Line 45: | ||
echo 'create a shortcut to /usr/local/bin/squid' | echo 'create a shortcut to /usr/local/bin/squid' | ||
ln -s /usr/local/bin/RunCache /usr/local/bin/squid | ln -s /usr/local/bin/RunCache /usr/local/bin/squid | ||
| − | echo 'setting up the squid user' | + | #squid user/group is already existing on all servers |
| − | adduser --system --no-create-home --disabled-login squid | + | #echo 'setting up the squid user' |
| − | echo 'setting up the squid group' | + | #adduser --system --no-create-home --disabled-login squid |
| − | addgroup --system --no-create-home squid | + | #echo 'setting up the squid group' |
| + | #addgroup --system --no-create-home squid | ||
echo 'creating /var/spool/squid and /var/log/squid' | echo 'creating /var/spool/squid and /var/log/squid' | ||
| Line 69: | Line 67: | ||
scp root@browne.wikimedia.org:/usr/local/bin/preplogs_for_move /usr/local/bin/preplogs_for_move | scp root@browne.wikimedia.org:/usr/local/bin/preplogs_for_move /usr/local/bin/preplogs_for_move | ||
scp root@browne.wikimedia.org:/etc/logrotate.d/squid /etc/logrotate.d/squid | scp root@browne.wikimedia.org:/etc/logrotate.d/squid /etc/logrotate.d/squid | ||
| − | crontab -u root /root/crontab | + | # don't overwrite root's crontab, as there maybe something important in there |
| + | # crontab -u root /root/crontab | ||
</pre> | </pre> | ||
* log transfer from /var/log/squid/outgoing/* to /home/wikipedia/logs/incoming/{hostname}/ | * log transfer from /var/log/squid/outgoing/* to /home/wikipedia/logs/incoming/{hostname}/ | ||
Revision as of 10:41, 19 September 2004
Contents |
- 4 squids - using 11 IPs, to allow fairly even load-balancing if 1 or 2 squids go down
- to add new IPs for squid:
- add the IPs in /var/named/master/wikipedia.zone in the obvious places (on zwinger)
- make sure to update the serial number in the zone file!
- run rndc reload to make it update - no need to restart named
- some useful commands:
- dig ANY en.wikipedia.org @zwinger.wikipedia.org
- dig +short ANY en.wikipedia.org
IPs of virtual ethernet interfaces
browne: 207.142.131.235, 207.142.131.247 coronelli: 207.142.131.236, 207.142.131.248, 207.142.131.205 maurus: 207.142.131.245, 207.142.131.246, 207.142.131.206 rabanus: 207.142.131.202, 207.142.131.203, 207.142.131.204
- According to damyta, should use 255.255.255.255 for netmask for the IPs of virtual eth interfaces - otherwise there can be routing confusion.
- We've been using this configuration for months now and it works fine. -- Jeronim 10:41, 19 Sep 2004 (UTC)
-
You must ping through the switch via a virtual interface which you have just brought up (using ping -I), to update the switch's idea of who has what IP. When pinging with -I207.142.131.248 or similar, use suda as the ping target; pinging the broadcast address does not always work.- Can use script takeip in /home/wikipedia/bin to take over an IP if a squid goes down.
- To take down a virtual eth interface, /sbin/ifconfig eth0:n down
- TODO: set up heartbeat for automatic IP takeover.
Logs
Currently on yongle in /var/backup/archiv, and/or on zwinger in /home/wikipedia/logs/archiv/
French squids
- 3 600MHz Celeron 1U machines with 20GB HDD and 128MB RAM (to be upgraded to 1GB?) - HP Web Hosting Server Appliance sa1100 - specs: [1]
- 2 serial ports, 2 NICs, no graphics card
- Debian
- names are chloe, bleuenn, and ennael
- to be located in a colo on Paris - provided free by http://www.lost-oasis.fr/
- serial cables running from ttyS1 on one machine to ttyS0 on another; getty running on ttyS0 on each machine, to allow communication using minicom
New squid setup
#!/bin/sh echo 'getting squid from browne' scp -r root@browne.wikimedia.org:/usr/local/squid /usr/local echo 'getting config' scp -r root@zwinger.wikimedia.org:/home/gwicke/squid/newsquid.conf /usr/local/squid/etc/squid.conf echo 'create a shortcut to /usr/local/bin/squid' ln -s /usr/local/bin/RunCache /usr/local/bin/squid #squid user/group is already existing on all servers #echo 'setting up the squid user' #adduser --system --no-create-home --disabled-login squid #echo 'setting up the squid group' #addgroup --system --no-create-home squid echo 'creating /var/spool/squid and /var/log/squid' mkdir /var/spool/squid chown -R squid:squid /var/spool/squid mkdir /var/log/squid chown -R squid:squid /var/log/squid mkdir /var/log/squid/outgoing chown -R squid:squid /var/log/squid/outgoing echo 'edit the configuration' vim /usr/local/squid/etc/squid.conf echo 'initializing cache dirs' /usr/local/squid/sbin/squid -z echo 'getting log rotation scripts' scp root@zwinger.wikimedia.org:/home/gwicke/squid/crontab /root scp root@browne.wikimedia.org:/usr/local/bin/preplogs_for_move /usr/local/bin/preplogs_for_move scp root@browne.wikimedia.org:/etc/logrotate.d/squid /etc/logrotate.d/squid # don't overwrite root's crontab, as there maybe something important in there # crontab -u root /root/crontab
- log transfer from /var/log/squid/outgoing/* to /home/wikipedia/logs/incoming/{hostname}/
