Squid logging

Revision as of 05:03, 7 August 2010

Logging packets are sent from the squids to the logging server (locke)

Squid side

We use our own UDP logging patch, described here. It is in the Wikimedia APT repository in squid_2.6.9-1wm1. Log lines are buffered and sent to locke in 1450-byte packets.

• Squid log format
• Currently aggregated on locke

Aggregation

UDP packets are sent from all squids to the log host locke.wikimedia.org. A program called udp2log runs there. udp2log logs to an arbitrary number of destination files or pipes, specified in /etc/udp2log. The configuration file format is line based, with each line containing:

pipe <factor> <command line>

or

file <factor> <filename>

Unlike the squid pipe log, these pipes are full shell commands and may contain spaces. There is no quoting, the command goes from after the factor to the end of the line.

<factor> is an integer sampling factor, one in every <factor> packets will be sent to the designated destination.

Comments are lines starting with "#".

Reconfiguration can be done by sending a HUP signal to udp2log. The sequence of events when HUP is received are as follows:

1. HUP is received, a flag is set
2. Wait for the next UDP packet
3. Load the new configuration, open all files and pipes specified
4. If all pipes and files were successfully opened, swap in the configuration and close all previously opened pipes and files. Otherwise, close the new pipes and files and retain the old configuration.
5. Process the received packet using the new configuration
6. Resume the recv() loop

This means that pipe scripts should not obtain any lock on a shared resource until after the first line is received on stdin.

udp2log is designed to run as a daemon for long periods of time with only HUP reloads, not restarts. Restarts should be avoided because they cause packet loss. I haven't quite gotten around to writing the daemonize code yet, so for now it runs in a root screen. It runs as the user "logger".

udp2log is available in svn (http://svn.wikimedia.org/svnroot/mediawiki/trunk/udplog), and in the APT repository in the udplog package.

Log files

Log files are stored in /a/squid.

sampled-1000.log 

A 1/1000 sampled log of all requests

support-requests.log 

Full log click through stats for fund raising stats. Needs to be retained.

This section should be updated with details of currently collected log files and aggregation data.