Varnish

Varnish is a fast caching proxy, and can be used as an alternative to Squid in a reverse caching accelerator setup.

We currently use Varnish for serving bits.wikimedia.org.

Configuration

Deployment of Varnish is done using Puppet, using class varnish in file manifests/varnish.pp.

We use the varnish package of Ubuntu/Debian, minimum version 2.1.2. Puppet installs this, and replaces its /etc/default/varnish file to set some startup parameters (discussed below). Varnish uses a VCL file (Varnish Configuration Language), a DSL where Varnish behaviour is controlled using subroutines that are compiled into C and executed during each request. The Wikimedia VCL file is

/etc/varnish/wikimedia.vcl

You also may want to read Bits varnish testing instead, for Domas his findings during a pilot project.

---

Mark intends to test Varnish for feasibility of using it instead of Squid for Wikimedia CDN purposes.

Things that need special consideration

• HTCP purging
• Immediate purging of cache objects (nuke?)
• Header normalization (Host, Accept-Encoding...)
• Two-layer setup (CARP style)
• Compatible logging
• Request stats

Would be nice

• SSL
• IPv6

Many of these are probably already taken care of by our friends at Wikia, and therefore possibly also within Varnish itself...

External links

• Varnish main web site
• OSCON presentation about Varnish at Wikia by Arthur Bergman, containing useful information about performance tuning and some neat features as well.