DNS

Revision as of 15:14, 13 February 2006

This page describes Wikimedia's DNS setup. Wikimedia use two separate kinds of DNS servers, authoritative nameservers (that respond to queries from third party nameservers for our domains) and resolvers (that resolve DNS queries for our own servers)

Authoritative nameservers

In the new DNS setup, Wikimedia have 3 authoritative DNS servers, all running PowerDNS. The three authoritative servers are:

• ns0.wikimedia.org - 207.142.131.207 (secondary IP on zwinger)
• ns1.wikimedia.org - 207.142.131.208 (larousse)
• ns2.wikimedia.org - 145.97.39.158 (secondary IP on pascal)

The servers are running with two PowerDNS backends each: the Bind backend (which emulates/reads Bind style zonefiles) and Geobackend (which is responsible for geographic DNS). The two backends are overlapping, meaning that for a given query the Bind backend is asked first, and if that declines to answer (because it doesn't have the requested record), the next backend, geobackend will be asked. Therefor, the usual contents of the zones are in regular Bind style zonefiles, and the geodns record rr.wikimedia.org. is added by geobackend.

Zwinger is the master nameserver, sort of, but the slaves are not using AXFR! Zonefiles and other configuration are replicated through the use of rsync in an update script.

All configuration files can be found in

/usr/local/etc/powerdns/

on all three hosts.

The main PowerDNS configuration file is /usr/local/etc/pdns.conf. Its configuration is documented on PowerDNS.

Additionally, there's a Bind backend configuration file, /usr/local/etc/bind.conf. It's compatible with Bind's own configuration format, but is only used to list the domains which the Bind backend has to serve. (Almost) all other options are ignored. In our setup, bind.conf is autogenerated from the domain templates.

Important differences from the previous DNS setup

• PowerDNS is used exclusively, instead of a mixed Bind/PowerDNS setup
• The wildcard records have been removed. This means that the zonefiles and/or /home/wikipedia/conf/langlist will need to be kept up to date!
• Geographic DNS and static DNS have been integrated into the same nameservers and wikimedia.org zone, which improves query latency because only a single query/response is needed.
• Zonefiles are generated from zone templates.
• No AXFR is being used; zones are replicated through rsync, and SOA serials are purely cosmetic.

Domain templates

Because Wikimedia have a lot of zones that essentially contain the same records (aliases for wikipedia.org and other projects), the old DNS setup used a single zonefile for multiple zones. That has the advantage that just a single change in a zonefile affects many zones. Unfortunately, it doesn't permit the use of $ORIGIN lines in the zonefile. In the new DNS setup, each zone gets its own zonefile, but multiple zonefiles can be generated from a single zone template.

The zone templates are (regular) files in

/usr/local/etc/powerdns/templates/

Each regular file in this directory corresponds to a zone with the same name. Each symbolic link to a regular file in this directory corresponds to a domain alias. So, in this example:

# ls -l templates/mediawiki*
lrwxrwxrwx    1 root     root           13 Jun 19 15:52 templates/mediawiki.com -> mediawiki.org
lrwxrwxrwx    1 root     root           13 Jun 19 15:52 templates/mediawiki.net -> mediawiki.org
-rw-r--r--    1 root     root         1500 Jun 19 15:12 templates/mediawiki.org

...one zone mediawiki.org is listed, with two alias zones, mediawiki.com and mediawiki.net.

Substitution variables

Within the zone template, a few predefined variables can be used, that will be substituted when the actual zonefiles are generated from the template. These variables include:

$zonename 

The actual zone qname (FQDN) of the zonefile to be generated

$serial 

The SOA serial number, derived from the current date and hour in YYYYMMDDHH format

$langlist 

A list of language subdomain CNAMEs, i.e. a list of all language abbreviations for all languages any Wikimedia project has, generated from /home/wikipedia/common/langlist.

gen-zones

The actual zonefiles are generated from the zone templates by a Python script, gen-zones. It simply reads all zone templates from the template directory, applies string substitutions, and writes the result to the

/usr/local/etc/powerdns/zones

directory, where PowerDNS can read them as regular zonefiles.

gen-bind.conf

gen-bind.conf is a Python script, that generates bind.conf by looking at the structure of the files and symlinks in /usr/local/etc/powerdns/templates/. For each regular file in that directory, it creates a corresponding block of zone statements for that zone and the zone aliases. For the example above, that would give rise to:

# mediawiki.org aliases
zone "mediawiki.com"    { type master;  file "mediawiki.com";   };
zone "mediawiki.net"    { type master;  file "mediawiki.net";   };
zone "mediawiki.org"    { type master;  file "mediawiki.org";   };

update

update is a simple shellscript, that automates the invocations of the scripts above. It goes through the following steps:

1. generation of a (new) list of language subdomain CNAMEs from /home/wikipedia/common/langlist
2. generation of the zonefiles from the zone templates
3. generation of bind.conf
4. reload of the local powerdns daemon (on zwinger)
5. synchronizing the slaves. for each slave:
   1. copying the langlist-cnames, zone templates and geomaps to the slave (using rsync)
   2. generation of the zonefiles on the slave
   3. generation of bind.conf on the slave
   4. reload of the remote powerdns daemon

Basically, update takes care of everything after you've edited the zonefiles.

Geographic DNS

Geographic DNS makes sure that clients end up using the Wikimedia cluster closest to them, by varying DNS responses based on the (country of the) resolver IP querying. Its configuration is still mostly the same as described on PowerDNS.

Geomaps are to be found in

/usr/local/etc/powerdns/geomaps

The IP->Country RBLDNS zonefile is located in

/usr/local/etc/powerdns/zz.countries.nerd.dk.rbldnsd

HOWTO

This section briefly explains how to do the most common DNS changes.

Changing records in a zonefile

1. Edit the template file /usr/local/etc/powerdns/templates/zonename on zwinger
2. Run /usr/local/etc/powerdns/update

Adding a new zone

1. First, decide if this new zone will use a new, independent zonefile, or will be an alias of another zone

   independent zonefile 

   Create the new zone template as /usr/local/etc/powerdns/templates/zonename (Copy an existing, relatively clean zonefile like wiktionary.org to start with).

   zone alias 

   Make a symbolic link /usr/local/etc/powerdns/templates/aliasname for the alias to the zone being aliased.

2. Run /usr/local/etc/powerdns/update

Removing a zone

1. Remove the corresponding file or symlink /usr/local/etc/powerdns/templates/zonename
2. Run /usr/local/etc/powerdns/update

Adding a new (language) wiki

1. Add the language code to </tt>/home/wikipedia/common/langlist</tt>
2. Run /usr/local/etc/powerdns/update

Resolvers

To be written. Is still using the old DNS setup.

srv1 and albert are the internal DNS resolvers, with srv1 being the master, and albert slaving. Both are running Bind (distribution RPMs), with /etc/named.conf as the configuration file, and /var/named (ns1) and /var/lib/named (albert) as zonefile directories. They have only 2 zones:

• wmnet. (internal wmnet. zone)
• 10.in-addr.arpa. (for 10.0.0.0/8 reverse DNS)

Old setup

Larousse, albert and zwinger are listed as authoritative name servers for *.pmtpa.wmnet. Larousse and zwinger are running PowerDNS. All three are running named. (BIND? version?)

Important files

/var/named/master/wmnet.zone 

This is the internal zone, it's the equivalent of /etc/hosts before we massacred that file

/var/named/master/wikimedia.zone 

This is the list of names available externally under *.wikimedia.org

Edit one of these files on zwinger, not forgetting to increment the SOA, and then restart with /etc/init.d/named restart.

Zones

*.pmtpa.wmnet 

10/8 addresses valid within our Tampa network

*.lopar.wmnet 

10/8 addresses valid within our Paris network

*.wikimedia.org 

Assorted external addresses from Tampa and Paris.

resolv.conf in Florida is set up to search *.pmtpa.wmnet then *.wikimedia.org. So if a machine does not have an internal address, the resolver automatically tries for an external address.