Network design
(Update for new racks in pmtpa) |
|||
| Line 17: | Line 17: | ||
=== Florida cluster (PMTPA) === | === Florida cluster (PMTPA) === | ||
| − | Wikimedia servers reside in | + | [[Image:floridaserversfront1.jpg|thumb]] [[Image:tampa topology.png|thumb]] |
| + | Wikimedia servers reside in seven racks along with Bomis servers, hosted at [http://powermedium.com/ PowerMedium]. Wikimedia/Bomis have a dedicated IP range, <tt>207.142.131.192/26</tt>. There are two gateways: <tt>207.142.131.193</tt> and <tt>207.142.131.225</tt>. Total burstable bandwidth is 2000 Mbit/s, delivered through two optic fiber 1000base-SX links. | ||
| − | Wikimedia owns | + | Wikimedia owns two Cisco C3560G-48TS switchs, three Netgear Gigabit [[switches]], and a Cisco 2950 switch. |
=== Amsterdam cluster (KNAMS) === | === Amsterdam cluster (KNAMS) === | ||
Revision as of 01:57, 29 November 2005
The purpose of this page is to give an overview of the current design of the network of the Wikimedia servers, and to provide a place to develop a new and improved network scheme.
Contents |
Overall system design
The following is the general system design plan which the network layer must efficiently accommodate.
- Databases in a central pool with each serving a subset of the wikis, so each has high cache efficiency and the total number needed to handle any query load is minimised. Database servers cost US$5,000-$8,000 each, depending on exact equipment.
- A central pair of old text database servers (part of the long term storage growth plan for the databases, to move this high volume and seldom accessed data off costly and comparatively small disk systems).
- Memcached caching spread on apaches across the whole cluster, producing one very large cache pool, accessible from any apache and stored on half or more of the apaches. Segmenting the pool would decrease the overall hit rate, increasing the number of apaches and database servers required for any given system load level.
- Load balancing of squids and apaches, currently expected to use two or three systems between the internet and the squids and the same set between the squids and the apaches.
A key network systems design requirement is efficient access from any apache to any apache running memcached (expected to be more than half of all apaches) and efficient access from any apache to any database server. Losing this capability would dramatically increase overall system cost.
Current situation
The cluster names are a two character code for the colo provider, and a 3 character code for the city location. (Candidhosting is now Power Medium :-).
Florida cluster (PMTPA)
Wikimedia servers reside in seven racks along with Bomis servers, hosted at PowerMedium. Wikimedia/Bomis have a dedicated IP range, 207.142.131.192/26. There are two gateways: 207.142.131.193 and 207.142.131.225. Total burstable bandwidth is 2000 Mbit/s, delivered through two optic fiber 1000base-SX links.
Wikimedia owns two Cisco C3560G-48TS switchs, three Netgear Gigabit switches, and a Cisco 2950 switch.
Amsterdam cluster (KNAMS)
The Kennisnet cluster's network follows a similar design as the Florida cluster, with one Cisco 3560G-24 core switch connected via a Gigabit ethernet port in routed mode to the uplink Kennisnet router. The L3 switch routes between "The Internet" and the Wikimedia VLANs:
- VLAN 100
- Public VLAN (145.97.39.128/27)
- VLAN 101
- Private VLAN (10.20.1.0/24)
- VLAN 10
- Installation VLAN
There is a separate management network (145.97.34.224/29), with a separate uplink connected to a Kennisnet firewall for out of band access. It's connected to the Service Processors of all Sun servers in a daisy chain.
