Volunteer Squid Sites
(bandwidth issues) |
(→Bandwidth) |
||
| Line 5: | Line 5: | ||
Wikipedia is a rather large site. [http://www.alexa.com Alexa] puts it in the 100 most frequented English-language sites, and 200 most frequented sites in the World. This means that it has requirements that exceed those of most sites, in terms of bandwidth and reliability. | Wikipedia is a rather large site. [http://www.alexa.com Alexa] puts it in the 100 most frequented English-language sites, and 200 most frequented sites in the World. This means that it has requirements that exceed those of most sites, in terms of bandwidth and reliability. | ||
| − | As of January 20, 2004, the Wikimedia squids located near Paris and serving French and English-language content, as well as multimedia content, to French, British, Belgian, Luxembourger and Swiss users have had outgoing traffic up to 8 Mbit/s, but this bar is likely to increase. The main Squid cluster in Florida currently has peaks to | + | As of January 20, 2004, the Wikimedia squids located near Paris and serving French and English-language content, as well as multimedia content, to French, British, Belgian, Luxembourger and Swiss users have had outgoing traffic up to 8 Mbit/s, but this bar is likely to increase. The main Squid cluster in Florida currently has peaks to 95 Mbit/s. |
This indicates that a site serving a country the size of France (about 60 million inhabitants) should be able to provide 5 Mbits/s outgoing bandwidth (and allow for larger spikes). | This indicates that a site serving a country the size of France (about 60 million inhabitants) should be able to provide 5 Mbits/s outgoing bandwidth (and allow for larger spikes). | ||
Revision as of 10:21, 22 January 2005
This page is to provide a place for Wikipedia staff to enumerate the questions and requirements that an organization volunteering to host a remote Squid cache site (a VSS) needs to consider before proceeding, and to capture the contact and other information from such volunteers until we can make use of it.
Questions
Bandwidth
Wikipedia is a rather large site. Alexa puts it in the 100 most frequented English-language sites, and 200 most frequented sites in the World. This means that it has requirements that exceed those of most sites, in terms of bandwidth and reliability.
As of January 20, 2004, the Wikimedia squids located near Paris and serving French and English-language content, as well as multimedia content, to French, British, Belgian, Luxembourger and Swiss users have had outgoing traffic up to 8 Mbit/s, but this bar is likely to increase. The main Squid cluster in Florida currently has peaks to 95 Mbit/s.
This indicates that a site serving a country the size of France (about 60 million inhabitants) should be able to provide 5 Mbits/s outgoing bandwidth (and allow for larger spikes).
Another issue is not only available bandwidth, but its possible costs to the donor. Certain organizations pay more for traffic for certain destinations (use of "transit" traffic instead of peering, etc.). These organizations may thus for instance prefer that the installed servers serve the networks that they have free or cheap traffic to; they should clearly delineate their choices.
Some donor organizations may be tempted to implement "rate limiters" to certain or all destinations, so as to limit costs. In our experience, this is a very bad idea, and we would considerably prefer that these organizations give us clear maximal rates which we would adhere to. Rate limiters are a terrible solution:
- They give the false impression to users that there is something wrong with the network or servers, without any clear feedback except long "ping" times and unreachable servers.
- They prevent redundancy with respect to network connections. Let us assume that our cache H is linked to the networks it normally serves through an unlimited link U, and to other networks using a limited link L. U, for some reason, fails or is being serviced; traffic is redirected through L. Instantaneously, the users experience extreme slowness or an unreachable site (this is based on real experience).
So, please, don't try to implement rate limiters; instead, discuss your bandwidth issues with Wikimedia.
Contact
Put your name, email address, organization name, and the size and average usage percentage of your uplink to the Internet here; as we work out our VSS protocol and implementation, we'll get back to you.
