Dumps/Dump servers
ArielGlenn (Talk | contribs) (gah why do we have this crazy naming scheme...) |
ArielGlenn (Talk | contribs) |
||
| Line 1: | Line 1: | ||
==XML Dump servers== | ==XML Dump servers== | ||
| + | |||
| + | ===Hardware=== | ||
We have three hosts: | We have three hosts: | ||
| Line 8: | Line 10: | ||
*:Note that this host also serves other public datasets such as some POTY files, the pagecount stats, etc. | *:Note that this host also serves other public datasets such as some POTY files, the pagecount stats, etc. | ||
* [[Dataset1001]] in D.C, waiting for setup and installation | * [[Dataset1001]] in D.C, waiting for setup and installation | ||
| + | |||
| + | ===Services=== | ||
| + | |||
| + | The production host serves dump files and other public data sets to the public. | ||
| + | |||
| + | It relies on lighttpd. Sometimes this service dies for no good reason. To restart it, | ||
| + | |||
| + | <code>/etc/init.d/lighttpd restart</code> | ||
| + | |||
| + | ===Deploying a new host=== | ||
| + | |||
| + | You'll need to set up the raid arrays by hand. We typically have two arrays so set up two raid 6 arrays with LVM to make one giant volume, xfs. | ||
| + | |||
| + | Install in the usual way (add to puppet, copying a pre-existing production dataset host stanza, set up everything for PXE boot and go). You may or may not want to include the download mirror classes from puppet for the new host. If you replace the host that is the current download mirror, make sure you tweak the cron job that generates the mirror file list, see [[Dumps/Snapshot hosts#Other_tasks]] for that and other jobs you might need to check. | ||
| + | |||
| + | ===Space issues=== | ||
| + | |||
| + | If we run low on space, we can keep fewer rounds of XML dumps; see [[Dumps#Space]] for how to do that. | ||
| + | |||
[[Category:Dumps]] | [[Category:Dumps]] | ||
Revision as of 17:20, 20 January 2012
Contents |
XML Dump servers
Hardware
We have three hosts:
- Dataset1 in Tampa, out of operation for over a year, still working with the vendor
- Dataset2 in Tampa, production: PowerEdge R410, Ubuntu 10.04, 2 MD1000 arrays, 16GB RAM, 4 6-core Xeon X5650 cpus
- Disk information: 144GB on the internal HDs with raid 1, 48T on the arrays with two raid 6 partitions set up as one LVM
- Note that this host also serves other public datasets such as some POTY files, the pagecount stats, etc.
- Dataset1001 in D.C, waiting for setup and installation
Services
The production host serves dump files and other public data sets to the public.
It relies on lighttpd. Sometimes this service dies for no good reason. To restart it,
/etc/init.d/lighttpd restart
Deploying a new host
You'll need to set up the raid arrays by hand. We typically have two arrays so set up two raid 6 arrays with LVM to make one giant volume, xfs.
Install in the usual way (add to puppet, copying a pre-existing production dataset host stanza, set up everything for PXE boot and go). You may or may not want to include the download mirror classes from puppet for the new host. If you replace the host that is the current download mirror, make sure you tweak the cron job that generates the mirror file list, see Dumps/Snapshot hosts#Other_tasks for that and other jobs you might need to check.
Space issues
If we run low on space, we can keep fewer rounds of XML dumps; see Dumps#Space for how to do that.
