Dumps/Snapshot hosts
From Wikitech
< Dumps(Difference between revisions)
ArielGlenn (Talk | contribs) (Created page with "==Snapshot (XML dumps generation) cluster information== ===Hardware=== We have two mini snapshot clusters. In Tampa: *snapshot1: '''operational''', PowerEdge 1950, Ubuntu 10...") |
ArielGlenn (Talk | contribs) |
||
| Line 18: | Line 18: | ||
*snapshot1003: waiting for drac issue fix, PowerEdge R410, Ubuntu 10.04, 16GB RAM, 2 6-core Xeons, 500GB HD | *snapshot1003: waiting for drac issue fix, PowerEdge R410, Ubuntu 10.04, 16GB RAM, 2 6-core Xeons, 500GB HD | ||
*snapshot1004: waiting for drac issue fix, PowerEdge R410, Ubuntu 10.04, 16GB RAM, 2 6-core Xeons, 500GB HD | *snapshot1004: waiting for drac issue fix, PowerEdge R410, Ubuntu 10.04, 16GB RAM, 2 6-core Xeons, 500GB HD | ||
| + | |||
| + | Ordinarily only one cluster will be running dump jobs at a time; the other is on standby in case of various failures. The two beefier servers (with 4 8-core cpus) are dedicated machines for the en wikipedia dumps; as with the other hosts, one of them is in opertion and the other is in standby. | ||
===Currently running=== | ===Currently running=== | ||
| Line 28: | Line 30: | ||
* [[snapshot4]] -- running en wiki dumps via | * [[snapshot4]] -- running en wiki dumps via | ||
*: <code>python ./worker.py --configfile wikidump.conf.enwiki enwiki</code> | *: <code>python ./worker.py --configfile wikidump.conf.enwiki enwiki</code> | ||
| + | |||
| + | [[Category:Dumps]] | ||
Revision as of 18:06, 19 January 2012
Snapshot (XML dumps generation) cluster information
Hardware
We have two mini snapshot clusters.
In Tampa:
- snapshot1: operational, PowerEdge 1950, Ubuntu 10.04, 8GB RAM, 2 quad-core Xeons, 80GB HD
- snapshot2: operational, PowerEdge 1950, Ubuntu 10.04, 8GB RAM, 2 quad-core Xeons, 80GB HD
- snapshot3: operational, PowerEdge 1950, Ubuntu 10.04, 8GB RAM, 2 quad-core Xeons, 80GB HD
- snapshot4: operational, PowerEdge R815, Ubuntu 10.04, 8GB RAM, 4 8-core Opterons, 2 80GB HDs
In D.C.:
- snapshot1001: base install done, PowerEdge R815, Ubuntu 10.04, 64GB RAM, 4 8-core Opterons, 2 80GB HDs
- snapshot1002: base install done, PowerEdge R410, Ubuntu 10.04, 16GB RAM, 2 6-core Xeons, 500GB HD
- snapshot1003: waiting for drac issue fix, PowerEdge R410, Ubuntu 10.04, 16GB RAM, 2 6-core Xeons, 500GB HD
- snapshot1004: waiting for drac issue fix, PowerEdge R410, Ubuntu 10.04, 16GB RAM, 2 6-core Xeons, 500GB HD
Ordinarily only one cluster will be running dump jobs at a time; the other is on standby in case of various failures. The two beefier servers (with 4 8-core cpus) are dedicated machines for the en wikipedia dumps; as with the other hosts, one of them is in opertion and the other is in standby.
Currently running
- snapshot1 -- currently running 3 worker processes for bigger wikis out of /backups-atg, via
python ./worker wikidump.conf.bigwikis
- snapshot2 -- running 4 processes for small wikis out of /backups-atg, via
-
./worker
-
- snapshot3 -- runs adds/changes dumps from cron
- snapshot4 -- running en wiki dumps via
-
python ./worker.py --configfile wikidump.conf.enwiki enwiki
-
