Offsite Backups (old)

From Wikitech
(Difference between revisions)
Jump to: navigation, search
m (Status)
(Amazon)
 
(11 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
{{Old-doc}}
 
Wikimedia is starting to use external partners to allow for better disaster recovery and quicker response time in case of a data center wide failure. The first large scale case use is being proxied thorough a company named '''eBart'''.
 
Wikimedia is starting to use external partners to allow for better disaster recovery and quicker response time in case of a data center wide failure. The first large scale case use is being proxied thorough a company named '''eBart'''.
  
 
= eBart =
 
= eBart =
  
* Hostname: '''wintermute.ebart.co.rs / wm''' (has to be accessed through a bastion host for now)
+
* Host: '''wm.arhiv.rs'''
 +
* Port: '''60222'''
 +
* Alternate entry: '''wintermute.ebart.co.rs''' -> '''wm'''
 
* Primary Contact: '''Milos Rancic''' millosh@gmail.com
 
* Primary Contact: '''Milos Rancic''' millosh@gmail.com
 
** Phone: '''Buisness''' +38164822939 '''Personal''' +381656455674
 
** Phone: '''Buisness''' +38164822939 '''Personal''' +381656455674
* Secondary Contact '''Branislav Jovanovic''' branej@gmail.com
+
* Secondary Contact: '''Branislav Jovanovic''' branej@gmail.com
 
** Phone: +381604828520
 
** Phone: +381604828520
* Tertiary Contact '''Nikola Blagojevic''' nblagojevic@arhiv.rs
+
* Tertiary Contact: '''Nikola Blagojevic''' nblagojevic@arhiv.rs
** Phone: 381648220904
+
** Phone: +381648220904
 
* User access: '''tomasz'''
 
* User access: '''tomasz'''
 
* Type of access: '''ssh'''
 
* Type of access: '''ssh'''
Line 23: Line 26:
 
=== Status ===
 
=== Status ===
  
I'm currently pushing our archives of 2008 XML snapshots as the first set of speed tests. After those are in place we'll come up how to organize this in a more automated fashion. --[[User:Tomasz Finc|Tomasz Finc]] 18:24, 8 July 2009 (UTC)
+
:All the 2008 archives are now moved over. --[[User:Tomasz Finc|Tomasz Finc]] 18:10, 30 July 2009 (UTC)
:Were seeing fairly consistent speed when doing these transfers. A solid and steady 1.6MB/sec (16Mb/sec) going from amane to wm. --[[User:Tomasz Finc|Tomasz Finc]] 20:25, 13 July 2009 (UTC)  
+
:Started to push [0-9]* & [a-z]* of Commons images --[[User:Tomasz Finc|Tomasz Finc]] 20:35, 10 September 2009 (UTC)
 +
:The older archives from brion's drive (2002, some 2003, 2005 and 2006 dumps) are on the download server in a private directory and on a hard drive in a cabinet in Tampa. [[User:ArielGlenn|ArielGlenn]] 17:09, 20 January 2012 (UTC)
  
Moving forward I'm going to push some more archives from Brion's archived drive. Since the Wikimedia Office has a fairly slow uplink, I'll be using my own link at home which can sustain an upload of 250kB/s (2Mb/sec). --[[User:Tomasz Finc|Tomasz Finc]] 20:25, 13 July 2009 (UTC)
+
= MIT =
:I'll do an archive of status successful and transfer those first. Any others will be judged by hand to see if they are worth archiving. --[[User:Tomasz Finc|Tomasz Finc]] 20:25, 13 July 2009 (UTC)
+
 
:: Still moving these archives over. Just moved past frwiki on brions old drive while I'm at svwiki from our 2008 archives --[[User:Tomasz Finc|Tomasz Finc]] 22:20, 27 July 2009 (UTC)
+
Samuel Klein is researching for dedicated space at the Harvard-MIT Data Center.
 +
 
 +
= Stanford =
 +
 
 +
Kul Wadha is exploring storage options at Stanford.
 +
[[Category:Risk management]]
 +
 
 +
= Amazon =
 +
 
 +
We've pushed our initial data set into [[Amazon_Public_Data_Sets|Amazon Public Data Sets]] and are awaiting legal and internal consistency sign offs.
 +
 
 +
OK this is very very old and not at all part of any ongoing plan. We're working with amazon on some data for ec2 users but that's it.
 +
 
 +
[[Category:Dumps]]

Latest revision as of 19:50, 29 October 2012

This page page may be outdated and could contain incorrect details. Please update it if you can.

Wikimedia is starting to use external partners to allow for better disaster recovery and quicker response time in case of a data center wide failure. The first large scale case use is being proxied thorough a company named eBart.

Contents

[edit] eBart

  • Host: wm.arhiv.rs
  • Port: 60222
  • Alternate entry: wintermute.ebart.co.rs -> wm
  • Primary Contact: Milos Rancic millosh@gmail.com
    • Phone: Buisness +38164822939 Personal +381656455674
  • Secondary Contact: Branislav Jovanovic branej@gmail.com
    • Phone: +381604828520
  • Tertiary Contact: Nikola Blagojevic nblagojevic@arhiv.rs
    • Phone: +381648220904
  • User access: tomasz
  • Type of access: ssh
  • Space: 6.9T

The plan for this is to have a snapshot of Commons available offsite if anything should happen to our data center. The total allotted space should reach upwards of 10T's in the next months as more drives are added to the system. Ideally this off site will host both a snapshot of Commons and possibly an archive of the XML snapshots.

Notes on coverage:

  • Between 10 and 17 CET/CEST (between 08-09 and 15-16 UTC) ability to reset the machine quickly.
  • Between 07 and 01 CET/CEST (between 05-06 and 23-00 UTC) ability to contact other staff with a 20 minute window to reboot.

[edit] Status

All the 2008 archives are now moved over. --Tomasz Finc 18:10, 30 July 2009 (UTC)
Started to push [0-9]* & [a-z]* of Commons images --Tomasz Finc 20:35, 10 September 2009 (UTC)
The older archives from brion's drive (2002, some 2003, 2005 and 2006 dumps) are on the download server in a private directory and on a hard drive in a cabinet in Tampa. ArielGlenn 17:09, 20 January 2012 (UTC)

[edit] MIT

Samuel Klein is researching for dedicated space at the Harvard-MIT Data Center.

[edit] Stanford

Kul Wadha is exploring storage options at Stanford.

[edit] Amazon

We've pushed our initial data set into Amazon Public Data Sets and are awaiting legal and internal consistency sign offs.

OK this is very very old and not at all part of any ongoing plan. We're working with amazon on some data for ec2 users but that's it.

Personal tools
Namespaces

Variants
Actions
Navigation
Ops documentation
Wiki
Toolbox