Swift/Deploy Plan - R720xds in tampa

From Wikitech
< Swift(Difference between revisions)
Jump to: navigation, search
(Created page with "Draft, edit at will ===Current status=== *Currently replaced: ms-be6,7,8,10 *Hosts with disk issues (so they should be replaced next): ms-be5,11,12 *Other hosts with ssds: ms...")
 
Line 3: Line 3:
 
===Current status===
 
===Current status===
 
*Currently replaced: ms-be6,7,8,10
 
*Currently replaced: ms-be6,7,8,10
 +
*Host which has fallen over a few times recently: ms-be3
 
*Hosts with disk issues (so they should be replaced next): ms-be5,11,12
 
*Hosts with disk issues (so they should be replaced next): ms-be5,11,12
 
*Other hosts with ssds: ms-be9
 
*Other hosts with ssds: ms-be9
*The rest: ms-be1-4
+
*The rest: ms-be1,2,4
  
 
===Proposed schedule===
 
===Proposed schedule===
Line 13: Line 14:
 
We should do these no more than two at a time.  Schedule for first deployment in cluster with 33% weight:
 
We should do these no more than two at a time.  Schedule for first deployment in cluster with 33% weight:
  
*ms-be5 and ms-be11 on Thursday Nov 29
+
*ms-be3 and ms-be5 on Thursday Nov 29
*ms-12 and ms-be9 on Dec 3 or 4
+
*ms-be5 and ms-be12 on Dec 3 or 4
*ms-be1 and ms-be2 on Dec 7
+
 
*me be-3 and ms-be4 on Dec 12
+
Replace these but turn them into ceph hosts:
 +
 
 +
*me-be9 and ms-be1 on Dec 7
 +
*me be-2 and ms-be4 on Dec 12
  
 
This schedule will be adjusted (shorter or longer) as we see how the cluster behaves.
 
This schedule will be adjusted (shorter or longer) as we see how the cluster behaves.
  
 
===Logistics===
 
===Logistics===
What is our deadline for returning the C2100s?
+
*What is our deadline for returning the C2100s?
 +
*Do we have enough ssds to put in all remaining 720xds?  If not let's get them (and what's the turnaround time for those?)

Revision as of 11:20, 27 November 2012

Draft, edit at will

Current status

  • Currently replaced: ms-be6,7,8,10
  • Host which has fallen over a few times recently: ms-be3
  • Hosts with disk issues (so they should be replaced next): ms-be5,11,12
  • Other hosts with ssds: ms-be9
  • The rest: ms-be1,2,4

Proposed schedule

Pushing our luck a little bit, we can go from 33 weight for the object rings to 66 in 3 days and from 66 to 100 in another 4. This is a compressed schedule, it really doesn't let network traffic to the new hosts settle down completely, but it will let us get the new boxes in and the old ones shipped.

We should do these no more than two at a time. Schedule for first deployment in cluster with 33% weight:

  • ms-be3 and ms-be5 on Thursday Nov 29
  • ms-be5 and ms-be12 on Dec 3 or 4

Replace these but turn them into ceph hosts:

  • me-be9 and ms-be1 on Dec 7
  • me be-2 and ms-be4 on Dec 12

This schedule will be adjusted (shorter or longer) as we see how the cluster behaves.

Logistics

  • What is our deadline for returning the C2100s?
  • Do we have enough ssds to put in all remaining 720xds? If not let's get them (and what's the turnaround time for those?)
Personal tools
Namespaces

Variants
Actions
Navigation
Ops documentation
Wiki
Toolbox