Switch master

From Wikitech
Revision as of 19:25, 29 October 2010 by ArielGlenn (Talk | contribs)

Jump to: navigation, search

The easy way

cd /home/wikipedia/src/mediawiki/tools/switch-master
php switch.php <old master> <new master>
cd /home/wikipedia/conf/nagios
./sync

The hard way

The script above (switch.php) does the following tasks, which you can do manually if you feel that way inclined:

  • Check that the new master has binlog enabled. All slaves should be configured this way, for convenience. All servers, including masters, should have read_only mode on by default.
  • Run RESET MASTER on the new master
  • Log in to the mysql on the old master as root. Check for any long running queries. Kill them. If there is a single long-running query such as a backup, which will restart after you kill it, then get its thread ID ready in a kill command, ready to press enter later.
  • Set read-only mode in db.php, sync it.
  • Set read-only mode on the old master using SET GLOBAL read_only=1;
  • Run FLUSH TABLES on the old master. This will block until all queries complete, so you may need to kill some now.
  • Check for slave lag. Slave lag must be zero on the new master before you continue.
  • Run STOP SLAVE on the new master
  • Run SHOW MASTER STATUS on the new master to get the new replication position
  • Run a CHANGE MASTER TO query on each of the slaves including the old master
  • Set read/write mode on the new master using SET GLOBAL read_only=0;
  • Edit db.php again, turning off read-only mode and simultaneously switching the loads configuration.
  • Run RESET SLAVE on the new master, to prevent it from replicating the old master after restart.

What could be easier than managing a replicated cluster in MySQL?

Personal tools
Namespaces

Variants
Actions
Navigation
Ops documentation
Wiki
Toolbox