Switch master
From Wikitech
(Difference between revisions)
(two more steps following observed pitfall) |
|||
| Line 7: | Line 7: | ||
* Edit config.s1, config.s2 or config.s3 depending on which master you are switching. Then copy it to config. Config is the file that is actually used. | * Edit config.s1, config.s2 or config.s3 depending on which master you are switching. Then copy it to config. Config is the file that is actually used. | ||
** ''The "slaves" line should list only those which are neither old nor new master.'' | ** ''The "slaves" line should list only those which are neither old nor new master.'' | ||
| + | * Run RESET MASTER on the new master | ||
* Log in to the mysql on the old master as root. Check for any long running queries. Kill them. If there is a single long-running query such as a backup, which will restart after you kill it, then get its thread ID ready in a kill command, ready to press enter later. | * Log in to the mysql on the old master as root. Check for any long running queries. Kill them. If there is a single long-running query such as a backup, which will restart after you kill it, then get its thread ID ready in a kill command, ready to press enter later. | ||
* Set read-only mode in db.php, sync it. | * Set read-only mode in db.php, sync it. | ||
| Line 17: | Line 18: | ||
* Press enter a few more times until it's done, these steps usually work. There may be an error from the old master saying that SLAVE STOP requires a running slave, you can ignore it. | * Press enter a few more times until it's done, these steps usually work. There may be an error from the old master saying that SLAVE STOP requires a running slave, you can ignore it. | ||
* Edit db.php again, turning off read-only mode and simultaneously switching the loads configuration. | * Edit db.php again, turning off read-only mode and simultaneously switching the loads configuration. | ||
| + | * Run RESET SLAVE on the new master, to prevent it from replicating the old master after restart. | ||
* Update [[nagios]] configuration so it doesn't whine about slaves off: | * Update [[nagios]] configuration so it doesn't whine about slaves off: | ||
Revision as of 15:59, 12 September 2008
To switch masters:
- Check that the new master has binlog enabled. All slaves should be configured this way, for convenience. All servers, including masters, should have read_only mode on by default.
cd ~tstarling/src/tools/switch-master
- Edit config.s1, config.s2 or config.s3 depending on which master you are switching. Then copy it to config. Config is the file that is actually used.
- The "slaves" line should list only those which are neither old nor new master.
- Run RESET MASTER on the new master
- Log in to the mysql on the old master as root. Check for any long running queries. Kill them. If there is a single long-running query such as a backup, which will restart after you kill it, then get its thread ID ready in a kill command, ready to press enter later.
- Set read-only mode in db.php, sync it.
- Start the script
./switch
- Press enter, this will attempt to switch the master to read-only. If there are long-running queries, kill them now. The script will block at FLUSH TABLES until you do.
- Check for slave lag. Slave lag must be zero on the new master before you continue.
- Press enter a few more times until it's done, these steps usually work. There may be an error from the old master saying that SLAVE STOP requires a running slave, you can ignore it.
- Edit db.php again, turning off read-only mode and simultaneously switching the loads configuration.
- Run RESET SLAVE on the new master, to prevent it from replicating the old master after restart.
- Update nagios configuration so it doesn't whine about slaves off:
(cd /home/wikipedia/conf/nagios && ./sync)
Script requirements
- ~/.my.cnf or environment set up with db root pw so you don't have to type it in
- if missing, it doesn't seem to bother prompting :P
