Apaches
(→All) |
|||
| (2 intermediate revisions by 2 users not shown) | |||
| Line 2: | Line 2: | ||
Apparently not :-P | Apparently not :-P | ||
| + | |||
| + | The Apache configs are both in puppet and in /home/wikipedia/conf/httpd. This guide mostly cover the later. We used to have a subversion repository tracking the changes, it has been made obsolete on July 11th 2012 so we are now fully using git to track change. | ||
==Deploying config== | ==Deploying config== | ||
| Line 9: | Line 11: | ||
#submit change to [[gerrit]] (project: operations/apache-config) | #submit change to [[gerrit]] (project: operations/apache-config) | ||
#merge via gerrit | #merge via gerrit | ||
| − | #git pull on [[fenari]] | + | #'git pull origin master' on [[fenari]] (in /h/w/conf/httpd/) |
| − | + | ||
#run sync-apache (also see [[Sync_scripts#Operating_on_apaches_and_image_scalers_dsh_groups|sync scripts]]) | #run sync-apache (also see [[Sync_scripts#Operating_on_apaches_and_image_scalers_dsh_groups|sync scripts]]) | ||
| Line 25: | Line 26: | ||
===All=== | ===All=== | ||
| − | Connect to [[fenari]] and type < | + | Connect to [[fenari]] and type <code>./[[apache-graceful-all]]</code>. This will restart all apache processes for servers in the apaches dsh node group. You must be root to do this. |
===One, to test a change=== | ===One, to test a change=== | ||
Latest revision as of 00:12, 10 August 2012
Apparently not :-P
The Apache configs are both in puppet and in /home/wikipedia/conf/httpd. This guide mostly cover the later. We used to have a subversion repository tracking the changes, it has been made obsolete on July 11th 2012 so we are now fully using git to track change.
Contents |
[edit] Deploying config
[edit] via gerrit (recommended)
- submit change to gerrit (project: operations/apache-config)
- merge via gerrit
- 'git pull origin master' on fenari (in /h/w/conf/httpd/)
- run sync-apache (also see sync scripts)
This list does not include a check if the config does not break Apache! You can do this by just syncing to a single server and restarting that manually before pushing out to all. There are also test scripts to check multiple URLs to see if redirect changes work. You can also use curl to ask a single server for a specific virtual host from fenari shell to confirm a change works before pushing it out.
- apache-graceful-all
- test change from external
[edit] local hacking
git remote update, git commit, git push master:refs/for/master, submit in gerrit, syn commit locally, sync-apache
[edit] Restarting
[edit] All
Connect to fenari and type ./apache-graceful-all. This will restart all apache processes for servers in the apaches dsh node group. You must be root to do this.
[edit] One, to test a change
Either connect to fenari and ssh wherever or connect to the web server you want to test on. Then type apache-graceful to restart apache on that web server only. Test your change with telnet, as with this foundation example, with what you type in italics:
telnet localhost 80
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET /fundraising HTTP/1.1
Host:wikimediafoundation.org
(an empty line)
The raw HTML for the page will now be displayed in your window. You can copy and paste that into a file on your hard drive and open it with your browser to see the effect. Host is the name of the web site after the http:// part in your browser URL area. GET /fundraising is the part after the site name. The example gets http://wikimediafoundation.org/fundraising.
[edit] Logging
Apache errors are logged to /home/wikipedia/logs/syslog/apache.log on fenari.
Apache access logs are mostly disabled. Webalizer statistics are made using the squids logs instead.
Text output on stderr from various external commands is in various log files in /home/wikipedia/logs on fenari. PHP errors are in /home/wikipedia/logs/syslog/syslog on fenari.
[edit] Apache setup checklist
- Follow the Automated installation instructions for the base install
- Run the following on the server:
- apt-get update && apt-get dist-upgrade -y && apt-get install wikimedia-task-appserver && reboot && exit
- Wait for the server to come back online, ensure it starts apache correctly
- echo 'GET /' | nc localhost 80 or any of the number of tests listed below
- If the server is part of the memcached group, follow instructions on Memcached
- Run the setup of Ganglia
- If the server is new, you will need to do the following:
- Login to the LVS server for apaches (lvs3 as of 2009-02-13) and add the new servers to /etc/pybal/apaches
- If the server is not new do the following:
- Ensure the server is now enabled in pybal on the LVS server in the file /etc/pybal/apaches
- You will need to add the server to DSH groups if new, or check if they are commented, if the server is not new:
- Add/Uncomment the host to /usr/local/dsh/node_groups/apaches and mediawiki-installation, as well as any other groups needed (apaches_pmtpa perhaps?)
- Reload nagios to accept the changes to the node groups:
- cd /home/wikipedia/conf/nagios && ./sync
- Verify that the server is tacking traffic and doing work
- ipvsadm -L | grep SERVERNAME
- traffic logs?
[edit] Test cases
Here are some test cases you can use to test the apache configuration after changing something.
GET /wiki/Foo HTTP/1.1 Host: en.wikipedia.org User-agent: testthing GET /wiki/Foo HTTP/1.1 Host: www.wikipedia.org User-agent: testthing GET / HTTP/1.1 Host: en2.wikipedia.org User-agent: testthing GET /wiki/Main_Page HTTP/1.1 Host: www.wikipedia.com User-agent: testthing GET / HTTP/1.1 Host: wikipedia.com User-agent: testthing GET / HTTP/1.1 Host: wikibooks.org User-agent: testthing GET / HTTP/1.1 Host: wikiquote.org User-agent: testthing GET / HTTP/1.1 Host: dk.wikipedia.org User-agent: testthing GET / HTTP/1.1 Host: foo.wikipedia.org User-agent: testthing GET /wiki/Main_Page HTTP/1.1 Host: test.wikipedia.org User-agent: testthing GET / HTTP/1.1 Host: webshop.wikipedia.org User-agent: testthing GET / HTTP/1.1 Host: boards.wikimedia.org User-agent: testthing GET /wiki/Foo HTTP/1.1 Host: en.wikipedia.org User-Agent: Exalead GET /wiki/Foo HTTP/1.1 Host: meta.wikimedia.org User-agent: testthing GET / HTTP/1.1 Host: en.wiktionary.org User-agent: testthing
