Swift/Testing the cluster
From Wikitech
< Swift
Contents |
overview
When testing Swift, there are several ways to get at the data, and choosing the right one requires an understanding of how data flows into and out of swift. Here's the short version, specific to thumbnails:
requesting an image
- a user calls a upload.wikimedia.org URL
- it hits LVS and flows through to squid
- on a cache miss, squid sends it to ms-fe.pmtpa.wmnet (held by another LVS server)
- ms-fe passes the request to ms-fe1 or ms-fe2.pmtpa.wmnet
- The swift frontend looks it up in the container, returns the object
- if the object doesn't exist, it calls out to ms5.pmtpa.wmnet
- ms5 returns the object if it has it
- if the object doesn't exist, it calls out to the rendering.pmtpa.wmnet cluster (held by LVS)
- the render LVS passes the request to an apache running mediawiki
renaming or purging
- a user calls a URL like http://commons.wikimedia.org/wiki/File:Little_kitten_.jpg?action=purge
- it hits a mediawiki app server
- the purge handler has a hook that calls out to swift.php
- swift.php authenticates against the cluster (using the mw:thumb account) and moves the file to the deleted container (deleting / recreating)
retrieve a file from the cluster
- Load the URL http://upload.wikimedia.org/wikipedia/commons/thumb/a/a2/Little_kitten_.jpg/900px-Little_kitten_.jpg
- Load a different size (choose a random number 100 < x < 1000, say 534) and use that for the size. This will likely generate a new thumbnail, forcing the new thumbnail to be generated
- get a listing for the container that should hold the thumbnail, verify the thumbnail was created
- purge all the thumbnails (http://commons.wikimedia.org/wiki/File:Little_kitten_.jpg?action=purge)
- get a listing, verify the thumbnail has been deleted
- Load the original URL with a different size (say 234)
- get a listing, verify the new thumbnail has been created
