Image scalers

Revision as of 20:42, 8 September 2009

Image scalers handle generation of thumbnails.

Temp files

The scalers need to have the directory /a/magick-tmp on them (owned and writable by apache). This matches the $wgImageMagickTempDir setting in CommonSettings.php. If the setting is changed, the directory on these servers must be changed as well. Imagemagick temp files have the form magick-XXnnnnnn and the files get left around when conversion dies for some reason such as overruning reousrces allocated by ulimit.

Ghostscript (gs) is called by Imagemagick when it converts pdfs to thumbnails. It leaves its temp files in whatever directory is pointed to be $wgTmpDirectory (curently /tmp). Its files look like gs_nnnnnn.

Current ulimits shoot gs when it creates a scratch file bigger than 100mb. This is more often than you might think; a pdf of about 10 mb can easily cause gs to write more than that. And if someone does a search on commons that results in one of these files showing up in the search results, every time they look at that result list it will try to create the thumb. A few of these can add up quickly, and we only have 4GB free on these boxes on the root partition.

To deal with this, a cron job clears out the above files from these directories every 5 minutes. It's been set up in puppet. (See /etc/puppet/manifests/imagescaler.pp on sockpuppet.) If you change these directories in the above config files, you need to change them in these cron jobs as well.