Planet.wikimedia.org

From Wikitech
(Difference between revisions)
Jump to: navigation, search
(bart -> singer)
Line 1: Line 1:
Blog aggregator of planet.*.org fame; http://planetplanet.org for the software
+
== What is planet? ==
  
== Installation ==
+
An RSS feed aggregator. See [[meta:Planet_Wikimedia]] for details.
  
On [[singer]] currently, installed from a nightly snapshot of planet 2.0.
+
== New planet (planet-venus) ==
  
Runs on http://planet.wikimedia.org/ CNAME
+
On Jan,23rd 2013 we switched from the [http://wikitech.wikimedia.org/index.php?title=Planet.wikimedia.org&oldid=25640 old planet] (http://www.planetplanet.org/) to  
 
+
the new planet-venus (http://www.intertwingly.net/code/venus/)
== Policy ==
+
 
+
Not 100% determined who will manage. Brion and Erik probably involved to some degree.
+
 
+
== Config ==
+
 
+
Config file and theme is in svn:
+
* http://svn.wikimedia.org/svnroot/mediawiki/trunk/tools/planet
+
 
+
Can be svn up'd on singer in /usr/local/planet/wikimedia. Hourly cronjob will update the live copy.
+
 
+
== How to set up a language subdomain ==
+
 
+
* Add directory & configuration file in SVN <tt>tools/planet</tt> if not yet present
+
* Add language to the updater script on singer (<tt>/usr/local/planet/update-wikimedia</tt>)
+
** wait for hourly cronjob to update or run it manually as user 'planet'
+
* Add CNAME for <tt>''lang''.planet.wikimedia.org</tt> to wikimedia.org [[DNS]]
+
* Add entry to Apache 2.2 config on singer (<tt>/etc/apache2/sites-enabled/planet.wikimedia.org</tt>)
+
* Restart Apache 2.2 on singer
+
 
+
== How to recover in a hurry ==
+
 
+
* Get [http://www.planetplanet.org/ Planet] source
+
* Untar it in /usr/local/planet
+
* svn co http://svn.wikimedia.org/svnroot/mediawiki/trunk/tools/planet wikimedia
+
* Make a docroot for planet.wikimedia.org
+
* (Check file permissions on dirs)
+
* Generate all required ''locales'' (see <tt>config.ini</tt> for each language)
+
* Set up a cronjob that runs planet on the various language configs
+
** fixme: put this script in svn too?
+
 
+
 
+
== If Planet stops updating ==
+
 
+
It happens that the updating process will stop and cause the feeds not to be updated anymore.
+
 
+
It seems like the cache can get corrupted and lead the python script in charge of updating the feeds to crash and prevent the update from happening (as well as creating threads on the server that fail to properly close).
+
 
+
In order to see the problem, you will have to manually run /bin/sh /usr/local/planet/update-wikimedia or  python /usr/local/planet/planet.py /usr/local/planet/wikimedia/LANG/config.ini if you know which LANG is not updating. If there is a problem, you should see a stack trace, with the last line ressembling something like
+
 
+
* KeyError: 'http://www.mw.net.tw/user/kaurjmeb/blog/2008/02/21/28420/78160/ content type'
+
 
+
To fix the issue, simply remove the cached data located in /usr/local/planet/wikimedia/LANG/cache/BLOG_NAME and run the process again.
+
 
+
NOTE: Make sure to run the above mentioned scripts as user planet, or the file permissions in the cache will be switched to root, causing all kinds of issues when updating.
+
  
 
[[Category:Services]]
 
[[Category:Services]]

Revision as of 22:07, 24 January 2013

What is planet?

An RSS feed aggregator. See meta:Planet_Wikimedia for details.

New planet (planet-venus)

On Jan,23rd 2013 we switched from the old planet (http://www.planetplanet.org/) to the new planet-venus (http://www.intertwingly.net/code/venus/)

Personal tools
Namespaces

Variants
Actions
Navigation
Ops documentation
Wiki
Toolbox