Solr

Revision as of 13:12, 23 December 2012

Solr is a Lucene-based search engine.

Current uses on WMF

• Translation memory - on shared server vanadium, by mw:Extension:Translate (see announcement and development information)
• Spatial searches - on solr*, see mw:Extension:GeoData

Puppet

modules/solr

Operations

How to poke it with a stick

Checking if it's OK:

maxsem@fenari:~$ curl http://solr1001:8983/solr/admin/cores?action=STATUS
<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">2</int></lst> (loads of XML)

Checking if search works:

$ curl 'http://solr1001:8983/solr/select/?q=*%3A*&version=2.2&start=0&rows=10&indent=on'
(loads of XML)

Get schema remotely:

$ curl 'http://solr1001:8983/solr/admin/file/?contentType=text/xml;charset=utf-8&file=schema.xml'

Restart

# service jetty restart

Upgrading schema:

• ☠Delete existing documents☠:

$ curl 'http://localhost:8983/solr/update?commit=true&stream.body=%3Cdelete%3E%3Cquery%3E*%3A*%3C%2Fquery%3E%3C%2Fdelete%3E'

• Stop the server:

# service jetty stop

• Update schema.xml e.g. by forcing a puppet run:

# puppetd -tv

• Reindex the data

Logs

• If jetty was restarted, it moves today's logs to *.<some number>

In /var/log/jetty:

• request.log - requests
• stderrout.log - errors and debug information

These logs are prefixed with "year_month_day.".

Examples:

• 2012_12_19.stderrout.log.100139764
• 2012_12_20.request.log

Packages

We use our own backports:

$ dpkg -l *solr*
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                       Version                    Description
+++-==========================-==========================-====================================================================
ii  libsolr-java               3.6.0+dfsg-1               Enterprise search server based on Lucene - Java libraries
ii  solr-common                3.6.0+dfsg-1               Enterprise search server based on Lucene3 - common files
ii  solr-jetty                 3.6.0+dfsg-1               Enterprise search server based on Lucene3 - Jetty integration

Links

• Documentation (pretty crappy)