Native full text search functionality will no doubt come to App Engine in due course.  But in the meantime my solution has been to use a remote-hosted Solr instance from WebSolr and a slightly modified version of PySolr to get the job done.  Why PySolr rather than other Python-based interface packages like Haystack or Sunburnt?  The simple reason is that none of these will work out-of-the box on App Engine and PySolr was the simplest of them all to modify for my (relatively modest) needs.  You can grab a copy of the PySolr code modifed for App Engine if you want it.

Here’s a quick overview of my setup in case you are looking to do something similar.  I use Django as my framework, so your specifics may vary.

1. Put a copy of PySolrGAE in your app directory so you’ll be able to import the module into your views as needed.
2. Add the following variables in your settings:
   SOLR_PATH = ‘http://index.websolr.com/solr/yourkey/’
   SOLR_BATCH_SIZE = 100
   MAX_RESULT_SIZE = 100
   (obviously, your values will differ!)
3. Set up a schema document (XML) and put it up on your Solr instance so it knows what particular fields you will be passing to it and how it should tokenise, stem and otherwise work its magic on the text within them.  The Solr documentation is pretty good, so it is easy to pick up.
4. Import the module (e.g. from apps.search.pysolrGAE import Solr) into your views and use it to interface with your solr instance.  The ‘readme’ included with the modified PySolr code gives an overview of the syntax for adding, deleting and modifying entries in your index.  I’ve managed to set up views to delete the index, re-create it, and return results which are then passed to a template.  You can also set up a hook in the ‘save’ method of your models to incrementally add/modify or delete items depending on what you’ve done to a particular entity.

One of the nice things about Solr is that you can pass it a field which will not be indexed but is stored alongside an entry.  You can get this field returned as part of a query response.  Hence, you can set up an HTML rendered version of the search result snippet for a particular entry and pass it to Solr at the time you add the entry to the index.  Then, when you run a query you can get that field back and simply pass it through to your template.  This saves you a round trip to the datastore to get a copy of the entity for presentation.  Sweet!

Are you are working through James Bennett’s ‘Practical Django Projects’ (second edition) and getting the above error when trying to view your first flatpage?  This could be because you did not put leading and trailing forward slashes in the flatpage URL field when setting up your flatpage in the admin panel.

The book actually tells you to do this in the example it gives, but in my speed-reading I ignored the slashes and treated the URL field as though I was entering a wordpress page slug.  Four hours, a django reinstall, and much angst later, I have relearned the lesson that it always pays to read the instructions carefully.

Perhaps you can avoid my mistake.
_______
Short URL for this post: http://wp.me/pnqr9-P