Permalink
Browse files

BACKWARD-INCOMPATIBLE: Removed ``RealTimeSearchIndex`` in favor of ``…

…SignalProcessors``.

This only affects people who were using ``RealTimeSearchIndex`` (or a
queuing variant) to perform near real-time updates. Those users should
refer to the Migration documentation.
  • Loading branch information...
1 parent 6065c85 commit 041c480c8260dbe54feeeba52182f0788541d6b7 @toastdriven toastdriven committed Jan 14, 2013
View
@@ -170,10 +170,10 @@ Those include files might look like::
# search/includes/blog/post.html
<div class="post_result">
<h3><a href="{{ result.object.get_absolute_url }}">{{ result.object.title }}</a></h3>
-
+
<p>{{ result.object.tease }}</p>
</div>
-
+
# search/includes/media/photo.html
<div class="photo_result">
<a href="{{ result.object.get_absolute_url }}">
@@ -197,21 +197,23 @@ might looks something like::
Real-Time Search
================
-If your site sees heavy search traffic and up-to-date information is very important,
-Haystack provides a way to constantly keep your index up to date. By using the
-``RealTimeSearchIndex`` class instead of the ``SearchIndex`` class, Haystack will
-automatically update the index whenever a model is saved/deleted.
+If your site sees heavy search traffic and up-to-date information is very
+important, Haystack provides a way to constantly keep your index up to date.
+
+You can enable the ``RealtimeSignalProcessor`` within your settings, which
+will allow Haystack to automatically update the index whenever a model is
+saved/deleted.
-You can find more information within the :doc:`searchindex_api` documentation.
+You can find more information within the :doc:`signal_processors` documentation.
Use Of A Queue For A Better User Experience
===========================================
By default, you have to manually reindex content, Haystack immediately tries to merge
it into the search index. If you have a write-heavy site, this could mean your
-search engine may spend most of its time churning on constant merges. If you can
-afford a small delay between when a model is saved and when it appears in the
+search engine may spend most of its time churning on constant merges. If you can
+afford a small delay between when a model is saved and when it appears in the
search results, queuing these merges is a good idea.
You gain a snappier interface for users as updates go into a queue (a fast
@@ -222,24 +224,40 @@ could live on a completely separate server from your webservers, allowing you
to tune more efficiently.
Implementing this is relatively simple. There are two parts, creating a new
-``QueuedSearchIndex`` class and creating a queue processing script to handle the
-actual updates.
-
-For the ``QueuedSearchIndex``, simply inherit from the ``SearchIndex`` provided
-by Haystack and override the ``_setup_save``/``_setup_delete`` methods. These
-methods usually attach themselves to their model's ``post_save``/``post_delete``
-signals and call the backend to update or remove a record. You should override
-this behavior and place a message in your queue of choice. At a minimum, you'll
-want to include the model you're indexing and the id of the model within that
-message, so that you can retrieve the proper index from the ``SearchSite`` in
-your consumer. Then alter all of your ``SearchIndex`` classes to inherit from
-this new class. Now all saves/deletes will be handled by the queue and you
-should receive a speed boost.
-
-For the consumer, this is much more specific to the queue used and your desired
-setup. At a minimum, you will need to periodically consume the queue, fetch the
-correct index from the ``SearchSite`` for your application, load the model from
-the message and pass that model to the ``update_object`` or ``remove_object``
-methods on the ``SearchIndex``. Proper grouping, batching and intelligent
-handling are all additional things that could be applied on top to further
+``QueuedSignalProcessor`` class and creating a queue processing script to
+handle the actual updates.
+
+For the ``QueuedSignalProcessor``, you should inherit from
+``haystack.signals.BaseSignalProcessor``, then alter the ``setup/teardown``
+methods to call an enqueuing method instead of directly calling
+``handle_save/handle_delete``. For example::
+
+ from haystack import signals
+
+
+ class QueuedSignalProcessor(signals.BaseSignalProcessor):
+ # Override the built-in.
+ def setup(self):
+ models.signals.post_save.connect(self.enqueue_save)
+ models.signals.post_delete.connect(self.enqueue_delete)
+
+ # Override the built-in.
+ def teardown(self):
+ models.signals.post_save.disconnect(self.enqueue_save)
+ models.signals.post_delete.disconnect(self.enqueue_delete)
+
+ # Add on a queuing method.
+ def enqueue_save(self, sender, instance, **kwargs):
+ # Push the save & information onto queue du jour here...
+
+ # Add on a queuing method.
+ def enqueue_delete(self, sender, instance, **kwargs):
+ # Push the delete & information onto queue du jour here...
+
+For the consumer, this is much more specific to the queue used and your desired
+setup. At a minimum, you will need to periodically consume the queue, fetch the
+correct index from the ``SearchSite`` for your application, load the model from
+the message and pass that model to the ``update_object`` or ``remove_object``
+methods on the ``SearchIndex``. Proper grouping, batching and intelligent
+handling are all additional things that could be applied on top to further
improve performance.
View
@@ -71,25 +71,14 @@ not running a ``rebuild_index`` to populate your index or having a blank
This is a Whoosh-specific traceback. It occurs when the Whoosh engine in one
process/thread is locks the index files for writing while another process/thread
-tries to access them. This is a common error when using ``RealTimeSearchIndex``
+tries to access them. This is a common error when using ``RealtimeSignalProcessor``
with Whoosh under any kind of load, which is why it's only recommended for
small sites or development.
-A way to solve this is to subclass ``SearchIndex`` instead::
-
- from haystack import indexes
-
- # Change from:
- #
- # class MySearchIndex(indexes.RealTimeSearchIndex, indexes.Indexable):
- #
- # to:
- class MySearchIndex(indexes.SearchIndex, indexes.Indexable):
- ...
-
-The final step is to set up a cron job that runs
+The only real solution is to set up a cron job that runs
``./manage.py rebuild_index`` (optionally with ``--age=24``) that runs nightly
-(or however often you need) to refresh the search indexes.
+(or however often you need) to refresh the search indexes. Then disable the
+use of the ``RealtimeSignalProcessor`` within your settings.
The downside to this is that you lose real-time search. For many people, this
isn't an issue and this will allow you to scale Whoosh up to a much higher
View
@@ -61,6 +61,7 @@ you may want to include in your application.
faceting
autocomplete
boost
+ signal_processors
multiple_index
rich_content_extraction
spatial
@@ -113,7 +113,7 @@ Examples::
deletions unless the ``--remove`` flag is provided. You might consider
a queue consumer if the memory requirements for ``--remove`` don't
fit your needs. Alternatively, you can use the
- ``RealTimeSearchIndex``, which will automatically handle deletions.
+ ``RealtimeSignalProcessor``, which will automatically handle deletions.
``rebuild_index``
@@ -12,6 +12,7 @@ they consisted of:
* The removal of ``SearchSite`` & ``haystack.site``.
* The removal of ``handle_registrations`` & ``autodiscover``.
* The addition of multiple index support.
+* The addition of ``SignalProcessors`` & the removal of ``RealTimeSearchIndex``.
* The removal/renaming of various settings.
This guide will help you make the changes needed to be compatible with Haystack
@@ -200,6 +201,28 @@ This allows for reliable swapping of the index that handles a model without
relying on correct import order.
+Removal of ``RealTimeSearchIndex``
+==================================
+
+Use of the ``haystack.indexes.RealTimeSearchIndex`` is no longer valid. It has
+been removed in favor of ``RealtimeSignalProcessor``. To migrate, first change
+the inheritance of all your ``RealTimeSearchIndex`` subclasses to use
+``SearchIndex`` instead::
+
+ # Old.
+ class MySearchIndex(indexes.RealTimeSearchIndex, indexes.Indexable):
+ # ...
+
+
+ # New.
+ class MySearchIndex(indexes.SearchIndex, indexes.Indexable):
+ # ...
+
+Then update your settings to enable use of the ``RealtimeSignalProcessor``::
+
+ HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.RealtimeSignalProcessor'
+
+
Done!
=====
View
@@ -1,3 +1,5 @@
+.. _ref-other_apps:
+
=============================
Haystack-Related Applications
=============================
@@ -13,7 +15,7 @@ queued_search
http://github.com/toastdriven/queued_search (2.X compatible)
-Provides a queue-based setup as an alternative to ``RealTimeSearchIndex`` or
+Provides a queue-based setup as an alternative to ``RealtimeSignalProcessor`` or
constantly running the ``update_index`` command. Useful for high-load, short
update time situations.
Oops, something went wrong.

0 comments on commit 041c480

Please sign in to comment.