Delta indexing - seeing lost record *during* reindexing #2

Open
ehansen486 opened this Issue May 9, 2013 · 0 comments

Comments

Projects
None yet
1 participant

I currently have a small enough dataset that I can do a full reindexing every 2 hours. However one of my clients is taking 9 minutes to do a full re-index. They have been complaining about records that disappear and then reappear.

I was able to reproduce this, here's how:

  1. Start up the rails server, and delayed job for delta indexing
  2. Create a new record AAA
  3. After delta indexing completes (which is very fast), AAA shows up in the sphinx index as expected
  4. Kick off a full reindex
  5. Watch the reindex output, and note when the model type of record AAA begins indexing. Note that at this point, sphinx does an update to set the delta flag to 0 for all records in that table.
  6. As soon as you see it start indexing AAA's model type, create a new record BBB of type AAA
  7. When the delayed-job driven delta indexing of BBB completes, search for AAA in the sphinx index, and note that it is not there but BBB is there

Both AAA and BBB will show up when the full index job completes. So if model type AAA takes a long time to full index, there is a period of time where the delta index returns incorrect results.

I'm wondering if anybody has encountered this and if there's a way around it, or simply something we have to live with?

Thanks,
Eric

Refer to the google group:
https://groups.google.com/forum/?hl=en&fromgroups=#!topic/thinking-sphinx/TrQoZfP7WT4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment