After purging a doc, its rows are left behind in view indexes

[snej]

Purging a document won't trigger views to update their indexes the next time they're queried, so they may return obsolete rows that were emitted by that document.

There are two problems:

1. Since purging doesn't increment the database's lastSequence, the quick check against the view's lastSequenceIndexed won't detect that the index needs updating.
2. The purge doesn't show up in the all-docs-by-sequence query that the indexer runs, so it won't remove the previous rows emitted by that doc.

CouchDB keeps a persistent purgeCount property on the database, which is incremented every time a purge operation happens. A view index also remembers the database's purgeCount when it updates its index. So when a view checks its index it compares purgeCounts, and if its purgeCount is behind it deletes its index completely and rebuilds it. (It can't just update it incrementally because it doesn't know which rows are obsolete.)

Actually there's a further optimization in CouchDB (probably added later) to keep the full reindexing from happening so often. The database also keeps track of which docIDs were purged in the last purge operation. Then when a view sees its purgeCount is out of date, but it's only behind by 1, it can patch up its index by deleting the rows emitted by the docIDs in the database's last-purged-IDs property.

I'm not sure if we need that second optimization. I know that CouchDB is used for some enormous databases where views can take literally hours to days to rebuild from scratch, so this may be aimed at that. In Couchbase Lite it may not be too much of a hit to have to rebuild views.

[snej]

Correction from @borrrden:

My testing shows that when a document is purged its revisions are removed from the maps table. Isn't it due to the ON DELETE CASCADE statement on the view index (for SQLite at least...not sure about ForestDB)?

That takes care of it with SQLite; a fix for ForestDB can go in CBForest so all 3 implementations get it.

There's a related problem that does occur with both storage engines, though: after a view index is altered by having purged rows removed, there's no indication that it's changed, so LiveQueries won't re-run the query or notify their observers that they changed.