I've been trying to find ways to improve the performance of secondary indexes. Obviously we'd rather go with native secondary indexes, but that may be a ways off, and I think there are ways we can at least stop the bleeding for current slow paths in secondary indexes.
However, for the vast majority of cases (and for the cases that affect secondary indexes), the changes() queries are rather simple, and can essentially be expressed as "fetch all documents with doc.seq >= seq and with limit limit for the given seq and limit". If none of the documents have later seqs (e.g. in the rare case of a non-winning document being replicated after a winning document), then this query is all we need, and can get done with two simple IDB operations: getAll() and getAllKeys() (for browsers that support it, currently Chrome and Firefox but Safari is implementing it as well and it's under consideration for Edge).
doc.seq >= seq
So my technique is to write a separate changes() implementation that uses the "fast" strategy, and then we fall back to the old "slow" strategy whenever necessary. Running the mapreduce and integration tests, I found that the fast path was hit 6587 times and the slow path was hit 545 times, so 92% of the time we're using the fast option (in Firefox/Chrome anyway).
Furthermore this has a huge impact on our perf. Using the temp-views test (which is a good proxy for intense secondary index usage), I got the following improvement (10 iterations):
(#6031) - faster IDB changes() using getAll()
Adding WIP because on second thought I think we need new tests and I want to ensure this implementation is bulletproof
On second thought I believe this introduces bugs (I can probably find a test that fails if you modify the same document multiple times and then multiple documents one time immediately after). Also I think this can be abstracted into a single implementation that does a "batched cursor" – i.e. a pseudocursor that uses getAll() with limit for browsers that support it and regular cursors for other browsers.
But at the very least this demonstrates the perf improvement!
Reopening because I want to remind myself to keep looking into this. The batched cursor will be challenging to implement, but I think it can be done.
(#6031) - faster IDB changes() with getAll()
(#6031) - faster IDB changes() with batched cursor