Implement startkey_docid and endkey_docid in allDocs/mapreduce

[nolanlawson]

These are just some of the CouchDB params we don't support yet (see also group_level in #967 and #1340 for some keys fixes).

[daleharvey]

I think this is fairly complicated and I think we wanna try on reducing some redundent logic here, we do want it but I dont think its a goodfirstpatch

[nolanlawson]

Yeah, you're right, since it involves a secondary index, meaning it will definitely need @neojski's indexable_string in order to work on leveldb and IE's idb.

[nolanlawson]

group_level was done in pouchdb/mapreduce#82.

[nolanlawson]

I started working on this, but I am pretty sure there is a bug in CouchDB 1.5.0. The behavior for startkey_docid and endkey_docid is really surprising. I assumed it was just a basic "sort by key, then docid" kind of pagination, but it's something much weirder. Also, offset is occasionally returning 0 even though I never set skip anywhere. Gonna need to hit up the couchdb folks on IRC tomorrow to see what they think.

[calvinmetcalf]

whats wrong with zero offset when you don't set skip?

On Sun, Apr 13, 2014 at 9:32 PM, Nolan Lawson notifications@github.comwrote:

I started working on this, but I am pretty sure there is a bug in CouchDB
1.5.0. The behavior for startkey_docid and endkey_docid is really
surprising. I assumed it was just a basic "sort by key, then docid" kind of
pagination, but it's something much weirder. Also, offset is occasionally
returning 0 even though I never set skip anywhere. Gonna need to hit up
the couchdb folks on IRC tomorrow to see what they think.

Reply to this email directly or view it on GitHubhttps://github.com//issues/1397#issuecomment-40326988
.

-Calvin W. Metcalf

[nolanlawson]

Isn't the offset always supposed to === skip?

[nolanlawson]

Ah okay, wow. The offset is supposed to be how many docs are before the first result. We totally implemented that wrong in both allDocs() and query(). We always just set offset to skip.

[nolanlawson]

Probably it would be good to put a note somewhere saying that PouchDB doesn't return the same offset as CouchDB, since it's just unfeasible for us to do that in anything but websql.

[calvinmetcalf]

really wow, I was just thinking that if you don't specify skip then it would be zero by default ...

[nolanlawson]

Nah, I'm testing it now in CouchDB 1.5.0, and it's definitely the number of documents before the first result returned. Definitely not feasible unless we want to implement a btree on top of level/idb.

[nolanlawson]

Okay, so I noted in that commit all the places where I was surprised by the functionality. In particular:

• If you set startkey="a" and startkey_docid="9" and there is no docid >= "9", then everything for "a" is returned, as if you had not put startkey_docid at all. I would expect docids < "9" with key "a" to be returned.
• If you set the same startkey but different startkey_docids, then even if those docids are valid for that key, you will get zero results back.

In general, if this is the desired functionality, then it's going to be tough for us to implement because we index everything by [key, docid, value].

[nolanlawson]

Okay, for the first thing, Kxepal told me what I was misunderstanding:

*_docid param is some sort of helper for *key one to navigate to the right view row for specific doc id
however, if key isn't found next one takes into account. if docid is missed - it's just ignored

[nolanlawson]

This was fixed awhile ago.