Support params on bulk_docs

[InTheFiveByFive]

I need to use _all_docs with the 'include_docs: true' parameter in order to fetch multiple documents in one call. I've tested in my app and I'm getting an immense performance improvement (e.g. from 5 mins down to ~15 seconds) when retrieving documents in batches of 100 at a time from a set of ~3000 documents. I cannot do this in nano today because bulk_docs does not take params. I see two potential solutions: minimal impact but breaking API change or add a new function.

Minimal: function bulk_docs(docs,callback) -> function bulk_docs(docs, params, callback)

New Function: function get_docs(doc_names, params, callback)

thoughts?

[PatrickHeneise]

You can use db.list() with an array of ids:

  db.list {keys: doc_keys, include_docs: true}, (error, data) ->

[InTheFiveByFive]

See my commit @ https://github.com/InTheFiveByFive/nano/commit/a96a50d2e1fd52ca69ca70bea6163e4445bd7c88

[PatrickHeneise]

You're using POST which is supposed to send data to the database, not GET something. Also, as mentioned, the function already exists, you just need to add {include_docs: true} as parameter:

function list_docs(params,callback) {
      if(typeof params === "function") {
        callback = params;
        params   = {};
      }
      return relax({db: db_name, path: "_all_docs", method: "GET", params: params},callback);
    }

[InTheFiveByFive]

Per the bulk document api (http://wiki.apache.org/couchdb/HTTP_Bulk_Document_API),

By POSTing to _all_docs you can get a set of documents with arbitrary keys

Simple example to fetch the keys bar and baz and include the complete document in the result set:

curl -d '{"keys":["bar","baz"]}' -X POST http://127.0.0.1:5984/foo/_all_docs?include_docs=true

db.list() will return all documents, not a subset. To retrieve a subset of documents you POST with the body set to the list of keys.

[dscape]

Hi @InTheFiveByFive

I'm currently away at a conference with limited time so can't follow up right now. If you can prepare a pull request I can check it out. If not just give me 1/2 days to review this and put in in npm. if this is a bug, i havent looked at it yet :)

[dscape]

Ok, I couldn't stop myself from reading this

@InTheFiveByFive please submit a pull request doing:

Minimal: function bulk_docs(docs,callback) -> function bulk_docs(docs, params, callback)

just if typeof params === function and preserve existing api behavior. so this should break any existing production nano code. am i missing something here?

if this is the case ill accept a pull request and publish to npm.

thanks for submiting this issue and working on this. we are always super excited to have more people invested in our little community. feel free to add your projects and community to our wiki :)

[PatrickHeneise]

Hmm, "keys" is not the same as "startkey" and "endkey", it does indeed return a subset with only the documents requested by key, so it's the same behavior as POSTing the keys in the body. However, I can't find that in the API, so I suppose it's better to use POST.

[dscape]

@InTheFiveByFive any updates on this or did this ended up not being necessary?

[InTheFiveByFive]

FYI, after testing the changes in my application, this issue is not fixed. It turns out the the bulk read api uses POST _all_docs path whereas the bulk write api uses the POST _bulk_docs path.

So for multiple downloads by key, list_docs() has the right path but the wrong verb and bulk_docs() has the right verb and the wrong path.

I'd propose we go with the original design of adding a new method to expose POST _all_docs. No need to revert pull request #54, as it just added params support which might be useful in other scenarios.