CouchDB river : Add support for views

[dadoonet]

Real implementation of pull request #1258, as there was a problem with GitHub.

In CouchDB, you can retrieve docs by GET, _changes API and views.
CouchDB river uses _changes API to get documents.

I would like to be able to get documents that changed (getting ID with the _changes API) using a view with parameter key="DOCID".

As views return a collection of results (aka rows), we will index in ES each row with an id like DOCID_seq where seq is the sequence number of each row.
If you get back 3 rows for one single change for document with ID=1234, the river will index 3 documents :

• 1234_1
• 1234_2
• 1234_3

To use it, you have to define a view in couchDB. For instance, _design/vues/_view/test_dpi with

function(doc) {
listArt=doc.document.articles;

for(var i=0; i<listArt.length;i++) {  
 var artJson = {};
 artJson = { 'docid' : doc._id, 'num' : listArt[i].numeroArticle };
 artJson =  JSON.stringify( artJson );
 emit(doc._id, eval('('+artJson+')') );
};
}

You can use it in your couchDb river as follow :

{
  "type":"couchdb",
  "couchdb": {
    "host":"localhost",
    "port":"5984",
    "db":"dau_test",
    "view":"vues/_view/test_dpi",
    "viewIgnoreRemove":false
  }
}

New options :

• view : if not null, couchDB river will not fetch content from _changes API but only IDs and then will use the view to retrieve rows using the ID as a key. By default : null
• viewIgnoreRemove : ask the river to ignore removal of rows if there is less rows after a document update. By default : false so non existing rows will be removed from elastic search.

For example, with the 3 rows described earlier, if you push a new version of the document 1234 in couchDB with only 2 docs,

If viewIgnoreRemove is false (default), then

• 1234_1 will be updated
• 1234_2 will be updated
• 1234_3 will be removed

If viewIgnoreRemove is true, then

• 1234_1 will be updated
• 1234_2 will be updated
• 1234_3 will not be updated

BTW, I will push an update to the pull request when the ids_prefix filter will be available to make code more efficient. (See issue #1259).

Thanks
David

[dadoonet]

Moved here : https://github.com/elasticsearch/elasticsearch-river-couchdb/issues/1