View changes #1

Closed
wants to merge 15 commits into
from

Projects

None yet

1 participant

@benoitc
Contributor
benoitc commented Mar 23, 2012

Pull request to track the view changes branch.

benoitc added some commits Feb 22, 2012
@benoitc benoitc Add `couch_mrview:view_changes_since` function.
This functions fetch all changes in a view since last update like the
`couch_db:changes_since` function for the databases. It returns an error
if the option `seq_indexed` hasn't been set to `true` in the design
document:

    {
        "_id": "_design/somedoc",
        "options": {
            "seq_indexed": "true"
        },
        "views": {
            "someview": {
                "map": "function(doc) { if (doc.field) emit(doc.field, null); }"
            }
        }
    }

This is done by adding an index to the view group (a view group is the
global index associated to a design doc basically). This index associate
each sequences & documents ID to a view . `{{ViewId, Seq}, DocId} is
stored in the seq_btree, DocId is the value.

example of usage:

    (refuge@127.0.0.1)3> F = fun(KV, Acc) -> {ok, [KV|Acc]} end.
    (refuge@127.0.0.1)4>
    (refuge@127.0.0.1)4> couch_mrview:view_changes_since(Db, DDoc,
<<"all">>, 9990, F).
    [info] [<0.220.0>] Opening index for db: testdb idx: _design/test
sig:
    "85b8fa1e39dc7fbe8b11b729b319c6bf"
    {ok,[{10000,{<<"72f9250618e65adc8cf4552758fff137">>,0}},
         {9999,{<<"72f9250618e65adc8cf4552758ffe85d">>,0}},
         {9998,{<<"72f9250618e65adc8cf4552758ffdbcf">>,0}},
         {9997,{<<"72f9250618e65adc8cf4552758ffd65e">>,0}},
         {9996,{<<"72f9250618e65adc8cf4552758ffd611">>,0}},
         {9995,{<<"72f9250618e65adc8cf4552758ffd045">>,0}},
         {9994,{<<"72f9250618e65adc8cf4552758ffc982">>,0}},
         {9993,{<<"72f9250618e65adc8cf4552758ffbfdb">>,0}},
         {9992,{<<"72f9250618e65adc8cf4552758ffb1fc">>,0}},
         {9991,{<<"72f9250618e65adc8cf4552758ffae18">>,0}}]}
    (refuge@127.0.0.1)5>
    (refuge@127.0.0.1)5> couch_mrview:view_changes_since(Db, DDoc,
<<"all">>, 10000, F).
    {ok,[]}
    (refuge@127.0.0.1)7> couch_mrview:view_changes_since(Db, DDoc,
<<"all">>, 10000, F).
    {ok,[{10002,{<<"31b0c9f8353b6b3e2a86c23d2c000a78">>,0}}]}
5d240fa
@benoitc benoitc `_view` filter is now hanled by couch_mrview:view_changes_since/7
Rather than passing all doc sent y couch_db:changes_since/5 to a
transformed view function we are now using the more efficient
`couch_mrview:view_changes_since/7` wich use the seq btree in the view
group.

Also add the parameters `view_filter=DName/FilterName` which allows to
filter any view changes using the fiter function `FilterName` from the
design document `DName`:

   GET http://<node>/<dbname>/_changes?filter=_view&view=<viewname>&view_filter<filtername>

Where variables are:

- <node>: URL of the CouchDB node
- <dbname>: Name of the database
- <viewname>: DesignId/ViewNameThe name of the view in the views properties
  from the design document DesignId`
- <filtername>: DesignId1/FilterName The name of the filter in
  the filters properties from the design document DesignId1

The difference with the old behaviour is that you really accessing the
view index instead of passing each map function to all the changed docs
wich is a way more efficient (you do'nt open a system process, serialize
the function, pass it to the process, wait for return, test it and
eventually return a change). You can also filter the results. Other
advantages are that the views are refreshed in real time when the
database content changes and the view changes are only triggered if the
index is updated.
ba9246a
@benoitc benoitc Get view changes using a view query.
Add the possibility to the function `couch_mrview:view_changes_since` to
accept some **view query parameters** like `key`, `start_key` &
`end_key`. Instead of fetching all changes in a view from last updated
it will fetch changes only from the results of the query.  It returns an
error  if the option `seq_indexed` hasn't been set to `true` in the
view options:

        {
            "_id": "_design/somedoc",
            "views": {
                "someview": {
                    "map": "function(doc) { if (doc.field)
emit(doc.field, null); }",
                    "options": {
                        "seq_indexed": "true"
                    }
                }
            }
        }

This is done by adding an index (b-tree) to each view. This index
associate the view key with the doc sequence. `{{Key, Seq}, {DocId, Value}}`
is stored in this btree.
c681e1c
@benoitc benoitc add support for `include_deleted: true` option in a design document.
This option allows a view to map deleted documents.

Remember that documents deleted with the DELETE HTTP verb will look like
{_id: id, _rev: rev, _deleted: true} to the indexer. If you want to
store extra data on the deleted document you can use _bulk_docs or
updating a document with the member '_deleted: true' using the HTTP verb
PUT.
e4b75f6
@benoitc benoitc fix _changes test
The `_view` filter now need a view group indexed by sequences.
282671e
@benoitc benoitc fix imports fce9e24
@benoitc benoitc change order 37efe00
@benoitc benoitc fix include: 950bde5
@benoitc benoitc Revert "fix include:"
This reverts commit ed0076d3fdbf2b2cf28acb204f48fe07db65440f.
deb4968
@benoitc @benoitc benoitc fix purge 93e1987
@benoitc benoitc filter fields in included doc when returned from a _changes
With this changes you can only return a specified list of fields in the
included docs by passing the list of them to an optionnal `fields`
parameter. For now nested fields are ignored.

    $ curl -XPUT localhost:5984/testdb
    {"ok":true}
    $ curl -XPUT localhost:5984/testdb/test -d'{"f1": 1, "f2": 2, "f3":
3}'
    {"ok":true,"id":"test","rev":"1-2b23983c05a9ca8015b6ea7bdc1b3478"}
    $ curl -XPUT localhost:5984/testdb/test1 -d'{"f1": 1, "f2": 4, "f3":
3}'
    {"ok":true,"id":"test1","rev":"1-2942771a428c4b45315710db0b6aaa7a"}
    $ curl -XPUT localhost:5984/testdb/test2 -d'{"f1": 1, "f2": 7, "f3":
8}'
    {"ok":true,"id":"test2","rev":"1-ac4cd62a83a0a98421e9ed74adcadcc3"}
    $ curl "http://localhost:5984/testdb/_changes?include_docs=true"
    {"results":[
    {"seq":1,"id":"test","changes":[{"rev":"1-2b23983c05a9ca8015b6ea7bdc1b3478"}],"doc":{"_id":"test","_rev":"1-2b23983c05a9ca8015b6ea7bdc1b3478","f1":1,"f2":2,"f3":3}},
    {"seq":2,"id":"test1","changes":[{"rev":"1-2942771a428c4b45315710db0b6aaa7a"}],"doc":{"_id":"test1","_rev":"1-2942771a428c4b45315710db0b6aaa7a","f1":1,"f2":4,"f3":3}},
    {"seq":3,"id":"test2","changes":[{"rev":"1-ac4cd62a83a0a98421e9ed74adcadcc3"}],"doc":{"_id":"test2","_rev":"1-ac4cd62a83a0a98421e9ed74adcadcc3","f1":1,"f2":7,"f3":8}}
    ],
    "last_seq":3}
    $ curl
'http://localhost:5984/testdb/_changes?include_docs=true&fields=\["f1"\]'
    {"results":[
    {"seq":1,"id":"test","changes":[{"rev":"1-2b23983c05a9ca8015b6ea7bdc1b3478"}],"doc":{"_id":"test","_rev":"1-2b23983c05a9ca8015b6ea7bdc1b3478","f1":1}},
    {"seq":2,"id":"test1","changes":[{"rev":"1-2942771a428c4b45315710db0b6aaa7a"}],"doc":{"_id":"test1","_rev":"1-2942771a428c4b45315710db0b6aaa7a","f1":1}},
    {"seq":3,"id":"test2","changes":[{"rev":"1-ac4cd62a83a0a98421e9ed74adcadcc3"}],"doc":{"_id":"test2","_rev":"1-ac4cd62a83a0a98421e9ed74adcadcc3","f1":1}}
    ],
    "last_seq":3}

    $ curl
'http://localhost:5984/testdb/_changes?include_docs=true&fields=\["f1","f2"\]'
    {"results":[
    {"seq":1,"id":"test","changes":[{"rev":"1-2b23983c05a9ca8015b6ea7bdc1b3478"}],"doc":{"_id":"test","_rev":"1-2b23983c05a9ca8015b6ea7bdc1b3478","f1":1,"f2":2}},
    {"seq":2,"id":"test1","changes":[{"rev":"1-2942771a428c4b45315710db0b6aaa7a"}],"doc":{"_id":"test1","_rev":"1-2942771a428c4b45315710db0b6aaa7a","f1":1,"f2":4}},
    {"seq":3,"id":"test2","changes":[{"rev":"1-ac4cd62a83a0a98421e9ed74adcadcc3"}],"doc":{"_id":"test2","_rev":"1-ac4cd62a83a0a98421e9ed74adcadcc3","f1":1,"f2":7}}
    ],
    "last_seq":3}
ebab8b6
benoitc added some commits May 16, 2012
@benoitc benoitc add a test for changes eventsource feed. 16db793
@benoitc benoitc fix whitespaces 4c9bafc
@benoitc benoitc make sure the changes test works on all browsers 56bb344
@benoitc benoitc Merge branch 'master' into view_changes
Conflicts:
	apps/couch_changes/include/couch_changes.hrl
	apps/couch_changes/src/couch_changes.erl
	apps/couch_changes/src/couch_httpd_changes.erl
	apps/couch_replicator/src/couch_replicator_api_wrap.erl
e942d68
Contributor
benoitc commented Jun 22, 2012

applied in las head

@benoitc benoitc closed this Jun 22, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment