Permalink
Commits on Jun 22, 2012
  1. get last indexed sequence of a view using the HTTP API.

    It's useful to get the last sequence indexed in a view (all the view or
    in a range). If the sequences are indexed it is like getting the last
    sequence. If sequences aren't indexed, the last updated sequence of the
    view group is returned.
    
    Example of usage:
    
        $ curl 'http://localhost:5984/testdb/_design/test/_view/test/_last_seq'
        {"name":"test","last_seq":8}
        $ curl 'http://localhost:5984/testdb/_design/test/_view/test/_last_seq?key=1'
        {"name":"test","last_seq":1}
        $ curl 'http://localhost:5984/testdb/_design/test/_view/test/_last_seq?key=8'
        {"name":"test","last_seq":8}
    benoitc committed Jun 22, 2012
  2. add `couch_mrview:get_last_seq/4`

    This function allows us to get the real last indexed sequece in a view.
    benoitc committed Jun 22, 2012
  3. unlink subscriber.

    If we don't unlink subscriber, any crash will affect us.
    benoitc committed Jun 22, 2012
  4. fix include docs issue

    benoitc committed Jun 22, 2012
Commits on Jun 21, 2012
  1. correctly unsubscribe.

    Even if the event handler si supposed to handle none existing pids it's
    cleaner to unsubscribe when we can.
    benoitc committed Jun 21, 2012
  2. `_changes` is now supporting view index updatei events

    if the _view filters is passed in the url, the request process subscribe
    to the index updates using the new `couch_mrview_events:subscribe/3`
    function. All events are then returned to the HTTP api.
    
    Ex:
    
        $ curl
    'http://localhost:5984/testdb/_changes?filter=_view&view=test/test&feed=eventsource&heartbeat=true'
        data:
    {"seq":1,"id":"330c284b190e1af436eab034040007cf","changes":[{"rev":"1-ea7a185b492abc69a6c8e0358d244a98"}]}
        id: 1
    
        data:
    {"seq":4,"id":"330c284b190e1af436eab03404001904","changes":[{"rev":"1-e930be0936bead4354f6a20203e9a9dc"}]}
        id: 4
    
        data:
    {"seq":6,"id":"9ec518da154ddc28420682144e000844","changes":[{"rev":"1-2fa51075d2e9812eaaabf98ababfdd3a"}]}
        id: 6
    benoitc committed Jun 21, 2012
  3. handle couch_mrview events.

    This change introduces an efficient way to manage couch_mrview events
    that notify a suscribed process about an index update.
    
    Internals:
    
    When a process subscribe to a an index updated using
    `couchdb_mrview_events:subscribe/3` it open a new indexer that will
    refresh the indexes each time the database is updated. When no more
    subscribers exists the automatic indexer is killed. Each indexer is
    supervised.
    
    The subscribe receive 2 kinds of notifications:
    
    - `{index_update, {DbName, DDocName}}` when an index is updated
    - `{index_shutdown, {DbName, DDocName}}` when the db or the index are
    deleted.
    
    Ex:
    
        1> couch_mrview_events:subscribe(<<"testdb">>, <<"_design/test">>).
        ok
        2> ok
        2>
    
        21:11:44.673 [info] 127.0.0.1 - - POST /testdb 201
        21:11:44.698 [info] Opening index for db: testdb idx: _design/test sig: "8152471699732c18e2a4954ac27b1049"
        21:11:44.699 [info] Starting index update for db: testdb idx: _design/test
        21:11:44.737 [info] Index update finished for db: testdb idx: _design/test
    
        2> flush().
        Shell got {index_update,{<<"testdb">>,<<"_design/test">>}}
        ok
        3>
    
        21:12:09.799 [info] 127.0.0.1 - - DELETE /testdb/_design/test?rev=1-114ec5224d4dc5d63912cd50536bdf48 200
        21:12:09.801 [info] Closing index for db: testdb idx: _design/test sig: "8152471699732c18e2a4954ac27b1049"
    
        3> flush().
        Shell got {index_shutdown,{<<"testdb">>,<<"_design/test">>}}
        ok
    benoitc committed Jun 21, 2012
  4. add `couch_mrview:refresh/2` function.

    Simple helper to refresh a view index.
    benoitc committed Jun 21, 2012
  5. add `seq_indexed` option to a design document.

    If the option `seq_indexed` is **true**, the views indexes will be also
    indexed by sequences. It is possible to retrieve changes using the
    `couch_mrview:view_changes_since/{5,6,7}`  function wich allows you to
    get all changes in a view or changes from a view in a range since last
    sequence. ex:
    
        couch_mrview:view_changes_since(Db, <<"_design/test">>, <<"test">>,
    0, fun(KV, Acc) -> io:format("kv ~p~n", [KV]), {ok, Acc} end,
    [{start_key, null}, {end_key, null}]).
        kv {{null,4},
            {<<"1f7d72c4ae8cab487c7efeff050011fe">>,
             {[{<<"_id">>,<<"1f7d72c4ae8cab487c7efeff050011fe">>},
               {<<"_rev">>,<<"1-9b2a5b981a9dd0ad1b3a3e826498e9bb">>},
               {<<"test">>,null}]}}}
        {ok,[]}
    
    If the range is omitted all changes will be retrieved since the last
    sequence.
    
    Internals:
    
    When `seq_indexed` is created 1 main sequence is created for all the
    view group with the key `{ViewId, Seq}` and the value `{DocId, Key,
    Val}`
    is added. There can be multiple lines by sequences. These sequences are
    retrieved by querying the b-tree in the range:  `[{ViewId, StartSeq+1},
    {ViewId, EndSeq}]`
    where EndSeq depending on the direction can be 0 or a the last big
    integer.
    
    To allows the querying of changes in a range, a new index is added per
    views with the composite key `{Key, Seq}` (instead DocId) and the value
    `{DocId, Val}` is indexed for this key. We query this index in the range
    `[{StartKey, StartSeq+1}, {EndKey, EndSeq}]` to retrieve the changes.
    
    Caveats:
    
    The size of the indexes will grow significantly with this changes and
    compaction is needed more often.
    benoitc committed Jun 21, 2012
Commits on Jun 18, 2012
  1. add `include_deleted` option to a view

    add the possibility to index deleted documents in a view.
    benoitc committed Jun 18, 2012
Commits on Jun 13, 2012
  1. fix typos

    benoitc committed Jun 13, 2012
Commits on Jun 12, 2012
  1. make couch_httpd more resistant to failures.

    If vhosts or the authentication cache module crash we need to make sure that
    HTTPd bindings are restarted as well.
    benoitc committed Jun 12, 2012
  2. Upgrade the cowboy reauest to a mochiweb request

    Instead of only using the cowboy requets pool we now use a cowboy
    handler that upgrades the connections to a mochiweb request. It will
    will quietly allow us to replace mochiweb by cowboy. While I was here I
    also changed the way config changes are handled: the HTTP(s) listener is
    not any more restarted when only the dispatching rules change. Instead
    we update the cowboy dispatching rules.
    benoitc committed Jun 9, 2012
Commits on Jun 8, 2012
  1. Initial steps to use mochicow.

    Replace mochiweb connection handling by the cowboy one. We are using
    mochiweb to skip the need to rewrite handlers for now. Most of tests
    pass, some are still hanging like the change one but it may be a time
    issue.
    benoitc committed Jun 8, 2012
Commits on May 22, 2012
  1. couch_stats is already loaded.

    benoitc committed May 22, 2012
  2. don't use the ini to load couch_uuids.

    This module is always needed, there is no need to keep it in the
    configuration file.
    benoitc committed May 22, 2012
  3. extract couch_stats in its own app.

    couch_stats is now a full erlang application booted at startup. Other
    changes are a better reinitialization of settings. Instead of exiting
    the gen_server, we just load new settings and overwrite older.
    benoitc committed May 22, 2012
Commits on May 18, 2012
  1. support system dbs. close #9 .

    Systems databases are databases only available and visible by admins.
    All systems databases are prefixed by `rc_` .
    
    Ex:
    
        $ curl -XPUT http://127.0.0.1:5984/rc_somesystemdb
        {"error":"unauthorized","reason":"You are not a server admin."}
        $ curl -XPUT http://admin:test@127.0.0.1:5984/rc_somesystemdb
        {"ok":true}
        $ curl http://admin:test@127.0.0.1:5984/_all_dbs
        ["_replicator","_users","rc_somesystemdb"]
        $ curl http://127.0.0.1:5984/_all_dbs
        ["_replicator","_users"]
        $ curl -XPUT http://admin:test@127.0.0.1:5984/rc_somesystemdb -d'{}'
        {"error":"file_exists","reason":"The database could not be created,
    the file already exists."}
        $ curl -XPUT http://admin:test@127.0.0.1:5984/rc_somesystemdb/test
    -d'{}'
        {"ok":true,"id":"test","rev":"1-967a00dff5e02add41819138abb3284d"}
        $ curl -XPUT http://127.0.0.1:5984/rc_somesystemdb/test1
    -d'{}'{"error":"unauthorized","reason":"You are not authorized to access
    this db."}
        $ curl
    http://127.0.0.1:5984/rc_somesystemdb/_all_docs{"error":"unauthorized","reason":"You
    are not authorized to access this db."}
        $ curl http://admin:test@127.0.0.1:5984/rc_somesystemdb/_all_docs
        {"total_rows":1,"offset":0,"rows":[
        {"id":"test","key":"test","value":{"rev":"1-967a00dff5e02add41819138abb3284d"}}
        ]}
    benoitc committed May 18, 2012
Commits on May 17, 2012
  1. vhosts values should not be empty.

    A vhost value can be empty when removed with the couch_config module.
    benoitc committed May 17, 2012
Commits on May 16, 2012
  1. all should be compile & deps.

    benoitc committed May 16, 2012
  2. fix whitespaces

    benoitc committed May 16, 2012
Commits on May 12, 2012
  1. filter fields in included doc when returned from a _changes

    With this changes you can only return a specified list of fields in the
    included docs by passing the list of them to an optionnal `fields`
    parameter. For now nested fields are ignored.
    
        $ curl -XPUT localhost:5984/testdb
        {"ok":true}
        $ curl -XPUT localhost:5984/testdb/test -d'{"f1": 1, "f2": 2, "f3":
    3}'
        {"ok":true,"id":"test","rev":"1-2b23983c05a9ca8015b6ea7bdc1b3478"}
        $ curl -XPUT localhost:5984/testdb/test1 -d'{"f1": 1, "f2": 4, "f3":
    3}'
        {"ok":true,"id":"test1","rev":"1-2942771a428c4b45315710db0b6aaa7a"}
        $ curl -XPUT localhost:5984/testdb/test2 -d'{"f1": 1, "f2": 7, "f3":
    8}'
        {"ok":true,"id":"test2","rev":"1-ac4cd62a83a0a98421e9ed74adcadcc3"}
        $ curl "http://localhost:5984/testdb/_changes?include_docs=true"
        {"results":[
        {"seq":1,"id":"test","changes":[{"rev":"1-2b23983c05a9ca8015b6ea7bdc1b3478"}],"doc":{"_id":"test","_rev":"1-2b23983c05a9ca8015b6ea7bdc1b3478","f1":1,"f2":2,"f3":3}},
        {"seq":2,"id":"test1","changes":[{"rev":"1-2942771a428c4b45315710db0b6aaa7a"}],"doc":{"_id":"test1","_rev":"1-2942771a428c4b45315710db0b6aaa7a","f1":1,"f2":4,"f3":3}},
        {"seq":3,"id":"test2","changes":[{"rev":"1-ac4cd62a83a0a98421e9ed74adcadcc3"}],"doc":{"_id":"test2","_rev":"1-ac4cd62a83a0a98421e9ed74adcadcc3","f1":1,"f2":7,"f3":8}}
        ],
        "last_seq":3}
        $ curl
    'http://localhost:5984/testdb/_changes?include_docs=true&fields=\["f1"\]'
        {"results":[
        {"seq":1,"id":"test","changes":[{"rev":"1-2b23983c05a9ca8015b6ea7bdc1b3478"}],"doc":{"_id":"test","_rev":"1-2b23983c05a9ca8015b6ea7bdc1b3478","f1":1}},
        {"seq":2,"id":"test1","changes":[{"rev":"1-2942771a428c4b45315710db0b6aaa7a"}],"doc":{"_id":"test1","_rev":"1-2942771a428c4b45315710db0b6aaa7a","f1":1}},
        {"seq":3,"id":"test2","changes":[{"rev":"1-ac4cd62a83a0a98421e9ed74adcadcc3"}],"doc":{"_id":"test2","_rev":"1-ac4cd62a83a0a98421e9ed74adcadcc3","f1":1}}
        ],
        "last_seq":3}
    
        $ curl
    'http://localhost:5984/testdb/_changes?include_docs=true&fields=\["f1","f2"\]'
        {"results":[
        {"seq":1,"id":"test","changes":[{"rev":"1-2b23983c05a9ca8015b6ea7bdc1b3478"}],"doc":{"_id":"test","_rev":"1-2b23983c05a9ca8015b6ea7bdc1b3478","f1":1,"f2":2}},
        {"seq":2,"id":"test1","changes":[{"rev":"1-2942771a428c4b45315710db0b6aaa7a"}],"doc":{"_id":"test1","_rev":"1-2942771a428c4b45315710db0b6aaa7a","f1":1,"f2":4}},
        {"seq":3,"id":"test2","changes":[{"rev":"1-ac4cd62a83a0a98421e9ed74adcadcc3"}],"doc":{"_id":"test2","_rev":"1-ac4cd62a83a0a98421e9ed74adcadcc3","f1":1,"f2":7}}
        ],
        "last_seq":3}
    
    Conflicts:
    
    	apps/couch_changes/src/couch_changes.erl
    benoitc committed May 12, 2012
Commits on May 10, 2012
  1. add support of Server-Sent Events protocol to db changes API.

    This patch add support for the new specification of w3c by adding a new
    feed type named `eventsource`:
    
    http://www.w3.org/TR/2009/WD-eventsource-20090423/
    
    ex:
    
        $ curl
    'http://127.0.0.1:5984/testdb/_changes?feed=eventsource&heartbeat=true&timeout=10000'
        data:
    {"seq":1,"id":"test","changes":[{"rev":"1-967a00dff5e02add41819138abb3284d"}]}
        id: 1
    
        data:
    {"seq":2,"id":"test1","changes":[{"rev":"1-967a00dff5e02add41819138abb3284d"}]}
        id: 2
    benoitc committed May 10, 2012