Permalink
Switch branches/tags
Commits on Dec 11, 2017
  1. Allow to use index with or (#1038)

    willholley committed Dec 11, 2017
    Since #816, mango JSON index on compound fields can be 
    selected only if the selector make sure that all the fields 
    listed in the index are always present. 
    
    This adds a special case where all clauses of an `$or` can
    ensure that a field is present.
    
    For instance, if I had an index:
    
    [A, B]
    
    is_usable would now return true for the selector:
    
    {
      "A": "foo",
      "$or": {
        "B": "bar",
        "B": "baz"
      }
    }
    
    but false for:
    
    {
      "A": "foo",
      "$or": {
        "B": "bar",
        "C": "bar"
      }
    }
Commits on Dec 8, 2017
  1. Make q configurable for peruser dbs

    jiangphcn committed with janl Nov 30, 2017
    issue 875
Commits on Dec 5, 2017
  1. Fix replicator create target options test

    nickva committed Dec 5, 2017
    Don't need to use local as source since it doesn't test anything specific to
    the code in question. Use credentials with http requests as it's a more
    realistic scenario. Avoid spawning couch server twice - once to start chttpd
    and then to start other couch servers.
Commits on Dec 1, 2017
  1. Remove references to etap

    kocolosk committed Dec 1, 2017
  2. Remove Bob's 2.0 TODO list

    kocolosk committed Nov 28, 2017
    :)
  3. Remove Spidermonkey as an "optional" depedency

    kocolosk committed Nov 28, 2017
    It's required, and thus would already have been isntalled via the
    instructions in the INSTALL file. Also we don't mention it for any
    other operating system here.
  4. Fix eunit "suites" example

    kocolosk committed Nov 28, 2017
    As far as I can tell rebar will run all tests in an application
    if the specified suites are not found. Also, rebar appends the
    _tests suffix to the name of the suite automatically when it
    executes the search.
Commits on Nov 30, 2017
  1. warn instead of error when use_index not valid (#962)

    willholley committed Nov 30, 2017
    If a user specifies a value for use_index that is not
    valid for the selector - i.e. it does not meet the coverage
    requirements of the selector or sort fields - attempt
    to fall back to a valid index (or database scan) rather
    than returning a 400 error.
    
    When a fallback occurs, populate the "warning" field
    in the response (as we already do when a full database
    scan takes place) with details of the fallback.
    
    This change is partially as mitigation for #816, which may
    lead to some previously valid indexes being deemed invalid,
    and also to make use_index less brittle in general. If
    an index that is used explicitly by active queries is removed,
    Couch will now generate warnings and there may be a performance
    impact, but the client will still get correct results.
  2. Remove invalid meck unload

    eiri committed Nov 30, 2017
    Remove invalid meck unload and catch around unload
    that was hiding this issue.
  3. Add missing methods to fake index

    eiri committed Nov 30, 2017
    Mocked index module missing a couple of methods
    called on late compaction stages. This leads
    to a crash, but since it's happening after
    the test's assertions, it bring the test to fail.
    
    Also small refactoring to encapsulate all mocking
    in a single function and move unrelated parts
    back to test's setup.
Commits on Nov 28, 2017
  1. Allow replicator documents to include params for db creation

    jiangphcn committed with nickva Nov 24, 2017
      - specify q in "create_target_params": {"q": "1", ...}
    issue-887
  2. Fix _explain for sort descending (#1025)

    willholley committed Nov 28, 2017
    Handle the case when startkey and endkey
    are reversed. To create valid JSON we need to
    replace the internal representation of the startkey
    with a string.
    
    Fixes #1023
Commits on Nov 23, 2017
  1. Test duplicate fields in Mango selector (#998)

    willholley committed Nov 23, 2017
    Adds a test to verify the behaviour of duplicate
    fields in a Mango selector.
    
    The fix for CVE-2017-12635 resulted in CouchDB's
    JSON parser only recognising the last instance
    of duplicated fields in a JSON object. This
    represents a breaking change to Mango (_find)
    because, previuously, all instances would have
    been considered when evaluating a selector.
    
    This test verifies that Mango now only considers
    the last instance of a field, silently ignoring
    those that appear before it.
    
    TBD whether we can or should show an error when
    this occurs, since this leads to predicates
    silently being ignored.
  2. Fix index validation for nested $and (#1014)

    willholley committed Nov 23, 2017
    mango_selector:has_required_fields checks that a list of
    indexed fields is covered by a given selector. The implementation
    recurses through the selector, tracking fields that encounters.
    
    Unfortunately, this skipped peers of combination operators. For
    example,
    
    "selector": {
    	"$and":[
    		"$and":[
    			"A": "foo"
    		],
    		"$and":[
    			"B": "bar"
    		]
    	]
    }
    
    would skip the first nested "$and" operator and only return "B"
    as a covered field.
    
    This commit explicitly handles this situation (the only combination
    operator we care about is $and), so for the above selector we
    would correctly indentify "A" and "B" as covered fields.
Commits on Nov 22, 2017
  1. Fix replicator cluster stability race condition

    nickva committed Nov 20, 2017
    Replicator clustering module is in charge of keeping track of when the cluster
    is stable or unstable. A cluster is said to be "stable" if there aren't any
    new nodes added or removed for some period of time.
    
    Replicator document processor uses cluster state to determine what to do with
    document updates as they come in. If a document update comes during a time when
    cluster is unstable, the update is skipped. The idea is that when cluster
    becomes stable again, all the documents will be rescanned anyway.
    
    In order for document updates to not be dropped, there is an implicit
    constraint when cluster becomes stable -- after(couch_replicator_notifier
    gen_event broadcasts `{cluster, stable}` event, any subsequent calls to
    `couch_replicator_clustering:is_stable()` would return `true`. If that's not
    the case then this sequence of events is possible:
    
    1. `mem3_cluster` process calls the `cluster_stable` callback
    
    2. `couch_replicator_notifier` broadcasts `{cluster, stable}` event
    
    3. `couch_replicator_doc_processor` starts processing documents
    
    4. On first document update `couch_replicator_clustering:is_stable()` is
    `false`, because that gen_server wasn't notifier yet.
    
    5. Document update is dropped.
    
    6. There won't be any rescans until cluster membership is changed again.
    
    To fix this, switch to setting stable state first via a `gen_server` call. This
    way after the `{cluster, stable}` has been called, `is_stable()` is guaranteed to
    return `true`.
    
    Note: This issue is mostly theoretical. It was noticed when examining the code
    related to another bug. The chance of the clustering process going to sleep
    immediately after gen_event broadcast, then not handling the cast long enough
    for a document to be processed by doc processor is pretty low in practice.
Commits on Nov 21, 2017
  1. Update COMMITTERS.md

    garbados committed with nickva Nov 21, 2017
Commits on Nov 17, 2017
  1. Move cluster_start_period and cluster_quiet_period to replicator section

    nickva committed Nov 17, 2017
    Somehow entries ended up under [couch_per_user] but should be in the
    [replicator] section.
  2. (typo) fetchig -> fetching

    garbados committed with nickva Nov 17, 2017
Commits on Nov 16, 2017
  1. Improve Mango test suite performance (#995)

    willholley committed Nov 16, 2017
    * Remove artificial delays from database and index create/delete.
    * Wait for indexes to report as created/deleted during test setup.
    * Skip unnecessary database delete/create cycles.
    * Default to n=1 when creating test databases. We don't have
     tests that explicitly test n=3 scenarios and the tests generally 
     run on a single-node harness. Defaulting to n=1 allows the test 
     behaviour to be consistent when run on multi-node clusters.
    * Add delay on cluster setup for Mango tests to mitigate tests
     running before async cluster setup completes.
Commits on Nov 15, 2017
  1. Throw error on invalid use_index value (#989)

    willholley committed Nov 15, 2017
    Throw an error when a user specifies a value in
    "use_index", of the form [<ddoc id>, <name>],
    that cannot be used for the current query selector.
    
    Fixes #988
Commits on Nov 14, 2017
  1. fix exists false when field is explicitly defined (#863)

    tonysun83 committed Nov 14, 2017
    When users explicitly defined a text index's fields, and used $exists
    with false, we tag false when constructing the query. This led to a
    function clause for indexable_fields since we did not account for it.
    This fix patches that up, but note that we don't care about the false
    value itself since we only care about fields.
  2. Fix Mango text index tests (#971)

    willholley committed Nov 14, 2017
    The text index tests are not routinely run by the Couch CI
    (due to an external dependency that isn't shipped with Couch).
    This fixes a number of tests that were broken as a result of
    recent feature changes.
Commits on Nov 12, 2017
Commits on Nov 9, 2017
Commits on Nov 6, 2017
  1. Fix EUnit test suite on Windows

    wohali committed Nov 6, 2017
    1. You cannot create file names with " in them on Windows...
    2. When you update Makefile, you have to update Makefile.win sometimes too.
Commits on Nov 3, 2017
  1. Fix user-specified ddoc name in Mango (#961)

    willholley committed Nov 3, 2017
    When creating a Mango index, we allow the user to specify
    a design document name. This can be of the form "_design/foo"
    or just "foo", with the expectation that Mango would
    automatically add the "_design" prefix.
    
    This fixes a bug whereby if the user specified "_design/foo",
    Mango was prefixing this again, creating a design document called
    "_design/_design/foo".
Commits on Nov 2, 2017
  1. Return error 400 on invalid new_edits value

    eiri committed Nov 2, 2017
    End-point `_bulk_docs` crashes if provided
    with non-boolean value for `new_edits` parameter.
    
    This patch changes it to return "Bad Request"
    error instead.
Commits on Nov 1, 2017
  1. Do not always return partial_filter_selector (#951)

    garrensmith committed Nov 1, 2017
    For Get / _index only return the partial_filter_selector for an index if
    it is set to something other than default
  2. fix external size bug when seq btree exists (#849)

    tonysun83 committed Nov 1, 2017
    When users specify "options":  {"seq_indexed": true}, we will run
    reduced_external_size/1 on seq btrees. The reduce will return an
    integer because the reduce function is:
    couch_db_updater:btree_by_seq_reduce/2.
    This leads to a bad match error. Thanks @jiangphcn for finding this.
Commits on Oct 31, 2017
  1. re-add query

    janl committed with wohali Oct 31, 2017
  2. Revert "faet: reject user docs with duplicate keys"

    janl committed with wohali Oct 31, 2017
    This reverts commit 42adcc1.