Permalink
Commits on May 25, 2017
  1. MB-16181: Filters with deleted collections

    jimwwalker authored and daverigby committed May 24, 2017
    Upstream integration revealed an issue with the VB::Filter.
    If we construct a filter with a collection "X" and "X" is actually
    marked as deleted, "X" should not be part of the filter.
    
    We fix this by replacing VB::Manifest::doesCollectionExist with
    VB::Manifest::isCollectionOpen which performs the correct checks.
    
    Tests updated to cover this case.
    
    Change-Id: I38d4ba025805da7653bf3a84053d4280f4f547d7
    Reviewed-on: http://review.couchbase.org/78570
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: Build Bot <build@couchbase.com>
Commits on May 24, 2017
  1. MB-23990: Fix intermittent hangs in unit test

    premkumr authored and trondn committed May 24, 2017
    In some scenarios, the io complete notification reached before we start
    waiting for it via the condition variable. This results in the test
    hanging forever. Now we track the no.of notifications and wait
    accordingly
    
    Change-Id: Id9d9404892e111d7d60760d0e9ce727b474c0997
    Reviewed-on: http://review.couchbase.org/78508
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: Build Bot <build@couchbase.com>
  2. MB-24293: Retry atmost 10 times if directory removal fails

    sriganes authored and daverigby committed May 17, 2017
    The database directory is removed in order to simulate a commit
    failure. The directory removal could fail for various reasons.
    Retry atmost 10 times to remove the directory
    
    Change-Id: I16e81f3572e0b7d58af3d5ee1f7849aec8cabf97
    Reviewed-on: http://review.couchbase.org/78254
    Tested-by: Build Bot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
  3. MB-24424: Pass correct parameters when making MutationResponse

    owendCB authored and daverigby committed May 18, 2017
    The isKeyOnly parameter was missing when passed into make_unique. This
    patch fixes this issue.  In addition it moves the creation of a
    PassiveStream into a separate function.  This enables the function that
    creates a PassiveStream to be overridden with one that creates a
    MockPassiveStream, which is used in testing.
    
    Change-Id: I58e2e8ca06acba24573d1b4a53aeed85dbcecffa
    Reviewed-on: http://review.couchbase.org/78302
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: Build Bot <build@couchbase.com>
  4. MB-16181: Add collections checks to more commands

    jimwwalker authored and daverigby committed May 22, 2017
    Currently only get/set have collections validation in-place, this
    needs extending to cover all "commands-paths" where a client is
    operating on a key.
    
    Change-Id: I976533788e3365fb4eb789bce6132812a4339e4c
    Reviewed-on: http://review.couchbase.org/78390
    Reviewed-by: Daniel Owen <owend@couchbase.com>
    Tested-by: Build Bot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
  5. Remove unnecessary #includes in DCP code

    daverigby committed May 17, 2017
    Change-Id: Id6bb0258230281647f191bb25282e60a533c32de
    Reviewed-on: http://review.couchbase.org/78322
    Tested-by: Build Bot <build@couchbase.com>
    Reviewed-by: Jim Walker <jim@couchbase.com>
  6. MB-24055: Change ht_size to 47; ht_resize_interval to 1s

    daverigby committed May 9, 2017
    To determine a good tradeoff between initial memory usage and
    rebalance speed, the following tests were performed:
    
        ht_size	    ht_resize_interval (sec)  Rebalance time (min)  Initial RSS (MB)
        3079 (default)  60 (default)	      4.5                   74.4
        769             1	                      3.8                   52.2
        193	            1                         4.3                   47
        47	            1                         4.1                   45.5
        13	            1                         4.4                   45
    
    Based on these results a ht_size of 47 (with resize_interval==1) has
    been picked - this gives rebalance time as good as current; but with a
    39% reduction in initial memory usage.
    
    Change-Id: I329e18240bace9f4905e3eb0c4d59abd97dd8cd1
    Reviewed-on: http://review.couchbase.org/77892
    Reviewed-by: Jim Walker <jim@couchbase.com>
    Reviewed-by: Manu Dhundi <manu@couchbase.com>
    Tested-by: Build Bot <build@couchbase.com>
Commits on May 23, 2017
  1. MB-24472 [Ephemeral]: Check for null vBucket in getNumPersistedDeletes

    daverigby committed May 22, 2017
    Check the vBucket obtained from the vbmap is non-null before
    dereferencing. If it is null, throw std::runtime_error.
    
    Change-Id: I181d8b7d696386bee5b18daa8b211cbc7238d87b
    Reviewed-on: http://review.couchbase.org/78406
    Tested-by: Build Bot <build@couchbase.com>
    Reviewed-by: Daniel Owen <owend@couchbase.com>
  2. MB-16181: Add Collections Filter classes and test

    jimwwalker authored and daverigby committed Mar 14, 2017
    Two classes exist for filtering.
    
    Collections::Filter
    Collections::VB::Filter
    
    The idea is that a DCP producer will establish a Collections::Filter
    that lives for the lifetime of the DCP producer.
    
    As the DCP producer creates streams, a Collections::VB::Filter is
    assigned to the stream which contains the real set of collections to
    filter (and also the actual "filter" function).
    
    Change-Id: I2f35b1698ce977116486a2e6940437eee25faef1
    Reviewed-on: http://review.couchbase.org/78137
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: Build Bot <build@couchbase.com>
  3. MB-16181: Build SystemEvent keys with the collections separator

    jimwwalker authored and daverigby committed Mar 23, 2017
    The keys were fixed as $collections::<event> but are now changed
    so that the :: is the collections separator.
    
    This allows code to split the event key if they wish using the
    same code they would split document keys.
    
    Change-Id: I48575d295f8c058a79cf208fe3c9d3a9b3c9ed15
    Reviewed-on: http://review.couchbase.org/78136
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: Build Bot <build@couchbase.com>
  4. MB-16181: Transfer the collection name over DCP

    jimwwalker authored and daverigby committed Mar 21, 2017
    The DCP mutation/deletion callbacks now take a collection_len field,
    the data in this field will be sent over DCP streams when a client
    has signalled they want collection-aware DCP.
    
    For example "dairy::cheese" will set a collection length of 5,
    default collection documents, set a collection length of 0.
    
    Change-Id: I303d9b18bc5d0fd0968708d84e461ee59577c003
    Reviewed-on: http://review.couchbase.org/78135
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: Build Bot <build@couchbase.com>
  5. MB-16181: Add collections.set_manifest support

    jimwwalker authored and daverigby committed Mar 10, 2017
    Add a method which will accept the new manifest and apply it to
    all active vbuckets.
    
    The latest manifest is saved in memory and also used for when any VB
    is set to active
    
    Change-Id: Ic6a339bc5af279d105b679f528ff3675d1f16ac7
    Reviewed-on: http://review.couchbase.org/77436
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: Build Bot <build@couchbase.com>
  6. [Fix coverity warning]: Initialize all members of RangeIteratorLL

    manudhundi authored and daverigby committed May 22, 2017
    We must always initialize all the non static members of the RangeIteratorLL
    class in the constructor.
    
    Below is the warning reported by coverity:
    *** CID 169606:  Uninitialized members  (UNINIT_CTOR)
    /ep-engine/src/linked_list.cc: 430 in BasicLinkedList::RangeIteratorLL::RangeIteratorLL(BasicLinkedList&)()
    424         std::lock_guard<std::mutex> listWriteLg(list.getListWriteLock());
    425         std::lock_guard<SpinLock> lh(list.rangeLock);
    426         if (list.highSeqno < 1) {
    427             /* No need of holding a lock for the snapshot as there are no items;
    428                Also iterator range is at default (0, 0) */
    429             readLockHolder.unlock();
       CID 169606:  Uninitialized members  (UNINIT_CTOR)
       Non-static class member "earlySnapShotEndSeqno" is not initialized in this constructor nor in any functions that it calls.
    430             return;
    431         }
    432
    433         /* Iterator to the beginning of linked list */
    434         currIt = list.seqList.begin();
    
    Change-Id: Ie3e26d685159e24054ad94d8f04bc455c8e3c4f2
    Reviewed-on: http://review.couchbase.org/78425
    Tested-by: Build Bot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
Commits on May 22, 2017
  1. MB-24294: Stop couch-kvstore changing the file revision

    jimwwalker authored and daverigby committed May 11, 2017
    There's a lot of code around the open paths which may open a different
    file-revision to the one requested, which is the cause of MB-24294.
    
    In summary we asked for 0.couch.6, which doesn't yet exist and the
    checkNewRevNum function (now removed) would then look for 0.couch.*
    files and depending on the async delete task's progress may find
    0.couch.5 and direct saveDocs into that VB, which is about to be
    deleted...
    
    The main fix here is to remove checkNewRevNum, if we have a
    dbFileRevMap value of n, we should open/create 0.couch.n and never
    anything else.
    
    This lead onto finding an issue where the RO instance of CouchKVStore
    was relying on populating it's "copy" of the revision map via failing
    to open 0.couch.0 and then looking for 0.couch.* and updating the map
    from what it finds. So this is fixed by having a single dbFileRevMap
    which is now referenced by the RW/RO pair.
    
    Change-Id: I03dbb0c0a3e635397a12d85ea6092aa1312354b7
    Reviewed-on: http://review.couchbase.org/78024
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: Build Bot <build@couchbase.com>
Commits on May 19, 2017
  1. MB-24376: Stop rangeRead on invalid Seqno

    jameseh96 authored and daverigby committed May 19, 2017
    An exception is thrown if the start of the readRange is updated with a
    value x < 0 || x > endSeqno, as this should not be done.
    
    When updating an item which is currently covered by a rangeRead, the
    following states occur in the seqList:
    
     A₁ B₂ C₃        Initial items
    [A₁ B₂ C₃]       rangeRead to highSeqno
    
    { Write Lock
    
    [A₁ B₂ C₃] A₋₁   ** Update; A₁ marked stale, new OSV appended.
                     NB: this is before queueDirty updates seqNo
    
    [A₁ B₂ C₃] A₄    Seqno updated in queueDirty
    
    }
    
    RangeReads do not hold the write lock, and stop only upon either
    reaching the end of the seqList, or reading an item with
    seqno > endSeqno.
    There is a brief window at the state marked ** in which a rangeRead
    may try to update the start of the readRange with -1:
    
    - rangeRead (endSeqno 3) reads C₃ normally
    - proceeds to next item in the seqList
    - checks if seqno > endSeqno - !(-1 > 3), don't stop yet
    - update the start of the readRange with -1
    
    This patch ends the readRange if the current item has a seqno < 0.
    
    Change-Id: Ia1e53e4c40705b673c1f01e1b15fecbed5dff2fc
    Reviewed-on: http://review.couchbase.org/78353
    Tested-by: Build Bot <build@couchbase.com>
    Reviewed-by: Manu Dhundi <manu@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
  2. MB-24426: Use virtual destructors for some of the DCP classes

    manudhundi authored and daverigby committed May 18, 2017
    DCP classes like 'DCPProducer', 'DCPConsumer', 'Consumer',
    'ActiveStream' and 'PassiveStream' have subclasses derived from them.
    It is appropriate to use virtual destructors to avoid any memory
    leaks during object deletion.
    
    Change-Id: I8b8457095b7c9914ca2c3d4b18a2251833e6c770
    Reviewed-on: http://review.couchbase.org/78334
    Tested-by: Build Bot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
Commits on May 18, 2017
  1. MB-24151: Implement memory managed backfills using Range Iterators

    manudhundi authored and daverigby committed May 17, 2017
    This commit implements memory management in the backfills of
    Ephemeral buckets. It borrows the idea from disk backfills where
    the backfill is suspended upon high memory usage, that is items
    are not put onto the readyQ of the stream.
    
    At the core is a Backfill state machine which is driven by
    the BackfillManagerTask. It keeps track of any suspended backill
    and also resumes from the suspended point. It also creates a range
    iterators on the in-memory sequence list and using this iterator
    it reads (backfills) items one by one.
    
    This commit also adds certain features to the sequence list iterators
    to get snapshot numbers, estimate num items in backfill etc.
    
    Change-Id: Id92b0693763e550f842fb7fb5911cfefd8935e79
    Reviewed-on: http://review.couchbase.org/78055
    Tested-by: Build Bot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
  2. MB-24430: Set the value correctly for an expired item in memory

    sriganes authored and daverigby committed May 17, 2017
    If an item is found to be expired in the hash table, then its value
    needs to be set correctly. For example, in case the item contains
    xattrs, the system xattrs will be retained in the value and hence
    needs to be set correctly
    
    Change-Id: I99a54ca82feb837f530b18eb3f245a9d92f12369
    Reviewed-on: http://review.couchbase.org/78220
    Tested-by: Build Bot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
  3. MB-16181: Add DCP system-event engine hooks

    jimwwalker authored and daverigby committed Mar 10, 2017
    Plug methods into the engine callbacks so that we can now have
    system events pushed in from memcached.
    
    Change-Id: I064b86542b2ab98f80e33097afe8a60242fd7147
    Reviewed-on: http://review.couchbase.org/77435
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: Build Bot <build@couchbase.com>
Commits on May 17, 2017
  1. MB-24294: Clean-up couch-kvstore openDB with reset=true

    jimwwalker authored and daverigby committed May 11, 2017
    The openDB reset flag is only used by the CouchKVStore::reset method
    when it calls set-state.
    
    This is no longer required because of the changes in MB-23714 (which
    also modified CouchKVStore::reset). There is no need to move the DB
    file revision back to 1, it should be monotonically incrementing.
    
    Change-Id: I50ba9873041157923ed99f34623f93ef67527641
    Reviewed-on: http://review.couchbase.org/78007
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: Build Bot <build@couchbase.com>
  2. Decouple StoredValue and HashTable class

    sriganes authored and daverigby committed May 15, 2017
    A StoredValue object doesn't necessarily have to be part of
    a HashTable. Hence, decouple the two
    
    Change-Id: I6b88a3e010494989d3c1ad938c329a230c79d91d
    Reviewed-on: http://review.couchbase.org/78170
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: Build Bot <build@couchbase.com>
  3. Improve test duration under Valgrind

    jameseh96 authored and daverigby committed May 12, 2017
    Compared to the parent commit, this change reduces the run time of tests
    under Valgrind from 1051s (17m31) to 295s (4m55) - roughly a 72%
    reduction.
    
    This reduction comes from the teardown phase of a number of tests.
    When destroying an ExecutorPool, all the taskables must be unregistered.
    
    Prior to this change, when unregistering the last taskable from an
    ExecutorPool, all threads were woken and prevented from sleeping (by
    pretending there is work, numReadyTasks[idx]++) /before/ they were
    marked as dead. During this time the woken threads all busy-wait for
    work.
    
    Usually this wouldn't be too problematic because the main thread would
    mark them dead soon after. However, under Valgrind only one thread can
    execute at a time, and it can take significant time before the main
    thread is able to mark all of the ExecutorThreads as dead.
    
    This change simply marks them dead before waking them. They check their
    status, find it is "dead", and exit.
    
    Change-Id: Ic88ccec46ac26d511a18a1370d20117af173703b
    Reviewed-on: http://review.couchbase.org/78083
    Tested-by: Build Bot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
  4. MB-16181: Make server deny creation of reserved collections

    jimwwalker authored and daverigby committed May 12, 2017
    The design has always stated that _ and $ would be reserved for
    system use. Make sure the server enforces this by checking the
    names on incoming manifests.
    
    1. Any _ prefixed collection will cause the Manifest construction to
     throw invalid_argument.
    
    2. Any $ prefixed collection which is not $default will cause the
     Manifest construction to throw invalid_argument.
    
    Change-Id: I1e5daf2ae87cba2a8dbcdda4c9bb0be66e40ffae
    Reviewed-on: http://review.couchbase.org/78074
    Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: Build Bot <build@couchbase.com>
  5. MB-24094: Add Item to SequenceList on TTL update

    premkumr authored and daverigby committed May 11, 2017
    In the GAT Path, when the item is expired, we update the expiry
    time. In the case of ephemeral buckets, that item should be updated on
    the sequence list.
    
    Change-Id: I2b83456e53cb2d2e4d762d939a716c39c9a725bd
    Reviewed-on: http://review.couchbase.org/78032
    Tested-by: Build Bot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
Commits on May 16, 2017
  1. MB-24054: Fix ref to temp string in testsuite_dcp

    OliverMD authored and daverigby committed May 15, 2017
    This patch ensures we pass a std::string to the initializer list rather
    than doing a pointless call to c_str() which just confuses MSVC2015.
    
    Change-Id: Idb29db461a1d2572660622d1166c680b599e2b57
    Reviewed-on: http://review.couchbase.org/78134
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: Build Bot <build@couchbase.com>
  2. MB-22602: Use CMake variables for target paths

    daverigby committed May 16, 2017
    Instead of manually specifying the path to various targets, use the
    appropriate CMake variable / generator expression. This has the
    advantage that if a target is moved to a different location, it will
    still be correctly referred to.
    
    In the case of ADD_TEST(), use the variant which specifies the NAME
    option - this enables expansion of executable targets to their
    absolute path.
    
    Change-Id: I625b9908824969010b7535f2df5a69456d3d5ae4
    Reviewed-on: http://review.couchbase.org/78181
    Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
    Tested-by: Build Bot <build@couchbase.com>
Commits on May 15, 2017
  1. MB-24244: Prevent duplicate items in an inMemoryBackfill

    jameseh96 authored and daverigby committed May 9, 2017
    Have rangeRead ignore stale items if their replacement also appears in
    the requested range, for example
    
     A₁  B₂  C₃           Initial items
    [A₁  B₂  C₃]          rangeRead 1-3
    [A₁  B₂  C₃] B₄       Update B
     A₁  B₂  C₃  B₄       RR ends
    [A₁  B₂  C₃  B₄]      new RR 1-4, B₂ should be ignored because B₄ is
    also in the snapshot
    [A₁  B₂  C₃  B₄] B₅   Another update, but B₄ must still be sent as B₅ is
    not in the range
     A₁  B₂  C₃  B₄  B₅   RR ends
    
    To achieve this, when marking items as stale, we store in it a pointer
    to its replacement. To avoid bloating the OSV, we reuse the UniquePtr
    which points to the items successor when in the HashTable. Once the item
    is stale, it will never be used for that purpose again, leaving it free
    for reuse.
    
    This is a basic solution. It would be better to not have to take the
    writeLock, but improvements of that form will be done in a separate
    patch.
    
    Change-Id: Id743da606e009d17f5e5af6f9344223a95aa4a38
    Reviewed-on: http://review.couchbase.org/78014
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: Build Bot <build@couchbase.com>
  2. MB-16181: Interface update, allow DCP open to accept a value

    jimwwalker authored and trondn committed Mar 14, 2017
    DCP consumers will be able to open a DCP stream and specify a filter,
    a list of collections they are interested in. This filter will be
    passed as a JSON 'value' to DCP open.
    
    Change-Id: Id33306f0d663b263840f70c44a16deff6db0a89e
    Reviewed-on: http://review.couchbase.org/78038
    Tested-by: Build Bot <build@couchbase.com>
    Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Commits on May 12, 2017
  1. MB-24246: Combine 'writeLock' & 'highSeqnosLock' into one in BasicLin…

    manudhundi committed May 12, 2017
    …kedList
    
    Functionally 'writeLock' and 'highSeqnosLock' both result in the
    serialization of the BasicLinkedList write and other list operations.
    Hence this commit combines the 2 locks into one lock.
    
    As a consequence this commit ensures that the writeLock is held on
    the BasicLinkedList until the sequence number for the newly added,
    updated and soft-deleted is generated. While this is strictly not
    needed in new add, it is necessary in update and soft delete as
    explained below.
    
    (a) New Add :
       A1, B2, C3 and we are trying to add D4
       A1, B2, C3, D? added but no seqno yet.
       Now we don't really care if range read starts here as it can read
       only A1, B2, C3 in the snapshot.
    
       But to maintain uniformity (hence simpler code) with update
       and soft-delete we grab writeLock such that we append to list
       + update highSeqno as an atomic operation.
    
    (b) Update (and Soft-delete):
       A1, B2, C3 and we are trying to update B to B4
       A1, B2, C3, B? added but no seqno yet and/or no B2_stale yet
       Now we cannot start the range read here because we do not wait
       for range read to finish to mark the item stale (note that we
       are planning to not send stale(duplicate) items in a snapshot).
    
       This was already in case (b) using highSeqnosLock, this commit
       uses just writeLock for the same.
    
    The commit also adds documentation regarding the desired heirarchy of
    the lock acquisition in BasicLinkedList.
    
    Change-Id: I10a80f55a763d1496adec24fa12bc75d83ea1573
    Reviewed-on: http://review.couchbase.org/77958
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: Build Bot <build@couchbase.com>
  2. Use 'const' for the member 'Item* value' in class 'GetValue'

    manudhundi authored and daverigby committed May 11, 2017
    Class GetValue contains a pointer to an item which is only used to
    transfer out the item contained from one layer to another. It does not/
    should not change the item it contains.
    
    Hence it would be best to explicitly use const so that the item
    contained is not modified by mistake in the future.
    
    Further, it could be a future task to use std::unique_ptr<Item> in
    GetValue to explicitly transfer the ownership of the item that
    gets handed over from one layer to another.
    
    Change-Id: Ic0a880f3070e299e8ff1628f3ac329aaa1a1b0cb
    Reviewed-on: http://review.couchbase.org/78040
    Tested-by: Build Bot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
  3. MB-24055: Change ht_size to same value as ns_server (3079)

    daverigby committed May 9, 2017
    In preparation for removing the default value for ht_size from
    ns_server where it currently is set (so KV-Engine owns the default),
    change the value in configuration.json from 3 to 3079 - matching
    ns_server's current default.
    
    Note that this value is expected to be subsequently changed (reduced)
    again as part of MB-24055 (to reduce memory overhead).
    
    Change-Id: I1025709029dc2e6807e859c27f6ae6725efb7f53
    Reviewed-on: http://review.couchbase.org/77890
    Tested-by: Build Bot <build@couchbase.com>
    Reviewed-by: James Harrison <james.harrison@couchbase.com>
  4. MB-24055: remove defaultNum{Buckets,Locks}, use config.json instead

    daverigby committed May 9, 2017
    HashTable defines it's own set of default values for the hash table
    initial size and number of locks. This appears to be a remnant of a
    time before the central configuration.json we now know and love.
    
    Having both is over-complex and confusing - it might /appear/ that the
    default ht_size is 47 (from configuration.json), but in fact the
    default is 3 - as hard-coded in hash_table.cc
    
    Given we have config.json now, just use that as the holder of the
    default sizes (and which can be overridden by ns_server if
    desired). As such HashTable now requires an explicit initialSize and
    nLocks in the constructor - which in normal usage will simply come
    from the config.
    
    Note that there should be no material change in the values used for
    ht_locks or ht_size when run as a full Couchbase Server instance - if
    a value is passed in by ns_server (via bucket config string) that will
    be used; otherwise the previous values of defaultNum{Buckets,Locks} -
    3 and 193 respectively - are used. In unit tests, any previously
    overridden values have been updated here to the same values.
    
    Change-Id: I8e3067014714a48acf1954310fdf13dce0b731d7
    Reviewed-on: http://review.couchbase.org/77886
    Reviewed-by: James Harrison <james.harrison@couchbase.com>
    Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
    Tested-by: Build Bot <build@couchbase.com>
  5. CTest: Increase maximum output size recorded to 1MB

    daverigby committed May 9, 2017
    While tests run via Jenkins record the output of each test, the size
    recorded is extremely limited - only 1KB for test which pass. This can
    be problematic when (for example) one is trying to analyse which tests
    in a suite were slow - for many of our test programs we run many
    (sometimes hundreds) of tests, and hence 1024 characters quickly cuts
    off interesting data.
    
    As such, increase the limit to 1MB.
    
    Change-Id: I5b2e5e966507951e01e088829120931a0e3bef11
    Reviewed-on: http://review.couchbase.org/77893
    Reviewed-by: James Harrison <james.harrison@couchbase.com>
    Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
    Tested-by: Build Bot <build@couchbase.com>
Commits on May 11, 2017
  1. MB-24055: change defaultNumLocks from 193 to 47 (logical no-op).

    daverigby committed May 9, 2017
    Change the value of HashTable::defaultNumLocks from 193 to 47. Note
    this is a no-op in a ns_server-controlled configutation as it already
    explicilty sets the number of ht_locks to 47; therefore the only time
    193 is used is in unit tests.
    
    The reason for changing to 47 is to simplify testing - (1) we should
    ideally test in a configuration as close to the "full" system, and (2)
    we actually need to reduce the number of locks to <64 so we can run
    successfully under ThreadSanitizer.
    
    Change-Id: I1bcecc86cc90fbac215b8ee231179bee5d8cca32
    Reviewed-on: http://review.couchbase.org/77889
    Tested-by: Build Bot <build@couchbase.com>
    Reviewed-by: James Harrison <james.harrison@couchbase.com>
    Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Commits on May 10, 2017
  1. MB-24151: Implement general range read iterator in seqlist

    manudhundi committed May 10, 2017
    This commit implements range read iterator in sequence list, the ordered
    data strucuture in Ephemeral buckets.
    
    We need range read iterators because we have more than one client that
    needs to do a range read (backfill, tombstone purger). Further the reads
    can also have more involved constraints like memory management in backfills.
    By having an API of read iterator we can just do the read from the sequence
    list and additional stuff like memory management can be done outside by the
    clients. That is, iterator clients can make progress at their own pace.
    
    The iterator is unidirectional (forward only) and cannot be invalidated
    while in use. That means reading the items in the iterator results in a
    valid point-in-time snapshot. But this adds a caveat that while an
    iterator instance is active, certain invariants are to be met to get the
    snapshot, and this happens at the expense of increased memory usage.
    
    For now, only one iterator can be created at a time.
    
    Change-Id: Idb320a148299255b74b7cf7e92cef35a20f483d4
    Reviewed-on: http://review.couchbase.org/77640
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: Build Bot <build@couchbase.com>