Commits on Jul 21, 2016
  1. Set the vbucket state after warmup is complete

         ==================
         WARNING: ThreadSanitizer: data race (pid=49270)
         Read of size 8 at 0x7d900001c000 by thread T4:
         #0 vbucket_state** std::__uninitialized_copy<false>::__uninit_copy<__gnu_cxx::__normal_iterator<vbucket_state* const*, std::vector<vbucket_state*, std::allocator<vbucket_state*> > >, vbucket_state**>(__gnu_cxx::__normal_iterator<vbucket_state* const*, std::vector<vbucket_state*, std::allocator<vbucket_state*> > >, __gnu_cxx::__normal_iterator<vbucket_state* const*, std::vector<vbucket_state*, std::allocator<vbucket_state*> > >, vbucket_state**) /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/bits/stl_construct.h:75 (ep-engine_ep_unit_tests+0x0000006e3980)
         #1 Warmup::populateShardVbStates() /home/couchbase/jenkins/workspace/ep-engine-threadsanitizer-master/ep-engine/src/warmup.cc:1030 (ep-engine_ep_unit_tests+0x00000067999a)
         #2 Warmup::initialize() /home/couchbase/jenkins/workspace/ep-engine-threadsanitizer-master/ep-engine/src/warmup.cc:459 (ep-engine_ep_unit_tests+0x0000006797ec)
    
          Previous write of size 8 at 0x7d900001c000 by thread T12 (mutexes: write M119516, write M119521):
          #0 KVStore::updateCachedVBState(unsigned short, vbucket_state const&) /home/couchbase/jenkins/workspace/ep-engine-threadsanitizer-master/ep-engine/src/kvstore.cc:122 (ep-engine_ep_unit_tests+0x0000006ade1c)
          #1 ForestKVStore::snapshotVBucket(unsigned short, vbucket_state&, VBStatePersist) /home/couchbase/jenkins/workspace/ep-engine-threadsanitizer-master/ep-engine/src/forest-kvstore/forest-kvstore.cc:1413 (ep-engine_ep_unit_tests+0x0000006e4249)
          #2 EventuallyPersistentStore::persistVBState(unsigned short) /home/couchbase/jenkins/workspace/ep-engine-threadsanitizer-master/ep-engine/src/ep.cc:1355 (ep-engine_ep_unit_tests+0x0000005b0fb8)
          #3 VBStatePersistTask::run() /home/couchbase/jenkins/workspace/ep-engine-threadsanitizer-master/ep-engine/src/tasks.cc:55 (ep-engine_ep_unit_tests+0x000000660a55)
          #4 ExecutorThread::run() /home/couchbase/jenkins/workspace/ep-engine-threadsanitizer-master/ep-engine/src/executorthread.cc:113 (ep-engine_ep_unit_tests+0x000000616ef1)
          #5 launch_executor_thread(void*) /home/couchbase/jenkins/workspace/ep-engine-threadsanitizer-master/ep-engine/src/executorthread.cc:31 (ep-engine_ep_unit_tests+0x000000616aa5)
          #6 platform_thread_wrap(void*) /home/couchbase/jenkins/workspace/ep-engine-threadsanitizer-master/platform/src/cb_pthreads.cc:54 (libplatform.so.0.1.0+0x00000000586b)
    
    Change-Id: I4e807ac9a23baf9e4d352f13ef65f3f5a345cd60
    Reviewed-on: http://review.couchbase.org/66050
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Sriram Ganesan committed Jul 20, 2016
Commits on Jul 20, 2016
  1. Synchronize access to cachedVBStates between a flusher task commit an…

    …d a snapshot task
    
    SUMMARY: ThreadSanitizer: data race /home/couchbase/jenkins/workspace/ep-engine-threadsanitizer-master/ep-engine/src/kvstore.cc:289 vbucket_state::toJSON() const
     ====================================
    WARNING: ThreadSanitizer: data race (pid=209164)
    Read of size 8 at 0x7d1400009e50 by thread T16 (mutexes: write M16145):
    
    Previous write of size 8 at 0x7d1400009e50 by thread T18 (mutexes: write M16141):
    
    Change-Id: Ie5c41c4e4ba0219903b85047787ece373c6a8231
    Reviewed-on: http://review.couchbase.org/65988
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: Sriram Ganesan <sriram@couchbase.com>
    Sriram Ganesan committed Jul 19, 2016
Commits on Jul 19, 2016
  1. @jimwwalker @daverigby

    MB-20061: Improved futureQueue heap property fix

    MB-9986 found that we break the heap property of the underlying
    container of the std::priority_queue used in scheduling. This is
    because we use the wakeTime to sort the queue, but then later
    change the value of a task which is still in the queue, thus
    order is broken. MB-9986 emptied the futureQueue and rebuilt it.
    
    In this fix we utilise the C++ standard which shows that we can
    access the container of the priority_queue. Thus we can sub-class
    it and create a method that allows us to update a tasks time and
    maintain the heap order.
    
    Change-Id: I238d36ea684d59ef06326183fa1f16c04f8d29ad
    Reviewed-on: http://review.couchbase.org/65394
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Manu Dhundi <manu@couchbase.com>
    Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
    jimwwalker committed with daverigby Jun 30, 2016
Commits on Jul 18, 2016
  1. Return error when invalid argument is passed to ForestDB fileOps

    In case an invalid handle is passed to the fileOps callback
    functions, return FDB_RESULT_INVALID_ARGS error to caller.
    
    Change-Id: Ibaaa30e805b8d3a427071df0617eb327e5205235
    Reviewed-on: http://review.couchbase.org/65872
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Sriram Ganesan committed Jul 16, 2016
Commits on Jul 15, 2016
  1. @hisundar

    Set the log callback for a ForestDB KVStore handle

    ForestDB doesn't have an explicit log file where error messages are
    logged. Hence, explicitly set a log callback for a ForestDB
    KVStore handle.
    
    Change-Id: Idd76fc8bf8b38dea5b0f979d8799abf2a72d478c
    Reviewed-on: http://review.couchbase.org/65816
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
    Sriram Ganesan committed with hisundar Jul 14, 2016
Commits on Jul 14, 2016
  1. Address text_MB-19687_fixed for forestdb as backend

    ForestKVStore doesn't have a separate read-only and read-write
    instance. CouchKVStore has separate statistics for read-only
    and read-write statistics. Make the statistics common to all
    KVStore instances. For specific instances, these statistics
    just won't get set.
    
    Change-Id: Id94f69061a99be6fdb890a9087a25134db85ae84
    Reviewed-on: http://review.couchbase.org/65640
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Sriram Ganesan committed Jul 9, 2016
Commits on Jul 12, 2016
  1. Skip test_tap_sends_deleted with forestdb as backend

    This test times out with forestdb as backend because deleted items
    don't get reflected in the final item count as forestdb doesn't
    provide an accurate item count until a wal flush takes place.
    
    Change-Id: I11df93c5f8c3682af25f78bbdd1b922e6f0872f0
    Reviewed-on: http://review.couchbase.org/65642
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: buildbot <build@couchbase.com>
    Sriram Ganesan committed Jul 9, 2016
  2. Perform an explicit commit after call to flushVBucket

    Although a flushVBucket writes the item to the underlying store,
    the item is not marked clean until an explicit commit is called.
    If the underlying store is couchstore, a commit is called with
    a flushVBucket but in the case of forestdb, a commit is not
    always called, hence call an explicit commit.
    
    Change-Id: I2feee981f68b9302b9c65d002dae55a19d1da4c2
    Reviewed-on: http://review.couchbase.org/65670
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: buildbot <build@couchbase.com>
    Sriram Ganesan committed Jul 11, 2016
Commits on Jul 8, 2016
  1. MB-19952: Ensure that simultaneous tasks use separate file/kvstore ha…

    …ndles
    
    ForestDB doesn't allow separate file and KV store handles be used
    simultaneously by 2 different threads. As part of this commit, the
    following changes are made to ForestKVStore
    
    - Use separate file handles for readers and writers.
    - Synchronize the writer tasks (flusher, snapshot, delete vbucket, flush bucket).
    - Open a separate file and kvstore handle for the infrequent tasks. For
      example, stat calls, backfill, getAllKeys.
    - There is a possibility that as compaction is running, a file is being moved
      from 0.fdb.1 to 0.fdb.2. A thread trying to open a handle on 0.fdb.1 could
      fail as soon as compaction completes. In order to prevent that, use a
      handleLock to synchronize closing the handle before compaction and a
      separate thread opening a new handle. As long as there is an open
      file handle on the file, the file should ideally not be removed
      according to ForestDB semantics.
    
    Change-Id: I34e4b610755d694e72e2d5080f6eac52e6afebdc
    Reviewed-on: http://review.couchbase.org/65466
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Sriram Ganesan committed Jun 21, 2016
  2. @daverigby

    Add setWithMeta tests to EPStoreEvictionTest

    Change-Id: I0f09624d84d01f5c2f1e2cb1cbfb6f4f3e8b82b7
    Reviewed-on: http://review.couchbase.org/64450
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Will Gardner <will.gardner@couchbase.com>
    daverigby committed Apr 28, 2016
Commits on Jul 7, 2016
  1. @manudhundi

    MB-20022: In DCP consumer, while adding new str on a vb, erase dead s…

    …tr if any
    
    In a DCP consumer we store streams in a map with vbid as the key. Before
    adding a new steam, we must erase any existing dead on that vb. Otherwise the
    new stream created will not be stored in the map and hence would be deleted
    immediately.
    
    Change-Id: Ie154133464db86364a9b6a17074f93485ffc1b7a
    Reviewed-on: http://review.couchbase.org/65421
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Will Gardner <will.gardner@couchbase.com>
    manudhundi committed Jul 7, 2016
  2. @daverigby

    Add 'add' tests to EPStoreEvictionTest

    Change-Id: I4a954eb1ca1b1fbb716c7d8de46dc6f58fe65bcc
    Reviewed-on: http://review.couchbase.org/64449
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Will Gardner <will.gardner@couchbase.com>
    daverigby committed Apr 28, 2016
  3. @daverigby

    Add set tests to EPStoreEvictionTest

    Change-Id: Ieb9dba8fc672eb089b49953e3cbd4b30174436ef
    Reviewed-on: http://review.couchbase.org/63462
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Will Gardner <will.gardner@couchbase.com>
    daverigby committed Apr 28, 2016
Commits on Jul 6, 2016
  1. @daverigby

    Add replace tests to EPStoreEvictionTest

    Test replace() operation in various eviction modes / scenarios.
    
    Change-Id: Ie86326e11bed27c41dd33b8d3698f3fcdffc31de
    Reviewed-on: http://review.couchbase.org/63461
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Jim Walker <jim@couchbase.com>
    daverigby committed Apr 28, 2016
Commits on Jul 4, 2016
  1. @daverigby

    MB-14193: Fix Windows build break due to missing CMake dependency

    Change-Id: Ibe41e530f01440db13c7351392073513d0273efe
    Reviewed-on: http://review.couchbase.org/65481
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    daverigby committed Jul 4, 2016
Commits on Jul 1, 2016
  1. @manudhundi

    MB-14193: Implement file ops for ForestKVStore

    Implement infrastructure to extract statistics from operations
    performed on forestdb files
    
    Change-Id: I451795e5eda86e5000e7eb4b78f4323383237e8d
    Reviewed-on: http://review.couchbase.org/63335
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
    manudhundi committed Jul 1, 2016
Commits on Jun 27, 2016
  1. Rename backfills map to scans maps as the scan context is more generic

    Performing a scan over the items over the KVStore is more generic
    and not strictly attached to backfill. Hence, rename backfills to
    scans.
    
    Change-Id: Iba5c9202bfba5a206066c0ecc57f95718a72c610
    Reviewed-on: http://review.couchbase.org/65248
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: buildbot <build@couchbase.com>
    Sriram Ganesan committed Jun 26, 2016
  2. MB-19976: Reschedule vbucket deletion in case compaction is running

    ForestDB doesn't support removal of a KV store when compaction is
    running on the same file. So, reschedule vbucket deletion as a
    workaround.
    
    Change-Id: Iaece1030686b7257e07c50b607d6e7156605f8fb
    Reviewed-on: http://review.couchbase.org/65214
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Sriram Ganesan committed Jun 24, 2016
Commits on Jun 24, 2016
  1. @trondn @daverigby

    Fixup: Use enum class for format types.

    Change-Id: I369cffa876ffdb99ff3571e7e653508aed8a818d
    Reviewed-on: http://review.couchbase.org/65206
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Will Gardner <will.gardner@couchbase.com>
    trondn committed with daverigby Jun 24, 2016
Commits on Jun 23, 2016
  1. MB-19978: fdb operations should be done on old file during compaction

    When a file is being compacted, all the forestdb operations should
    be performed on the newly compacted file only after compaction
    completes.
    
    Change-Id: I2850e82e27e43e92367bb5003c2185318cc69faf
    Reviewed-on: http://review.couchbase.org/65177
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Sriram Ganesan committed Jun 22, 2016
  2. @mattcarabine @daverigby

    MB-19746: Add xml output to ep_perfsuite

    - The new performance commit validation, CBNT, requires input data
      in XML format which mirrors that of the GTest XML output
    - This commit adds this functionality to ep_perfsuite, providing
      the ability to output the results in textual format or xml
    - It can be invoked by using the new `-f` flag, followed by either
      `xml` or `text`, if this flag is not specified then the default
      behaviour of textual output is assumed
    
    Change-Id: I153a816493181acb091e8d4433f5be543742231f
    Reviewed-on: http://review.couchbase.org/65005
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    mattcarabine committed with daverigby Jun 17, 2016
Commits on Jun 21, 2016
  1. Make database file revision number as atomic

    KV-engine increments a file revision number every time a ForestDB
    file gets compacted. As a file gets compacted, another thread might
    want to open a separate database file handle and perform a forestdb
    operation. In this case, the other thread needs to atomically read
    the file revision number in order to construct the file name.
    
    Change-Id: I75a8eeecfc49968529bfb83a15f3bcd4937c0d0f
    Reviewed-on: http://review.couchbase.org/65082
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Reviewed-by: Will Gardner <will.gardner@couchbase.com>
    Sriram Ganesan committed Jun 20, 2016
Commits on Jun 20, 2016
  1. @hisundar @hisundar

    Add option EP_DEBUG_BUILD to turn off optimizations

    Change-Id: Ib792aae7d61100864ca08be801a37eecc89a9a80
    Reviewed-on: http://review.couchbase.org/64918
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: Sundararaman Sridharan <sundar@couchbase.com>
    hisundar committed with hisundar Jun 13, 2016
  2. @manudhundi

    MB-19929: Fix sporadic failure in 'dcp cursor dropping' test

    In cursor dropping test the max_size and chk_max_items should be such
    that 2 checkpoints are created. The test simulates a case where of
    the 2 checkpoints, the DCP cursor from one of them is dropped.
    
    Change-Id: Ieb5c6dfba43bea2f8256fa01104f9ddf83a9781f
    Reviewed-on: http://review.couchbase.org/65028
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Will Gardner <will.gardner@couchbase.com>
    manudhundi committed Jun 17, 2016
Commits on Jun 17, 2016
  1. @jimwwalker @daverigby

    MB-19948: Return an error code for invalid metadata

    The current code throws an exception (previously an assert).
    However we should return an error code (which gets logged)
    and not throw for such errors.
    
    Change-Id: I2ee7967d069075417200e549b7a93450dca580e6
    Reviewed-on: http://review.couchbase.org/64975
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: buildbot <build@couchbase.com>
    jimwwalker committed with daverigby Jun 16, 2016
Commits on Jun 16, 2016
  1. @daverigby

    ep_unit_tests: Correctly enable DEBUG level logging

    debug-level logging was not correctly enabled - while the ep-engine
    Logger class was set to DEBUG, the underlying mock server API's logger
    level was still at the default (INFO), so no debug messages were
    printed to stderr.
    
    Change-Id: I4e6e5e91459bfeef80f1e9bca0256767f7e811a6
    Reviewed-on: http://review.couchbase.org/64927
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Manu Dhundi <manu@couchbase.com>
    daverigby committed Jun 14, 2016
Commits on Jun 14, 2016
  1. @daverigby

    Remove misleading comments from PassiveStream::process{Mutation,Delet…

    …ion}
    
    These comments imply that errors during
    PassiveStream::process{Deletion,Mutation} are not handled; however
    this is incorrect - the caller checks the return value and re-queues
    the mutations in the event of an error.
    
    Change-Id: I25719a7b5dede5512528acd8f1643eb12f83fe47
    Reviewed-on: http://review.couchbase.org/64602
    Reviewed-by: Will Gardner <will.gardner@couchbase.com>
    Tested-by: buildbot <build@couchbase.com>
    daverigby committed May 31, 2016
Commits on Jun 13, 2016
  1. @manudhundi

    MB-19833: Set a minimum boundary for config param bfilter_key_count

    We can have a 'divide by zero' error if we have bfilter_key_count set
    to 0 in configuration. Further it does not make sense to have the
    param, which expects estimated key count per vbucket, to be set to 0.
    
    This commit hence adds a validator to the config param to make sure
    it is not below 1.
    
    Change-Id: I5f361e87e053121b56a77593daab6c46990562dc
    Reviewed-on: http://review.couchbase.org/64914
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Will Gardner <will.gardner@couchbase.com>
    manudhundi committed Jun 13, 2016
  2. Perform an explicit commit after call to flushVBucket

    Although a flushVBucket writes the item to the underlying store,
    the item is not marked clean until an explicit commit is called.
    If the underlying store is couchstore, a commit is called with
    a flushVBucket but in the case of forestdb, a commit is not
    always called, hence call an explicit commit.
    
    Change-Id: I577877d1665137717c0d7db1979d1bf1da5486e6
    Reviewed-on: http://review.couchbase.org/64861
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Sriram Ganesan committed Jun 9, 2016
Commits on Jun 8, 2016
  1. Increase max_size for test_dcp_cursor_dropping_backfill to 6 MiB

    With forestdb as a storage option, at least 4 MiB is already occupied
    on startup. So, increase max_size to 6 MiB at least.
    
    Change-Id: I2722e713657c9a70107fa5541643a80d7be47b4a
    Reviewed-on: http://review.couchbase.org/64797
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Sriram Ganesan committed Jun 7, 2016
  2. Don't retrieve values for KEYS_ONLY filter

    While scanning for keys from forestdb, only retrieve the keys and meta data
    for the KEYS_ONLY filter
    
    Change-Id: I7b71b86ec9498c3aee642cfaa4b98f4a78d464af
    Reviewed-on: http://review.couchbase.org/64794
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: buildbot <build@couchbase.com>
    Sriram Ganesan committed Jun 7, 2016
Commits on Jun 6, 2016
  1. @manudhundi

    Merge remote-tracking branch 'couchbase/watson'

    *
    |\
    | * 8500fdb 2016-06-03 | MB-19836:  Correctly set PendingOpsNotifcation task priority (gerrit/watson, couchbase/watson) [Daniel Owen]
    | * 8bea4e0 2016-06-02 | MB-18452: Single threaded test harness improvements [Jim Walker]
    | * da4a825 2016-06-03 | MB-9897: Handle slow stream by dropping cursor and switching to backfill state [Manu Dhundi]
    | * c72b7b0 2016-06-03 | MB-19813: [Refactor Test Code] Common func for min items sanity check [Manu Dhundi]
    | * 1f7b21a 2016-06-01 | MB-19813: Increase memory quota for a dgm test [Manu Dhundi]
    | *   5ac463e 2016-06-03 | Merge remote-tracking branch 'couchbase/4.5.0' into watson [Dave Rigby]
    | |\
    | | * 9d77ae2 2016-06-01 | MB-19815: doDcpVbTakeoverStats, addTakeoverStats: 0 deleted items on exception (gerrit/4.5.0, couchbase/4.5.0) [Manu Dhundi]
    | | * 7e6eb75 2016-05-20 | MB-19605: Add more tests for stats [Trond Norbye]
    | * | 7b002e2 2016-06-01 | MB-19807: EPStoreEvictionTest.GetKeyStatsEjected incorrectly compares strings as ptrs [Dave Rigby]
    | * | 1b4f629 2016-05-25 | MB-19428: Don't schedule backfills on dead vbuckets [Jim Walker]
    * | | 31d9b25 2016-06-03 | Address compiler warning by using temporary integer for snprintf (m/master, gerrit/master, couchbase/master) [Sriram Ganesan]
    
    Change-Id: I16e3fee0f4662533e6e3533d11beeaa76de13fbf
    manudhundi committed Jun 6, 2016
  2. @owendCB @daverigby

    MB-19836: Correctly set PendingOpsNotifcation task priority

    The PendingsOpsNotification task currently has the priority
    VBMemoryDeletionPriority (set to 6).  It should have
    the priority PendingOpsPriority (set to 0).
    
    Change-Id: I488d8eae7347eb65fe0f8474ae60e939234b972b
    Reviewed-on: http://review.couchbase.org/64714
    Well-Formed: buildbot <build@couchbase.com>
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Tested-by: buildbot <build@couchbase.com>
    owendCB committed with daverigby Jun 3, 2016
  3. @jimwwalker @daverigby

    MB-18452: Single threaded test harness improvements

    Refactor parts of the very new evp_store_single_threaded_test so that
    it's simpler to drive tasks making new tests easier to write.
    
    The main change is to provide helper methods for running any task from
    a queue (with some checks) and a way to push a clean shutdown.
    
    Change-Id: I19782dcacb36048151bc073a377f28603ff83033
    Reviewed-on: http://review.couchbase.org/64690
    Reviewed-by: Dave Rigby <daver@couchbase.com>
    Well-Formed: buildbot <build@couchbase.com>
    Tested-by: buildbot <build@couchbase.com>
    jimwwalker committed with daverigby Jun 2, 2016
Commits on Jun 3, 2016
  1. @manudhundi

    MB-9897: Handle slow stream by dropping cursor and switching to backf…

    …ill state
    
    There can be slow DCP streams which can hold cursors on the checkpoints
    causing the memory usage to shoot up. This can also result in deadlocks.
    
    Initially cursor dropping was implemented by closing and re-opening the
    slow streams. The re-opening of slow streams caused problems because
    ns-server also tried to re-open the closed streams.
    
    This approach tries to solve the problem by switching to
    backfilling state from in-memory state when we see the memory usage
    in checkpoints going high due to slow streams.
    
    This switch from in-memory to backfill state does not interfere with
    a snapshot that is being sent to the client. The change in state
    happens only after all the items in a current snapshot is sent.
    
    Hence clients work with the existing DCP protocol without any change.
    
    Change-Id: If4c128df60fc0249cadf08158a04911120de4c99
    Reviewed-on: http://review.couchbase.org/64375
    Well-Formed: buildbot <build@couchbase.com>
    Tested-by: buildbot <build@couchbase.com>
    Reviewed-by: Manu Dhundi <manu@couchbase.com>
    manudhundi committed Jun 3, 2016