Permalink
Browse files

MB-6796 Prioritize flushing pending vbuckets over regular vbuckets

This change supports the prioritization of pending vbuckets
(i.e., vbuckets whose active ownerships are currently being
transfered) over regular vbuckets on each destination node.

The flusher basically maintains the list of high priority
vbuckets ,which are explicitly requested by the cluster manager,
and interleaves persisting regular vbuckets with those high
priority vbuckets.

A command CMD_CHECKPOINT_PERSISTENCE (0xb1) is newly introduced
and uses a default binary protocol format whose message body
contains a checkpoint id (8 bytes) requested for persistence.

When the engine receives the above command from the cluster
manager, it registers the requested vbucket and its checkpoint
id into the flusher, so that the flusher can prioritize flusing
that vbucket over other vbuckets. When the requested checkpoint
is persisted, the flusher wakes up the blocked memcached
connection for the cluster manager and removes the requested
vbucket from its high priority vbucket list.

The timeout to a persistence request is set to 10 seconds
by default. If the timeout happens, the engine sends an TMP_FAIL
response to the cluster manger and removes the requested vbucket
from the high priority vbucket list.

Change-Id: I6173a25b81e146f1cb0643ac8f4c3799e086a718
Reviewed-on: http://review.couchbase.org/21523
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Jin Lim <jin@couchbase.com>
  • Loading branch information...
1 parent 4f9646c commit e9ce877041d101efee775e9c65ea6b9eef914926 @chiyoung chiyoung committed with Farshid Ghods Oct 11, 2012
Showing with 445 additions and 163 deletions.
  1. +2 −0 docs/stats.org
  2. +5 −0 include/ep-engine/command_ids.h
  3. +231 −108 src/ep.cc
  4. +25 −19 src/ep.hh
  5. +34 −0 src/ep_engine.cc
  6. +45 −23 src/flusher.cc
  7. +25 −10 src/flusher.hh
  8. +4 −0 src/stats.hh
  9. +5 −1 src/vbucket.hh
  10. +14 −1 src/vbucketmap.cc
  11. +5 −1 src/vbucketmap.hh
  12. +11 −0 tests/ep_test_apis.cc
  13. +2 −0 tests/ep_test_apis.h
  14. +37 −0 tests/ep_testsuite.cc
View
@@ -449,6 +449,7 @@ form to describe when time was spent doing various things:
| get_vb_cmd | servicing vbucket status requests |
| set_vb_cmd | servicing vbucket set state commands |
| del_vb_cmd | servicing vbucket deletion commands |
+| chk_persistence_cmd | waiting for checkpoint persistence |
| tap_vb_set | servicing tap vbucket set state commands |
| tap_vb_reset | servicing tap vbucket reset commands |
| tap_mutation | servicing tap mutations |
@@ -689,6 +690,7 @@ Reset Histograms:
| bg_wait |
| bg_tap_load |
| bg_tap_wait |
+| chk_persistence_cmd |
| data_age |
| del_vb_cmd |
| disk_insert |
@@ -138,6 +138,11 @@
*/
#define CMD_CHANGE_VB_FILTER 0xb0
+/**
+ * Command to wait for the checkpoint persistence
+ */
+#define CMD_CHECKPOINT_PERSISTENCE 0xb1
+
/**
* TAP OPAQUE command list
Oops, something went wrong.

0 comments on commit e9ce877

Please sign in to comment.