Expose Backend Size to Handoff for Progress Tracking #526

Merged
merged 5 commits into from Apr 25, 2013

Projects

None yet

4 participants

@jrwest
Contributor
jrwest commented Apr 1, 2013

This PR exposes the necessary information for tracking handoff progress (added in basho/riak_core#290) of kv_vnodes using the memory, level, or bitcask backends. Each backend exposes an upper-bound on the amount of data stored, but each one comes with some caveats. Although these caveats are inconvenient, the metrics still provide a significantly better indication of progress than having no indication. The API has been designed so more accurate (or less accurate even) metrics can be used, in the future, instead.

The bitcask backend uses the existing knowledge of keydir size to expose the total number of keys that will be transferred. This is an ok but not great indicator of progress because it doesn't account for varying object sizes (transferring 99 1k objects and one 20MB object).

The eleveldb backend uses the support added in basho/leveldb#74 to directly query the number of bytes in sst files. This has the downside that the returned number may include overwritten keys whose bytes will not be transferred. However, this still means that we will always have made equal or more progress than what is indicated, never less, which is sufficient. If basho/leveldb#74 is not used, tracking handoff progress in eleveldb is not supported.

The memory backend uses built in stats info in ets to return the number of keys that will be transferred. This also has the same drawback as the bitcask backend. In addtion, since the ets fold is not a snapshot, the backend uses the dynamic size feature to make sure progress is tracked correctly.

The multi backend is not supported.

Example output of riak-admin transfers with these changes: https://gist.github.com/jrwest/588d85bbeaebaf940e59

This PR also completes the move of transfers/1 to riak_core_console started in basho/riak_core#290

Associated riak PR: basho/riak#303

@jrwest jrwest referenced this pull request in basho/riak Apr 1, 2013
Merged

update riak-admin transfers #303

@jrwest jrwest was assigned Apr 1, 2013
@engelsanchez engelsanchez was assigned Apr 12, 2013
@engelsanchez
Contributor

Blergh, I didn't realize that this has now changed to depend on unmerged leveldb code before I started re-reviewing and testing. Please let me know when the leveldb code has finished review and has been merged to complete this review. @jtuple @jrwest

@jrwest
Contributor
jrwest commented Apr 24, 2013

@engelsanchez this branch should be able to merge independently of the the leveldb changes. As for review, my personal suggestion would be to review this w/ the leveldb backend w/ and w/o @jtuple's changes.

The leveldb PR for reference: basho/leveldb#74

@engelsanchez
Contributor

Is the leveldb branch really ready? I keep trying to compile this with the branch and I am not getting progress reports when using leveldb like I used to when I tested this last week. It might be that my build is not clean, but confirmation would be nice @jtuple @jrwest

@engelsanchez
Contributor

Nevermind @jtuple @jrwest, test is working after manually removing a bunch of C build artifacts. I will complete the review shortly.

@jrwest
Contributor
jrwest commented Apr 24, 2013

The leveldb branch was ready and working last I tested. I think I saw some changes related to how leveldb is built recently but iirc it took me a while to figure out how to get the correct version of leveldb checked out to test it (you have to edit a sha in eleveldb Makefile)

@engelsanchez
Contributor

Ok, I've reviewed the code and tested the memory, bitcask and leveldb backends. Leveldb works as advertised when the data size branch mentioned above is used, and simply displays N/A when not. I've found no issues here, so 👍 💃 ⛵️

@jrwest jrwest merged commit 81ced6a into master Apr 25, 2013

1 check failed

default The Travis build failed
Details
@seancribbs seancribbs deleted the jrw-handoff-progress branch Apr 1, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment