handoff batching #316

Merged
merged 22 commits into from May 22, 2013

Conversation

Projects
None yet
2 participants
Contributor

chardan commented May 9, 2013

Allows groups of objects to be sent during handoff.
Adds handoff_perftool.erl, useful for benchmarking.

@evanmcc evanmcc and 1 other commented on an outdated diff May 14, 2013

src/riak_core_handoff_sender.erl
%% IFF the vnode is using an async worker to perform the fold
%% then sync_command will return error on vnode crash,
%% otherwise it will wait forever but vnode crash will be
%% caught by handoff manager. I know, this is confusing, a
%% new handoff system will be written soon enough.
- R = riak_core_vnode_master:sync_command({SrcPartition, SrcNode},
- Req,
- VMaster, infinity),
- if R == {error, vnode_shutdown} ->
+
+ AccRecord0 = riak_core_vnode_master:sync_command({SrcPartition, SrcNode},
+ Req,
+ VMaster, infinity),
+
+ %% Send any straggler entries remaining in the buffer:
+ AccRecord = send_objects(AccRecord0#ho_acc.item_queue, AccRecord0),
+
@evanmcc

evanmcc May 14, 2013

Contributor

alignment error

@chardan

chardan May 15, 2013

Contributor

Corrected, thank you!

evanmcc was assigned May 14, 2013

Contributor

evanmcc commented May 21, 2013

+1, code looks good, nice to have another tunable parameter. Performance testing on our pathological cases shows only minor improvements, but for the most part we're hitting other bottlenecks there.

chardan merged commit aa98167 into master May 22, 2013

1 check failed

default The Travis CI build failed
Details

seancribbs deleted the jfw-handoff_batching branch Apr 1, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment