Refactor aloc_sink #3058
Merged
Refactor aloc_sink #3058
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Parameterize using_2pc * WIPs * WIP cleanups ... self._txn_state mgmt is messed up * Works for './master-crasher.sh 2' and './master-crasher.sh 2 run_custom_tcp_crash0' * Add TODO item * Fix txnlog bug copy-and-paste-o
* WIP: change offset bookkeeping, broken! * WIP: change offset bookkeeping, works!
* WIP: add TwoPC_TxnlogHelper * Split aloc_sink and new aloc_sink_impl.py * Adjust when a TwoPC_Output class is created
slfritchie
force-pushed
the
refactor-aloc-sink
branch
from
672289d
to
911516a
Compare
JONBRWN
approved these changes
Nov 18, 2019
There was a problem hiding this comment.
@slfritchie this looks good to me based on my limited exposure to this side of the ALOC work. I think this is a good start and would be fine with this getting merged.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Split the
aloc_sinkcode into pieces that implement sink protocol & internal logic separately from what's necessary to manage the sink's persistent state. Code that implements the former is not refactored and thus remains ugly. However, it's our sink protocol reference implementation, and it works. If/when we want to refactor it, we've got a solid baseline.Reviewers should focus on the latter: managing the sink's persistent state. This refactoring is in advance of anticipated work that will send
aloc_sinkoutput to Kafka instead of the local file system. The API for theTwoPC_Outputbase class may not be perfect for the Kafka case, but @JONBRWN and I believe it's probably good enough for now. Suggestions for better Python style are appreciated.