New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SAMZA-2493: Keep checkpoint manager consumer open for repeated polling #1327
Conversation
@bharathkk @xinyuiscool can you take a look over this? |
samza-core/src/main/java/org/apache/samza/config/TaskConfig.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor comment around the configuration. Since its an internal config, there isn't going to be configuration document and it will be useful to have a sentence or two describing its purpose.
Looks good otherwise.
samza-kafka/src/main/scala/org/apache/samza/checkpoint/kafka/KafkaCheckpointManager.scala
Show resolved
Hide resolved
Do you plan to add tests? I noticed TBD in the description. |
@mynameborat yeah, added them |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for adding the tests.
Will merge it in once travis checks pass.
Feature: Allow control of whether the underlying consumer backing a checkpoint manager will be left open (until stopped), or closed after the initial read of checkpoints
Changes: Introduces an internal config, injected by SamzaContainer, that is injected dependent on whether the container is standby or not, to control whether the container's checkpoint manager should have it's underlying consumer stopped after initial read or not.
Tests: Added a unit test each for the config set to true / false
API Changes: None
Upgrade Instructions: None
Usage Instructions: None