Fix ConcurrentModificationException in IncrementalPublishingKafkaIndexTaskRunner #5907
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gianm
reviewed
Jun 28, 2018
| @@ -1504,7 +1505,7 @@ public DateTime getStartTime(@Context final HttpServletRequest req) | |||
| private final int sequenceId; | |||
| private final String sequenceName; | |||
| private final Map<Integer, Long> startOffsets; | |||
| private final Map<Integer, Long> endOffsets; | |||
| private final ConcurrentHashMap<Integer, Long> endOffsets; | |||
There was a problem hiding this comment.
Please include a comment that explains two things:
- why this is a
ConcurrentHashMap - who is allowed to modify it (do you need to be a specific thread or acquire a specific lock)? I bet that if multiple threads modify this concurrently then unexpected things will happen.
There was a problem hiding this comment.
Well, the problem is that setEndOffsets() can be called by both the main thread and the http thread. Actually setEndOffsets() also sets checkpointed variable too, so I changed to acquire a lock before modifying endOffsets and checkpointed variables.
I bet that if multiple threads modify this concurrently then unexpected things will happen.
I have no idea about this.
gianm
approved these changes
Jul 1, 2018
jihoonson
added a commit
to implydata/druid-public
that referenced
this pull request
Jul 3, 2018
…xTaskRunner (apache#5907) * Fix ConcurrentModificationException in IncrementalPublishingKafkaIndexTaskRunner * fix lock and add comments
gianm
added a commit
to implydata/druid-public
that referenced
this pull request
Jul 5, 2018
…xTaskRunner (apache#5907) Ported without cherry-pick, since the original commit depends on the patch that splits KafkaIndexTask.
jihoonson
pushed a commit
to jihoonson/druid
that referenced
this pull request
Jul 7, 2018
…xTaskRunner (apache#5907) Ported without cherry-pick, since the original commit depends on the patch that splits KafkaIndexTask.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #5745.
This change is