[ISSUE-322] Fix archive consume log exception #323
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 9 commits
December 12, 2020 11:55
archive issue: consume log exception jd-opensource#322 归档功能异常：消费归档功能里，在批量进行读取消费记录日志文件的时候，如果读取是在两个文件里进行的(比如批量1000条，500条读取完后再进行下一个文件的500条读取)，操作hbase存储如果出现异常的话进行位置回退处理，此时的回退处理是对1000条整体位置的偏移量进行回退，但偏移量横跨两个文件进行读取计算的，所以会导致回退偏移量设置异常，从而影响整个消费记录日志的存储线程。
llIlll
changed the title
added 10 commits
December 23, 2020 12:22
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
archive issue: consume log exception #322
归档功能异常：消费归档功能里，在批量进行读取消费记录日志文件的时候，如果读取是在两个文件里进行的(比如批量1000条，500条读取完后再进行下一个文件的500条读取)，操作hbase存储如果出现异常的话进行位置回退处理，此时的回退处理是对1000条整体位置的偏移量进行回退，但偏移量横跨两个文件进行读取计算的，所以会导致回退偏移量设置异常，从而影响整个消费记录日志的存储线程。