Fix file_offset_of_buffer_end <= getFileSize() assertion in Log/StripeLog on S3#100763
Merged
alexey-milovidov merged 1 commit intomasterfrom Mar 28, 2026
Merged
Conversation
…StripeLog` on S3 The `StripeLogSource` and `LogSource` release the read lock before lazily opening the data file. On S3 object storage, the `AsynchronousBoundedReadBuffer` caches file size from metadata at open time, which may differ from the snapshotted size the index was built against if concurrent writes modified the file metadata between lock release and file open. Fix by calling `setReadUntilPosition(file_size)` after opening the file, bounding reads to the snapshotted file size. This is a no-op on local disk and only takes effect on object storage where `AsynchronousBoundedReadBuffer` is used. Fixes #82555 https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=99537&sha=7338429ff8d3349ba147f9ba05558bbd6b84e4b2&name_0=PR&name_1=BuzzHouse%20%28amd_msan%29 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
|
Workflow [PR], commit [8add632] Summary: ❌
AI ReviewSummaryThis PR addresses an S3/object-storage read bound mismatch in ClickHouse Rules
Final Verdict
|
clickhouse-gh
Bot
added
the
pr-bugfix
Contributor
LLVM Coverage Report
Changed lines: 100.00% (11/11) · Uncovered code |
kssenii
approved these changes
Mar 27, 2026
robot-clickhouse-ci-2
added
the
pr-synced-to-cloud
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
StripeLogSourceandLogSourcerelease the read lock before lazily opening the data file. On S3 object storage, theAsynchronousBoundedReadBuffercaches file size from metadata at open time, which may differ from the snapshotted size the index was built against if concurrent writes modified the file metadata between lock release and file open.Fix by calling
setReadUntilPosition(file_size)after opening the file, bounding reads to the snapshotted file size. This is a no-op on local disk and only takes effect on object storage whereAsynchronousBoundedReadBufferis used.Found by BuzzHouse (amd_msan): https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=99537&sha=7338429ff8d3349ba147f9ba05558bbd6b84e4b2&name_0=PR&name_1=BuzzHouse%20%28amd_msan%29
#99537
Fixes #82555
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):
Fix
file_offset_of_buffer_end <= getFileSize()assertion failure (exception in debug builds) when reading fromLogorStripeLogtables on S3 object storage with concurrent writes.Documentation entry for user-facing changes