Skip to content

[core] Fix shared decompression buffer in stable manifest blocks - #9430

Merged
yuzelin merged 1 commit into
apache:masterfrom
yuzelin:fix-manifest-stable-block-buffer
Aug 27, 2026
Merged

[core] Fix shared decompression buffer in stable manifest blocks#9430
yuzelin merged 1 commit into
apache:masterfrom
yuzelin:fix-manifest-stable-block-buffer

Conversation

@yuzelin

@yuzelin yuzelin commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Purpose

ManifestAvroReader.RawBlock.stableCopy() copied the compressed Avro block but retained the original mutable DecoderContext. With ZSTD buffer reuse, decoding a stable previous block could overwrite the cached decompressed bytes of the current block. Reading the current block again could then fail with EOF/trailing-byte errors or silently decode incorrect content.

This change gives every stable block an independent decoder context and decompression buffer.

Tests

  • testFullCompactionWithStableLegacyManifestBlocks

@yuzelin
yuzelin force-pushed the fix-manifest-stable-block-buffer branch from b62274b to 4c12aa8 Compare August 27, 2026 12:45

@JingsongLi JingsongLi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@yuzelin
yuzelin merged commit 55fe85f into apache:master Aug 27, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants