Skip to content

minor: add fileGroup to V10 segment SegmentFileContainerMetadata, switch default metadata compression to zstd#19468

Merged
clintropolis merged 5 commits into
apache:masterfrom
clintropolis:v10-segment-add-filegroup-to-containers
May 18, 2026
Merged

minor: add fileGroup to V10 segment SegmentFileContainerMetadata, switch default metadata compression to zstd#19468
clintropolis merged 5 commits into
apache:masterfrom
clintropolis:v10-segment-add-filegroup-to-containers

Conversation

@clintropolis
Copy link
Copy Markdown
Member

@clintropolis clintropolis commented May 17, 2026

Description

Adds a fileGroup field to SegmentFileContainerMetadata of V10 segments, which after #19375 associated containers with startFileGroup to place all internal files under the same group in the same container. This just adds that field to the container so on the read side we have an easy way to get a list of the internal file bundles present in a v10 segment, making some things easier for partial download. The segment file builder now also enforces that columns and files added during a file group are prefixed with the file group name.

While I was here, I also set the default metadata compression for v10 segments to use zstd instead of uncompressed. I can't really think of much downside to doing this, I just hadn't got around to switching it yet, so did it now since I had to update the expected size of a test anyway.

Copy link
Copy Markdown
Member

@FrankChen021 FrankChen021 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have reviewed the code for correctness, edge cases, concurrency, and integration risks; no issues found.

Reviewed 7 of 7 changed files.


This is an automated review by Codex GPT-5.5

@clintropolis clintropolis merged commit f0f69ff into apache:master May 18, 2026
88 of 90 checks passed
@clintropolis clintropolis deleted the v10-segment-add-filegroup-to-containers branch May 18, 2026 16:56
@github-actions github-actions Bot added this to the 38.0.0 milestone May 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants