Skip to content

Preserve field metadata when coercing INT96 timestamps - #24790

Open
ryzhyk wants to merge 1 commit into
apache:mainfrom
ryzhyk:int96-preserve-field-metadata
Open

Preserve field metadata when coercing INT96 timestamps#24790
ryzhyk wants to merge 1 commit into
apache:mainfrom
ryzhyk:int96-preserve-field-metadata

Conversation

@ryzhyk

@ryzhyk ryzhyk commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

Int96Coercer rebuilds every struct, list and map field with Field::new_struct, Field::new_list and Field::new, which start from empty metadata. Leaf fields go through field_with_new_type, which clones the field, so only container fields are affected, and only when the file holds an INT96 column: the same file read without coerce_int96 keeps its metadata.

What changes are included in this PR?

Carry current_field.metadata() across in the three constructors.

Are these changes tested?

yes

Are there any user-facing changes?

Fixes apache#24786

`Int96Coercer` rebuilds every struct, list and map field with
`Field::new_struct`, `Field::new_list` and `Field::new`, which start from empty
metadata. Leaf fields go through `field_with_new_type`, which clones the field,
so only container fields are affected, and only when the file holds an INT96
column: the same file read without `coerce_int96` keeps its metadata.

Carry `current_field.metadata()` across in the three constructors.

Signed-off-by: Leonid Ryzhyk <ryzhyk@gmail.com>
@github-actions github-actions Bot added the datasource Changes to the datasource crate label Aug 30, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.96970% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 81.53%. Comparing base (4ada0dc) to head (8bb2406).

Files with missing lines Patch % Lines
...tafusion/datasource-parquet/src/schema_coercion.rs 96.96% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #24790      +/-   ##
==========================================
- Coverage   81.53%   81.53%   -0.01%     
==========================================
  Files        1123     1123              
  Lines      406041   406074      +33     
  Branches   406041   406074      +33     
==========================================
+ Hits       331049   331073      +24     
- Misses      55631    55636       +5     
- Partials    19361    19365       +4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

datasource Changes to the datasource crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Int96Coercer drops the metadata of struct, list and map fields.

2 participants