Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(parquet): support for reading structs nested within lists #1187

Merged
merged 3 commits into from
Jan 18, 2022

Conversation

helgikrs
Copy link
Contributor

@helgikrs helgikrs commented Jan 15, 2022

Which issue does this PR close?

Closes #1186.

What changes are included in this PR?

Adds support reading lists of structs in the parquet reader. Adds a test
roundtrip writing and reading structs in a list.

Are there any user-facing changes?

No.

@@ -1770,4 +1770,104 @@ mod tests {
let stats = column.statistics().unwrap();
assert_eq!(stats.null_count(), 2);
}

#[test]
fn test_list_of_struct_roundtrip() {
Copy link
Contributor Author

@helgikrs helgikrs Jan 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: This test will fail until #1185 is merged, as the data written does not match the original.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1185 has now been merged 🚀

@github-actions github-actions bot added the parquet Changes to the parquet crate label Jan 15, 2022
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me -- thank you @helgikrs

@alamb
Copy link
Contributor

alamb commented Jan 17, 2022

I merged apache/master to this branch to pick up the changes in #1185 -- hopefully we'll get a clean CI run and we'll merge this in

@alamb alamb merged commit 4f1064e into apache:master Jan 18, 2022
@alamb
Copy link
Contributor

alamb commented Jan 18, 2022

Thanks again @helgikrs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
parquet Changes to the parquet crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Parquet reader should be able to read structs within list
2 participants