Skip to content

fix: add nullability to struct type String()#700

Merged
zeroshade merged 1 commit intoapache:mainfrom
Willem-J-an:main
Mar 10, 2026
Merged

fix: add nullability to struct type String()#700
zeroshade merged 1 commit intoapache:mainfrom
Willem-J-an:main

Conversation

@Willem-J-an
Copy link
Contributor

@Willem-J-an Willem-J-an commented Mar 10, 2026

Rationale for this change

We frequently run into panic:
panic: arrow/array: column "colName" type mismatch, got=.., want=...
The got and want will be exactly the same schema, the only difference is in the nullability of certain struct fields. The only way to figure out which fields are the problem is to go through the debugger.

The nullability is rendered for other types like map and list, so it seems like it should be there for struct fields as well.

What changes are included in this PR?

  • Add nullable in String() representation of StructType fields
  • Corrected tests that were affected by this
    • Some were just adding nullable where it's now added to the string and it previously was not
    • Some appear to be actual fixes to the tests, where the schema's did not match, but that was not detected in the schema string comparison because nullability was not included for StructType.

Are these changes tested?

ran make test, didn't have the parquet test files, so didn't run those.

Are there any user-facing changes?

Yes, the String() representation for data types is user facing.

@Willem-J-an Willem-J-an requested a review from zeroshade as a code owner March 10, 2026 17:02
Copy link
Member

@zeroshade zeroshade left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this! It looks good to me

@zeroshade zeroshade merged commit 1a28af3 into apache:main Mar 10, 2026
69 of 77 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants