Skip to content

Conversation

@Jefffrey
Copy link
Collaborator

@Jefffrey Jefffrey commented Jan 5, 2025

Can see the commits for details of the individual refactors which have been grouped into this PR.

Mainly focusing on refactoring stripe/column related code, to ideally aim for:

  • Unification of structs/enums with writer versions
  • Move away from exposing proto types in API (prefer to have our own types which then map to proto)

Getting unsigned/signed integer decoder shouldn't need knowledge of
a column; only cares about if you need V1 or V2, so refactor to
accomodate this. Also simplifies some methods by removing the check for
invalid column encoding as this shouldn't be checked at decoder
retrieval time anyway.
These are just indirections and serve no purpose as an abstraction,
as they don't actually rely on fields from Column.
@Jefffrey Jefffrey requested a review from Xuanwo January 5, 2025 03:49
Copy link
Collaborator

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @Jefffrey for working on this.

@WenyXu WenyXu merged commit d8595b6 into datafusion-contrib:main Aug 15, 2025
10 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants