Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

420 polars avro reader writer #447

Merged
merged 4 commits into from Oct 9, 2023
Merged

420 polars avro reader writer #447

merged 4 commits into from Oct 9, 2023

Conversation

ghost
Copy link

@ghost ghost commented Oct 8, 2023

Polars Avro Data Saver /Loader.

Implement https://pola-rs.github.io/polars/py-polars/html/reference/io.html#avro as a data saver/loader

Changes

  • Implement the reader and writer for Avro
  • Add unit tests
  • Add the materialize example

How I tested this

  • Added the unit tests for the implemented functionality

Checklist

  • PR has an informative and human-readable title (this will be pulled into the release notes)
  • Changes are limited to a single goal (no scope creep)
  • Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future TODOs are captured in comments
  • Project documentation has been updated if adding/changing functionality.

@skrawcz
Copy link
Collaborator

skrawcz commented Oct 8, 2023

@kokobhara you'll need to rebase this pull request.

IIRC the steps to this are:

  1. Pull the latest to your fork.
  2. Rebase this branch against your fork's main.
  3. Fix any conflicts & verify that things still work.
  4. Force push it.

Otherwise looks good, thanks!

@skrawcz skrawcz merged commit 08c216c into DAGWorks-Inc:main Oct 9, 2023
21 checks passed
@skrawcz skrawcz mentioned this pull request Oct 9, 2023
@ghost ghost deleted the 420-polars-avro-reader-writer branch October 9, 2023 03:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant