Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-export Arrow and Parquet crates from DataFusion #39

Merged
merged 3 commits into from
Apr 24, 2021

Conversation

returnString
Copy link
Contributor

Which issue does this PR close?

Closes #36.

What changes are included in this PR?

  • The datafusion crate now re-exports its versions of arrow and parquet
  • The benchmarks and datafusion-examples crates have been updated to make use of these exports in lieu of declaring their own dependencies on the arrow-rs repo

Bonus subsection: not changed

  • The various Ballista crates have not been updated to make use of this at present as I'm not familiar enough with Ballista's needs and current status
  • datafusion-examples still manually depends on the arrow-rs repo for Flight support, as DataFusion doesn't use this currently

Are there any user-facing changes?

This only provides users with an extra option for referencing arrow and parquet, without breaking existing setups - we should probably add this to an installation/usage section of the user guide once that settles a bit though.

@codecov-commenter
Copy link

Codecov Report

Merging #39 (de2a6ef) into master (713ba10) will not change coverage.
The diff coverage is 0.00%.

❗ Current head de2a6ef differs from pull request most recent head 1ddbbf4. Consider uploading reports for the commit 1ddbbf4 to get more accurate results
Impacted file tree graph

@@           Coverage Diff           @@
##           master      #39   +/-   ##
=======================================
  Coverage   75.83%   75.83%           
=======================================
  Files         135      135           
  Lines       23152    23152           
=======================================
  Hits        17557    17557           
  Misses       5595     5595           
Impacted Files Coverage Δ
benchmarks/src/bin/nyctaxi.rs 0.00% <ø> (ø)
benchmarks/src/bin/tpch.rs 38.86% <0.00%> (ø)
datafusion-examples/examples/flight_server.rs 0.00% <0.00%> (ø)
datafusion-examples/examples/simple_udaf.rs 0.00% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 713ba10...1ddbbf4. Read the comment docs.

Copy link
Member

@jorgecarleitao jorgecarleitao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks a lot, @returnString !

Could you rebase / address conflicts?

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree -- super helpful @returnString . Thank you

@returnString
Copy link
Contributor Author

No problem at all :) Just rebased to fix up the conflicts with the benchmark refactoring.

@andygrove
Copy link
Member

I merged this into master locally and the tests pass so I will merge this. Thanks @returnString !

@andygrove andygrove merged commit 9ba214a into apache:master Apr 24, 2021
@houqp houqp added enhancement New feature or request datafusion Changes in the datafusion crate labels Jul 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datafusion Changes in the datafusion crate enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Publically export Arrow crate from datafusion
6 participants