Skip to content

Conversation

@andygrove
Copy link
Member

@andygrove andygrove commented Dec 17, 2023

Which issue does this PR close?

N/A

Rationale for this change

Prepare for next release

What changes are included in this PR?

  • Upgrade to DataFusion 34
  • Update changelog

Are there any user-facing changes?

@andygrove andygrove changed the title WIP: Upgrade to DataFusion 34 Upgrade to DataFusion 34 Dec 27, 2023
@andygrove andygrove marked this pull request as ready for review December 27, 2023 20:02
@andygrove
Copy link
Member Author

@jdye64 fyi

Copy link
Contributor

@jdye64 jdye64 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andygrove andygrove requested a review from viirya December 28, 2023 16:19
@andygrove andygrove changed the title Upgrade to DataFusion 34 Prepare 34.0.0 Release Dec 28, 2023
#[pyclass(name = "DataFrame", module = "datafusion", subclass)]
#[derive(Clone)]
pub(crate) struct PyDataFrame {
pub struct PyDataFrame {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also due to 34.0.0?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think directly. I will say having lots of these "(crate)" level visibility of structs makes using ADP as a library difficult since they are not visible outside the crate. I had started removing them awhile back and think Andy was just removing another one of them. I should probably just make a PR to remove them all, where it makes sense of course

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the (crate) restriction to work around many warnings like the following. I wonder if this was caused by a recent Rust version upgrade rather than anything related to 34.0.0.

warning: type `PyDataFrame` is more private than the item `PySessionContext::sql`
   --> src/context.rs:282:5
    |
282 |     pub fn sql(&mut self, query: &str, py: Python) -> PyResult<PyDataFrame> {
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ method `PySessionContext::sql` is reachable at visibility `pub`
    |
note: but type `PyDataFrame` is only usable at visibility `pub(crate)`
   --> src/dataframe.rs:39:1
    |
39  | pub(crate) struct PyDataFrame {
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    = note: `#[warn(private_interfaces)]` on by default

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a reference to private_interfaces in the release notes for Rust 1.74.0, which was released last month.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Thanks for the reference.

@andygrove andygrove merged commit 2889de0 into apache:main Dec 28, 2023
@andygrove andygrove deleted the df-34 branch December 28, 2023 20:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants