Skip to content

Make arrow-csv and arrow-json optional for applicable crates #17791

@crepererum

Description

@crepererum

Is your feature request related to a problem or challenge?

Currently importing crates like datafusion-expr or datafusion-common pulls in arrow with its default features, which includes the rather heavy arrow parts arrow-csv and arrow-json. Since these parts are really only required during physical execution of the respective operators & data sources, that seems rather unnessary.

Describe the solution you'd like

I see two options:

  • set default-features = false for the workspace-level arrow dependency and then request the right features in the Cargo.toml files of the respective datafusion crates
  • better: do NOT depend on the arrow umbrella/meta crate at all but directly on the subcrates that we need

Describe alternatives you've considered

Keeping the dependencies simple but instead suffer from longer compile times and larger build artifacts.

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions