Make parquet support optional

[alamb]

Is your feature request related to a problem or challenge?

DataFusion aspires to be a modular query engine, and not all users need support for parquet

The parquet crate has a non trivial number of dependencies (some of which prevent compiling DataFusion to WASM -- see #7652)

Also there have been reports like #2042 where some of the native dependencies like zstd cause build issues

Describe the solution you'd like

I would like to make parquet support optional, the same way avro support is

Avro is marked as optional: https://github.com/apache/arrow-datafusion/blob/5f38135d5d21160d6b1ef7213578dd5eddfa4f95/datafusion/core/Cargo.toml#L53

It would be great to mark parquet as optional too
https://github.com/apache/arrow-datafusion/blob/5f38135d5d21160d6b1ef7213578dd5eddfa4f95/datafusion/core/Cargo.toml#L82

In order to make this work, we would likely need to encapsulate the parquet code in a more modular fashion (rather than sprinkling #[cfg(feature = parquet)] all over the code

Describe alternatives you've considered

No response

Additional context

No response

[alamb]

The same thing probably applies to CSV and JSON support

[alamb]

If someone wanted a fun software engineering challenge that is well specified, I think this one is pretty reasonable

It isn't a great "new to rust programming" project, but it is a good "new to datafusion" project for someone already familiar with rust

[universalmind303]

arrow-rs dependency may make this a bit tricky as they don't have a parquet feature flag.

see apache/arrow-rs#4764

[tustvold]

arrow-rs dependency may make this a bit tricky as they don't have a parquet feature flag.

Parquet depends on arrow-rs not the other way round?

[universalmind303]

arrow-rs dependency may make this a bit tricky as they don't have a parquet feature flag.

Parquet depends on arrow-rs not the other way round?

sorry I misunderstood the issue linked. Please disregard.

[fudini]

Maybe this issue needs reopening because it no longer compiles without default features with following error:

error[E0412]: cannot find type `ParquetSink` in this scope
   --> datafusion/proto/src/physical_plan/mod.rs:962:32
    |
962 |                 let data_sink: ParquetSink = sink
    |                                ^^^^^^^^^^^ not found in this scope
    |
help: consider importing one of these items
    |
18  + use crate::protobuf::ParquetSink;

[alamb]

Maybe this issue needs reopening because it no longer compiles without default features with following error:

@fudini -- thanks for the report. Can you provide a command to reproduce this error?

[fudini]

Maybe this issue needs reopening because it no longer compiles without default features with following error:

@fudini -- thanks for the report. Can you provide a command to reproduce this error?

Sure, it's:
cargo build --no-default-features

[alamb]

argo build --no-default-features

Thank you -- filed #8844