Skip to content

[Rust] [datafusion] Allow integration in non libc environments #102

@alamb

Description

@alamb

Note: migrated from original JIRA: https://issues.apache.org/jira/browse/ARROW-10053

DataFusion currently cannot be integrated with codebases which are built for platforms that don't provide a libc implementations. This is because some of the dependencies have features flags enabled which pull in libc linked dependencies.

Specifically the datafusion cargo.toml:

  • doesn't define clap as an optional dependency (even if cli feature is disable its pulled in)
  • the arrow crate has the prettyprint feature set
  • the parquet crate doesn't have default-features disabled which pull in a libc dependent crate

Ideally it would be possible to make these feature flags configurable or add a nonlibc feature which disables those. Just to five an idea here's the diff that I'm using right now to allow integration.
{code:java}
diff --git a/rust/datafusion/Cargo.toml b/rust/datafusion/Cargo.toml
index 71c16576f..e48bc3216 100644
--- a/rust/datafusion/Cargo.toml
+++ b/rust/datafusion/Cargo.toml
@@ -41,14 +41,14 @@ path = "src/bin/main.rs" [features]
default = ["cli"]
-cli = ["rustyline"]
+cli = ["rustyline", "clap"] [dependencies]
fnv = "1.0"
-arrow = { path = "../arrow", version = "2.0.0-SNAPSHOT", features = ["prettyprint"] }
-parquet = { path = "../parquet", version = "2.0.0-SNAPSHOT", features = ["arrow"] }
+arrow = { path = "../arrow", version = "2.0.0-SNAPSHOT"}
+parquet = { path = "../parquet", version = "2.0.0-SNAPSHOT", default-features = false, features = ["arrow"] }
sqlparser = "0.6.1"
-clap = "2.33"
+clap = { version = "2.33", optional = true }
rustyline = {version = "6.0", optional = true}
crossbeam = "0.7"
paste = "0.1"

{code}

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions