Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Expose the ability to compile DataFusion with SIMD #2118

Closed
philippemnoel opened this issue Jan 25, 2024 · 1 comment
Closed

Expose the ability to compile DataFusion with SIMD #2118

philippemnoel opened this issue Jan 25, 2024 · 1 comment
Labels
binding/rust Issues for the Rust crate enhancement New feature or request

Comments

@philippemnoel
Copy link
Contributor

Description

DataFusion, in their README, mention that compiling with feature = simd (which requires Rust nightly) improves performance.

We're using DataFusion via delta-rs, and would like the ability to compile with SIMD.

Would it be possible to expose a feature in delta-rs that can be to enable the SIMD feature in DataFusion?

Use Case
Faster performance!

Related Issue(s)
N/A
@philippemnoel philippemnoel added the enhancement New feature or request label Jan 25, 2024
@ion-elgreco ion-elgreco added the binding/rust Issues for the Rust crate label Jan 25, 2024
@philippemnoel philippemnoel mentioned this issue Jan 25, 2024
Closed
@philippemnoel
Copy link
Contributor Author

It turns out Arrow-rs 50 brings auto-vectorizarion without the SIMD feature, so this is not needed. See #2121, which brings DataFusion 35 (which brings Arrow 50)

@philippemnoel philippemnoel closed this as not planned Won't fix, can't repro, duplicate, stale Jan 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
binding/rust Issues for the Rust crate enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ion-elgreco @philippemnoel