New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: change default value of use_parquet2 to 0. #13754
Conversation
I downloaded the parquet file and located the problem: |
Let's draft this PR. |
But In better hardware, it may have better performance. And �I think correctness is high priority than performance, parquet2 lacked much features or bugs which were hard to fix. Such as :
which is found by @everpcpc |
I agree with this point. In addition, with the further maturity of the Arrow Rust ecosystem, it can be expected that the performance of Parquet will improve, while the maintainability of Parquet2 is almost no longer guaranteed. |
@sundy-li agreed, I understand your situation. let's continue this PR and keep the use_parquet setting. |
unloading is using parquet2, |
I'll follow up to fix this. |
Docker Image for PR
|
I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/
Summary
we do not hope user to use this option unless nessary.
now only affect copy from parquet and select parquet.
later when other component mig to parquet rs, may resue this setting with diff meaning, or use a more specific setting name?
if any one feel there are some risk to use parquet_rs as default?we can stop and test and review more.
@b41sh @sundy-li about the data type
@RinChanNOWWW @Dousir9 about the read progress
Result::unwrap()
on anErr
value: Overflow #13753This change is