[Not for merge] Make ConvertedType optional in the Parquet schema#9980
[Not for merge] Make ConvertedType optional in the Parquet schema#9980etseidl wants to merge 10 commits into
ConvertedType optional in the Parquet schema#9980Conversation
|
run benchmark metadata, parquet_round_trip |
|
Hi @etseidl, thanks for the request (#9980 (comment)). Supported benchmarks:
Usage: Per-side configuration ( env:
SHARED_SETTING: enabled
baseline:
ref: v45.0.0
env:
DATAFUSION_RUNTIME_MEMORY_LIMIT: 1G
changed:
ref: v46.0.0
env:
DATAFUSION_RUNTIME_MEMORY_LIMIT: 2GFile an issue against this benchmark runner |
|
run benchmarks metadata parquet_round_trip |
|
🤖 Arrow criterion benchmark running (GKE) | trigger CPU Details (lscpu)Comparing optional_conv_type (efb7f60) to fd1c5b3 (merge-base) diff File an issue against this benchmark runner |
|
🤖 Arrow criterion benchmark running (GKE) | trigger CPU Details (lscpu)Comparing optional_conv_type (efb7f60) to fd1c5b3 (merge-base) diff File an issue against this benchmark runner |
|
🤖 Arrow criterion benchmark completed (GKE) | trigger Instance: CPU Details (lscpu)Details
Resource Usagebase (merge-base)
branch
File an issue against this benchmark runner |
|
🤖 Arrow criterion benchmark completed (GKE) | trigger Instance: CPU Details (lscpu)Details
Resource Usagebase (merge-base)
branch
File an issue against this benchmark runner |
|
run benchmark parquet_round_trip env:
BENCH_FILTER: write StringView |
|
🤖 Arrow criterion benchmark running (GKE) | trigger CPU Details (lscpu)Comparing optional_conv_type (1517231) to fd1c5b3 (merge-base) diff File an issue against this benchmark runner |
|
🤖 Arrow criterion benchmark completed (GKE) | trigger Instance: CPU Details (lscpu)Details
Resource Usagebase (merge-base)
branch
File an issue against this benchmark runner |
|
run benchmarks arrow_reader arrow_writer |
|
🤖 Arrow criterion benchmark running (GKE) | trigger CPU Details (lscpu)Comparing optional_conv_type (70752e5) to fd1c5b3 (merge-base) diff File an issue against this benchmark runner |
|
🤖 Arrow criterion benchmark running (GKE) | trigger CPU Details (lscpu)Comparing optional_conv_type (70752e5) to fd1c5b3 (merge-base) diff File an issue against this benchmark runner |
|
🤖 Arrow criterion benchmark completed (GKE) | trigger Instance: CPU Details (lscpu)Details
Resource Usagebase (merge-base)
branch
File an issue against this benchmark runner |
|
🤖 Arrow criterion benchmark completed (GKE) | trigger Instance: CPU Details (lscpu)Details
Resource Usagebase (merge-base)
branch
File an issue against this benchmark runner |
|
Curiosity satisfied. There doesn't appear to be any performance impact, but there's also no compelling correctness nor ergonomic argument to be made for this breaking change. |
Which issue does this PR close?
Rationale for this change
Test of making converted type an optional field (as it is in the Parquet spec).
What changes are included in this PR?
Convert
converted_typein theBasicTypeInfoto anOption<ConvertedType>, remove theConvertedType::NONEvariant (which is not in the spec).Are these changes tested?
Are there any user-facing changes?