Test for reading read statistics from parquet files without statistics and boolean & struct data type #10608
Conversation
alamb
changed the title
| ) | ||
| } | ||
|
|
||
| pub fn parquet_file_one_column_stats( |
There was a problem hiding this comment.
Can we please add some doc comments about what this function does / how it is different? It is clear from this PR that it adds enable_stats but that might not be obvious in the future
While we are on this topic, I personally find a struct with named parameters easier to read than something the following where you have to look up parquet_file_one_column_stats to know what the 0, 4 and 7 mean.
let reader =
parquet_file_one_column_stats(0, 4, 7, row_per_group, EnabledStatistics::None);I wonder if we could change the code to something like this to make it easier to read 🤔
let reader = TestFile {
num_null: 0
no_null_values_start: 4,
no_null_values_end: 7,
row_per_group,
enable_stats: EnabledStatistic::None,
}
.build();There was a problem hiding this comment.
If we want to do this, we need to think about parquet_file_many_columns, too.
I have to sign off now. How about we do this in following PR?
|
This PR also appears to have a conflict now |
NGA-TRAN
changed the title
|
Since this was test only code I just merged it in to keep things moving |
…s and boolean & struct data type (apache#10608) * read statistics from parquet without statistics * tests for boolean and struct array * link to a the struct array bug * Make builder style API * Port to builder style * simlify * Add column name to test --------- Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Which issue does this PR close?
More tests for #10453
Rationale for this change
What changes are included in this PR?
Are these changes tested?
Are there any user-facing changes?