-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Right now, ListingOptions is a public struct with some required fields and some that have reasonable default values
This means that to create one you need to provide all fields, which means that adding new fields requires changes to many places in DataFusion as well as downstream creates -- most recently #4177
For example, here is how you have to create one today:
ListingOptions {
format: Arc::new(file_format),
collect_stat: false,
file_extension: self.file_extension.to_owned(),
target_partitions,
table_partition_cols: self.table_partition_cols.clone(),
file_sort_order: None,
}
}Describe the solution you'd like
What I would like is a builder style API for this struct such that new() takes the required arguments and there are with_ functions to set them
ListingOptions::new(file_format)
.with_collect_statistics(false)
.with_file_extension(self.file_extension.to_owned()),
.with_target_partitions(target_partitions)
.with_table_table_partition_cols(self.table_partition_cols.clone())Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Flagging this as good first issue as it is a software engineering effort that can be mostly guided by the compiler and would be a good way to get familiar with the codebase
cc @tustvold