Skip to content

Improve ergonomics of creating ListingOptions #4178

@alamb

Description

@alamb

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

Right now, ListingOptions is a public struct with some required fields and some that have reasonable default values

This means that to create one you need to provide all fields, which means that adding new fields requires changes to many places in DataFusion as well as downstream creates -- most recently #4177

For example, here is how you have to create one today:

    ListingOptions {
            format: Arc::new(file_format),
            collect_stat: false,
            file_extension: self.file_extension.to_owned(),
            target_partitions,
            table_partition_cols: self.table_partition_cols.clone(),
            file_sort_order: None,
        }
    }

Describe the solution you'd like
What I would like is a builder style API for this struct such that new() takes the required arguments and there are with_ functions to set them

ListingOptions::new(file_format)
  .with_collect_statistics(false)
  .with_file_extension(self.file_extension.to_owned()),
  .with_target_partitions(target_partitions)
  .with_table_table_partition_cols(self.table_partition_cols.clone())

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Flagging this as good first issue as it is a software engineering effort that can be mostly guided by the compiler and would be a good way to get familiar with the codebase

cc @tustvold

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions