Is your feature request related to a problem or challenge? Please describe what you are trying to do.
A Physical plan that is executed has several parameters that would be good to have a way to control via configuration variables. We should have a mechanism, sort of like Ballista, where users can set Environment Variables, load a TOML configuration file, or pass in parameters to function calls via a Python dict with key/value pairs.
Describe the solution you'd like
- Reasonable default values for configurations
- File based configurations possible for overrides
- Functions that trigger "runs" to have a Python Dict that allows for override of global configuration values.
Describe alternatives you've considered
None
Additional context
Add any other context or screenshots about the feature request here.