How to handle replacement of single config entries inside a nested configuration?

[on3iro]

Hi,

First of all: I just recently started using Figment and really like it - thanks for your work :)
I did not really know where the best place to ask the following question would be, therefore I decided to create an issue.
However of course feel free to point me in another direction and close the issue, if this is the wrong place to ask questions ;)

Now to my question:
I am currently struggling with merging multiple configuration sources for a nested configuration.
Here's a simplified example, to illustrate my setup:

Source A - toml file

I have a toml file structured like this:

[config section A]
sectionAOption1 = 'abc'
sectionAOption2 = 'xyz'

[config section B]
sectionBOption1 = 'foo'

Source B - struct with defaults

#[derive(Deserialize, Serialize, Clone, Default, Debug)]
pub struct Config {
    pub sectionA: SectionAConfig,

    pub sectionB: SectionBConfig,
}

#[derive(Deserialize, Serialize, Clone, Default, Debug)]
struct SectionAConfig {
    section_a_option_1: String
    section_b_option_2: String
}

#[derive(Deserialize, Serialize, Clone, Default, Debug)]
struct SectionBConfig {
    section_b_option_1: String
}

Source C - some arguments parsed by clap

#[derive(Parser)]
#[command(author, version, about, long_about = None)]
struct Cli {
    // Other args shared with some sub commands
    #[command(flatten)]
    common_args: CommonArgs,

    // Some configuration options which are also available as arguments
    #[command(flatten)]
    some_config_options: SomeConfigOptions,

    #[command(subcommand)]
    command: Option<Commands>,
}

#[derive(clap::Args)]
struct SomeConfigOptions {
    #[arg(long, short = 's', default_value = "default")]
    section_a_option_1: String
}

Goal

My struct provides my configuration defaults. Then the toml file takes priority and clap arguments have the highest priority.

Question

What would be the best/simplest way to merge these CLI-arguments with my other sources? As shown above I can't just use my whole CLI-configuration as a source, because a) it has other arguments and commands as well and b) its structure is different to my actual configuration structure. I thought of implementing From<SomeConfigOptions> on my Config, however then I would need to provide default values to the fields missing from my clap args and these would then override parts of my configuration from e.g. the toml file. Another way I thought of would be to create some kind of DTO struct which imitates the structure of my config, so that the nesting would be correct. But that would mean I have to write such a struct for every optional argument and that would feel really hacky. Or do I need to write a provider? But even if so, how would that look like so that it solves my problem?
I feel like I am missing something here and am probably overthinking the whole thing. But I just can't figure out what a good way to do this would be.

Thanks in advance :)

EDIT: Hm 🤔 one way I could think of, is to manually put the values from the arguments into nested HashMaps - that should probably work, but isn't really pretty either...

[SergioBenitez]

All you need to do is ensure that the CLI arguments structure serializes in such a way that's compatible with the structure of your other configuration options. Serde has an incredible number of options to configure the automatic derivation of Serialize; see https://serde.rs for all of its docs, in particular https://serde.rs/attributes.html and the following pages.

Alternatively, you could manually convert your CLI config structure into a Config and merge the resulting value.

[on3iro]

Thanks for your answer - I didn't know about serde attributes, yet. That looks very promising.
I worked around my issue by building a PartialConfigBuilder which converts my CLI-config into a HashMap-representation of the config. This works well, but needs quite a lot of boilerplate, so I'll definitely see, if I can solve this with attributes. (They reason why my CLI currently can't exactly match the config is, because I want these arguments to be optional, but the config itself requires them and sets defaults instead when initializing the Figment - Maybe this is bad design? I am not quite sure. I just didn't want to handle lots of Options inside my code...)

Edit:
I tried to refactor my code and use serde attributes instead, as you suggested. It works like a charm and I could get rid of a lot of boilerplate. Here is what I did:

1. Add additional sub-structs to my clap parser to match the structure of my configuration
2. Because these sub-structs are named differently than the actual structs used inside my real config, I used `#[serde(rename(serialize = ))] container attributes, to make sure they match after serialization.
3. Because my parser arguments are wrappen with Option I also added #[serde(skip_serializing_if = "Option::is_none")] to each field, so that they are simply ignored if they are not set.

ApparentlySerde + Figment take care of the rest of the work and make sure, the arguments are unwrapped correctly to their inner values and merging the result with my base figment works like a charm.