Matching environment variables with snake_case and nested structs

[battila7]

Hey,

first of all, many thanks for maintaining this library. It is truly of great help for developing configurable applications.

My issue is as follows. In my application I prefer to use nested structs for my configuration, for example:

struct Config {
  log: Log,
}

struct Log {
  format: LogFormat,
  level: LogLevel,
}

Since I want stuff to be configurable via environment variables, I use Env::prefixed along with split to handle nestings correctly. So APP_LOG_FORMAT works.

However, things start to get messy when I add snake_case fields to the mix:

struct Config {
  node_identifier: String
}

In this case, Figment is unable to correctly resolve APP_NODE_IDENTIFIER as it assumes a node.identifier nesting when employing split. Thus, I have to choose between "nesting with no multi-word/snake_case field names" and "snake_case but no nesting".

Is there any way to use environment variables, nested structs and snake_case field names at the same time, out-of-the-box?

Bests,
Attila

[SergioBenitez]

Have you read the Env docs on nesting? The .split() method is just a convenience for mapping the _ character in environment variable names to . to get nesting. If this doesn't work for you, you can map() anyway you'd like.

In your case, one quick-and-easy solution is to remove your use of split() in favor of the following .map():

env.map(|k| match k {
    k if k == "node_identifier" => k.into(),
    k => k.as_str().replace("_", ".").into()
})

[battila7]

Sure, I read the docs on nesting, with an emphasis on the mentioned split and map functions. However, I ditched the map function (and its use as proposed by you), as it did not seem to be that scalable for me, when it comes to multiple levels of nesting and many snake_case fields.

Carefully registering each nesting and field by hand in the configuration loading code did not seem to be the way to go. I mean, it seemed to be a better choice to drop nesting as opposed to managing a complex configuration loading logic.

Thus, I actually opened this issue aware of your proposed solution, but with the thought that '"there must be a cleaner way than this" 😄

[SergioBenitez]

Got it.

The difficulty here is that there does not exist an unambiguous way to split an environment variable name into nestings. For example, the name A_B_C could be A[B][C] or A_B[C] or A[B_C] or A_B_C (indeed, there are 2^n options). In some cases, knowing the structure of the final config value would help disambiguate. In this example, for instance, if a_b_c is a field in the structure while a_b and a are not, then we should clearly prefer to parse A_B_C without splitting. But what happens if both a and a_b are fields in the structure? Or if a_b and a_b_c are fields? And so on? Clearly, we've hit an irresolvable ambiguity.

This is why Figment supports the unambiguous {} syntax. The four cases above become:

• A[B][C] -> A={B={C}}
• A_B[C] -> A_B={C}
• A[B_C] -> A={B_C}
• A_B_C -> A_B_C

Nevertheless, for cases where an ambiguity does not exist, or a preference strategy is desired, we could imagine a Provider being given access to the Figment as it exists thus-far, at which point it could use existing config keys to try to parse smarter splits. For unambiguous cases, this would allow us to write a split() that just works. For ambiguous cases, it's unclear what such a "smart" split() would do, if anything.

[battila7]

Yeah, that's completely reasonable, thanks for the detailed explanation.

Most probably, I got confused because the frameworks and libraries I worked with so far either required explicit environment variable names (such as node-convict) or inferred the environment variable names based on the target class/object/struct hierarchy (such as config or the Spring framework).

Thanks again for the provided insights, I may revisit this issue going down the Provider path you mentioned if I have additional time. Anyway, this is a great library, and I really appreciate the effort you put into it!