feat: load a config from a file

[ramfox]

• use config create to build config
• implement config::Source on each config struct
• tests in:
  • gateway
  • p2p
  • store
  • ctl
  • util
• more manual testing for adding non-default addrs

we can now use a iroh_util::make_config function that takes:

1. default config
2. a list of possible file paths where a config can be loaded
3. a prefix that this config's env vars will be labeled (eg,
   IROH_GATEWAY_PORT=4000, the prefix is IROH_GATEWAY & the field
   that you are trying to set is port
4. command line flag overrides

The function layers these options, starting with the default config and
ending with the command line flags.

• add metrics config to each config file
• add IROH_METRICS as a valid env var prefix
• add IROH_INSTANCE_ID & IROH_ENV to set metrics::Config.instance_id & metrics::Config::service_env, respectfully
• refactor each process to get metrics configuration using our new make_config code path

[ramfox]

changed this not because I necessarily think it's better, but just so that the way we are handling the rpc_addr and rpc_clients is uniform over each config.

[ramfox]

needed to test the conversion between config::Config and the http::HeaderMap, so split this out so it could be used to help with tests separate from the gateway::Config

[Arqu]

Code wise LGTM. Have some functional requests though.
There's a common set of env vars that I use across services which are mainly utilized by the metrics and ops side of things.
Every service has to define their:

• IROH_INSTANCE_ID (which should actually be IROH_INSTANCE_NAME)
• IROH_ENV
  Which you can see at
• https://github.com/n0-computer/iroh/blob/main/iroh-gateway/src/metrics.rs#L10
• https://github.com/n0-computer/iroh/blob/main/iroh-p2p/src/metrics.rs#L10
• https://github.com/n0-computer/iroh/blob/main/iroh-store/src/metrics.rs#L10

Note that CARGO_PKG_VERSION & CARGO_PKG_NAME are compile time env vars and are not in this bunch, so just ignore those.

And then there are metrics specific ones at https://github.com/n0-computer/iroh/blob/main/iroh-metrics/src/config.rs#L30
Which include:

• IROH_METRICS_DEBUG
• IROH_METRICS_COLLECTOR_ENDPOINT
• IROH_METRICS_PROM_GATEWAY_ENDPOINT

These should also be present in all the services. Ideally default for endpoints should be "" which is interpreted as a --no-metrics arg but that can be a separate thing. I want to split metrics and traces anyways.

It would be great to absorb these into the config too so we have a unified interface for managing those.

[ramfox]

@Arqu

Okay, latest commit has these changes!

The biggest change is that the metrics::Config creation process has been refactored to match the other config structs a bit more. For example, it is expected to have a `Config::default()), after which we layer on values found in the config files, then values found in the env vars, then flag overrides (and unique to the metrics config, we layer on compile-time values after that).

This also de-dups some of the work we were previously doing around pulling from env vars (except for the compile-time values)!

I've done some manual testing on top of the tests we already have (by replacing random metrics values & seeing if they properly output in a printed config), but would love to know if this disrupts any services we already have running.

[Arqu]

Functionally perfectly fine!