Support HOCON references across sources

[janotav]

Environment Details

• Helidon Version: 2.5.0
• Helidon SE
• JDK version: 14
• OS: Linux
• Docker version (if applicable):

---

Problem Description

According to spec:

Substitution processing is performed as the last parsing step, so a substitution can look forward in the configuration. If a configuration consists of multiple files, it may even end up retrieving a value from another file.

When following configuration is split into two files that are both added as separate ConfigSource:

env {
  publicDomainName: "foo"
}

some {
  value: ${env.publicDomainName}
}

The substitution is not performed.

[spericas]

The substitution across sources is very difficult to do as each config source is handled and resolved independently. The spec is naturally not aware of config sources per se, so it's likely referring to (included) files.

[spericas]

PR #4167
PR #4175

[janotav]

FYI the default choice of resolvingEnabled = false is backward incompatible change that possibly breaks applications migrating from Helidon 2.5.0: something that happened to us and took us quite some time to figure out the root cause...

[spericas]

@janotav So the lazy resolution of references did not work for you for some reason?

[janotav]

No, it did not work in our case.

In our hocon conf we use following pattern frequently:

  endpoint: "https://secret-service-ce."${infra.iaasDomainName}"/v1"
  endpoint: ${?SECRET_SERVICE_ENDPOINT}

We basically provide same standard/pattern value, but allow overriding it using environment variable in the environment where the service is running.

This ${infra.*} thing is basically a map of several infrastructure related properties. We currently use customly written ConfigResolver to implement it.

With cross-source support now implemented in Helidon, I thought we might be able to avoid this custom code and simply use stock MapConfigSource as additional source for these references. It doesn't work however due to:

2022-09-20_09:18:41.71081 Caused by: java.net.URISyntaxException: Illegal character in fragment at index 1: # unresolved merge of 2 values follows (
2022-09-20_09:18:41.71081 # this unresolved merge will not be parseable because it's at the root of the object
2022-09-20_09:18:41.71082 # the HOCON format has no way to list multiple root objects in a single file
2022-09-20_09:18:41.71082 #     unmerged value 0 from Reader: 72
2022-09-20_09:18:41.71083 "https://secret-service-ce/."${infra.iaasDomainName}"/v1",
2022-09-20_09:18:41.71083 #     unmerged value 1 from Reader: 73
2022-09-20_09:18:41.71084 ${?SECRET_SERVICE_ENDPOINT}
2022-09-20_09:18:41.71084 # ) end of unresolved merge
2022-09-20_09:18:41.71084

It is not a big deal though and we can stick to the original workaround of using custom ConfigResolver that takes care of the resolution during parsing stage.