Resolving doesn't work #141

markehammons · 2021-01-27T12:35:45Z

I have a config like so:

env_name=local
installation_directory_path=${exePath}/tmp/test_local_docker
#me_user=${whoami}

docker {
    network=test_env
    internal_storage=1
    default_mount=false
    persistent_data=false
}

windows_env=false

clickhouse {
    global {
        replication_factor=1
        image_file=clickhouse-server
        image_name=yandex/clickhouse-server
        container_name=local_clickhouse_server
    }

    mapping {
        http_port=8223
        tcp_port=9101
        interserver_http_port=9101
        docker {
            data_path=/clickhouse/clickhouse-data
            logs_path=/clickhouse/clickhouse-logs
            file_path=/clickhouse/clickhouse-files
        }
        host {
            data_path=${installation_directory_path}/clickhouse-data
            logs_path=${installation_directory_path}/clickhouse-logs
            file_path=${installation_directory_path}/clickhouse-files
        }
        max_server_memory_usage_to_ram_ratio=0.2
    }
}

I merge it with a programmatically generated config

exePath=/home/mhammons/Documents/Projects/opsScala/core/target/scala-2.13

and call resolve, but no matter what I do I get the following error:

org.ekrich.config.ConfigException$UnresolvedSubstitution: String: 102: Could not resolve substitution to a value: ${installation_directory_path}
        at java.lang.Throwable.fillInStackTrace(Unknown Source)
        at org.ekrich.config.ConfigException$UnresolvedSubstitution.<init>(Unknown Source)
        at org.ekrich.config.impl.ConfigReference.resolveSubstitutions(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.realResolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.resolve(Unknown Source)
        at org.ekrich.config.impl.ConfigConcatenation.$anonfun$resolveSubstitutions$2(Unknown Source)
        at org.ekrich.config.impl.ConfigConcatenation$$Lambda$2.apply(Unknown Source)
        at scala.collection.IterableOnceOps.foreach(Unknown Source)
        at scala.collection.AbstractIterable.foreach(Unknown Source)
        at org.ekrich.config.impl.ConfigConcatenation.resolveSubstitutions(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.realResolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.resolve(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject$ResolveModifier.modifyChildMayThrow(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.$anonfun$modifyMayThrow$1(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject$$Lambda$7.apply(Unknown Source)
        at scala.collection.IterableOnceOps.foreach(Unknown Source)
        at scala.collection.AbstractIterable.foreach(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.modifyMayThrow(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.resolveSubstitutions(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.realResolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.resolve(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject$ResolveModifier.modifyChildMayThrow(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.$anonfun$modifyMayThrow$1(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject$$Lambda$7.apply(Unknown Source)
        at scala.collection.IterableOnceOps.foreach(Unknown Source)
        at scala.collection.AbstractIterable.foreach(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.modifyMayThrow(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.resolveSubstitutions(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.realResolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.resolve(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject$ResolveModifier.modifyChildMayThrow(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.$anonfun$modifyMayThrow$1(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject$$Lambda$7.apply(Unknown Source)
        at scala.collection.IterableOnceOps.foreach(Unknown Source)
        at scala.collection.AbstractIterable.foreach(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.modifyMayThrow(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.resolveSubstitutions(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.realResolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.resolve(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject$ResolveModifier.modifyChildMayThrow(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.$anonfun$modifyMayThrow$1(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject$$Lambda$7.apply(Unknown Source)
        at scala.collection.IterableOnceOps.foreach(Unknown Source)
        at scala.collection.AbstractIterable.foreach(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.modifyMayThrow(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.resolveSubstitutions(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.realResolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.resolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext$.resolve(Unknown Source)
        at org.ekrich.config.impl.SimpleConfig.resolveWith(Unknown Source)
        at org.ekrich.config.impl.SimpleConfig.resolveWith(Unknown Source)
        at ops.environments.Local$.<init>(Unknown Source)
        at <none>._SM23ops.environments.Local$G4load(Unknown Source)
        at ops.environments.package$.<init>(Unknown Source)
        at <none>._SM25ops.environments.package$G4load(Unknown Source)
        at ops.Ops$.<init>(Unknown Source)
        at <none>._SM8ops.Ops$G4load(Unknown Source)
        at <none>.main(Unknown Source)
        at <none>.__libc_start_main(Unknown Source)
        at <none>._start(Unknown Source)
        ```

The text was updated successfully, but these errors were encountered:

ekrich · 2021-01-27T17:35:42Z

Thank-you for trying out the library. Sorry you are not finding the library working as expected. I am not much of an expert on using the library so if you could provide some minimized code so I could test it out that would really help.

Another option is to try the lightbend config library with the JVM to see if it works in that environment. I have not been able to follow all the changes they have made so it could be fixed in that library but not this one.

markehammons · 2021-01-28T15:26:55Z

env=local
basedOnEnv=${env}

This is a minimal example. Loaded with

  val config2 = {
    ConfigFactory
      .parseString(
        new String(
          Files.readAllBytes(filesPath.resolve("parameters_base.conf"))
        )
      )
      .resolve()
  }

  println(config2.getString("basedOnEnv"))

Stack trace is:

org.ekrich.config.ConfigException$NotResolved: need to Config#resolve(), see the API docs for Config#resolve(); substitution not resolved: ConfigReference(${env})
        at java.lang.Throwable.fillInStackTrace(Unknown Source)
        at org.ekrich.config.impl.ConfigReference.notResolved(Unknown Source)
        at org.ekrich.config.impl.ConfigReference.valueType(Unknown Source)
        at org.ekrich.config.impl.ConfigReference.valueType(Unknown Source)
        at org.ekrich.config.impl.AbstractConfigValue.equals(Unknown Source)
        at java.util.Objects$.equals(Unknown Source)
        at java.util.AbstractCollection.$anonfun$contains$1(Unknown Source)
        at java.util.AbstractCollection$$Lambda$1.apply(Unknown Source)
        at java.util.ScalaOps$JavaIteratorOps$.exists$extension(Unknown Source)
        at java.util.ScalaOps$JavaIterableOps$.exists$extension(Unknown Source)
        at java.util.AbstractCollection.contains(Unknown Source)
        at java.util.Collections$WrappedCollection.contains(Unknown Source)
        at java.util.Collections$$anon$22.contains(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.realResolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.resolve(Unknown Source)
        at org.ekrich.config.impl.ConfigReference.resolveSubstitutions(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.realResolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.resolve(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject$ResolveModifier.modifyChildMayThrow(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.$anonfun$modifyMayThrow$1(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject$$Lambda$7.apply(Unknown Source)
        at scala.collection.IterableOnceOps.foreach(Unknown Source)
        at scala.collection.AbstractIterable.foreach(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.modifyMayThrow(Unknown Source)
        at org.ekrich.config.impl.SimpleConfigObject.resolveSubstitutions(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.realResolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext.resolve(Unknown Source)
        at org.ekrich.config.impl.ResolveContext$.resolve(Unknown Source)
        at org.ekrich.config.impl.SimpleConfig.resolveWith(Unknown Source)
        at org.ekrich.config.impl.SimpleConfig.resolve(Unknown Source)
        at org.ekrich.config.impl.SimpleConfig.resolve(Unknown Source)
        at ops.environments.Local$.<init>(Unknown Source)
        at <none>._SM23ops.environments.Local$G4load(Unknown Source)
        at ops.environments.package$.<init>(Unknown Source)
        at <none>._SM25ops.environments.package$G4load(Unknown Source)
        at ops.Ops$.<init>(Unknown Source)
        at <none>._SM8ops.Ops$G4load(Unknown Source)
        at <none>.main(Unknown Source)
        at <none>.__libc_start_main(Unknown Source)
        at <none>._start(Unknown Source)

Another option is to try the lightbend config library with the JVM to see if it works in that environment. I have not been able to follow all the changes they have made so it could be fixed in that library but not this one.

I can try that tomorrow if you'd like, but I imagine resolving works there

ekrich · 2021-10-25T00:49:17Z

@markehammons Update: while porting tests to Scala Native, I ran across this that failed only only Scala Native.

[error] Test org.ekrich.config.impl.PropertiesTest.makeListWithNumericKeysAndMerge failed: org.ekrich.config.ConfigException$UnresolvedSubstitution: String: 5: Could not resolve substitution to a value: ${a} was part of a cycle of substitutions involving ${a}, ${a}, took 0.004 sec

ekrich · 2021-10-30T01:40:05Z

@markehammons If you are following, I determined the issue. java.util.IdentityHashMap needed fixing on Scala Native. Luckily I created a good version on Scala.js that I could port over. I put in a pull request there. scala-native/scala-native#2425

We will have to wait for a new release of Scala Native and then this PR where I extend the code coverage can be merged and then we can release another version of sconfig. Kind of a long path but good either way. #205

markehammons · 2021-11-02T12:51:27Z

glad to hear it. I haven't been using scala-native much (mainly because scala 3 has spoiled me), but I'm looking forward to its changes, and hopefully the project I'm currently working on can be used to bridge between scala native and scala on the jvm.

…

On Sat, Oct 30, 2021 at 3:40 AM Eric K Richardson ***@***.***> wrote: @markehammons <https://github.com/markehammons> If you are following, I determined the issue. java.util.IdentityHashMap needed fixing on Scala Native. Luckily I created a good version on Scala.js that I could port over. I put in a pull request there. scala-native/scala-native#2425 <scala-native/scala-native#2425> We will have to wait for a new release of Scala Native and then this PR where I extend the code coverage can be merged and then we can release another version of sconfig. Kind of a long path but good either way. #205 <#205> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#141 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAT5CP3K3KPBL5KCAD2GWQLUJNEIDANCNFSM4WVF6TEA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

ekrich · 2021-12-07T02:55:02Z

Closed via this PR: #205

So using Scala Native 0.4.2 and sconfig 1.4.6 should resolve the issue.

ekrich added bug Something isn't working Scala Native labels Oct 28, 2021

ekrich closed this as completed Dec 7, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Resolving doesn't work #141

Resolving doesn't work #141

markehammons commented Jan 27, 2021

ekrich commented Jan 27, 2021

markehammons commented Jan 28, 2021

ekrich commented Oct 25, 2021

ekrich commented Oct 30, 2021

markehammons commented Nov 2, 2021 via email

ekrich commented Dec 7, 2021

Resolving doesn't work #141

Resolving doesn't work #141

Comments

markehammons commented Jan 27, 2021

ekrich commented Jan 27, 2021

markehammons commented Jan 28, 2021

ekrich commented Oct 25, 2021

ekrich commented Oct 30, 2021

markehammons commented Nov 2, 2021 via email

ekrich commented Dec 7, 2021