Custom configuration not working

[chrisbitmead]

Expected Behavior

I have an existing grails 3 app, trying to upgrade to grails 5. It loads its own configuration like this:

package services3

import grails.boot.GrailsApp
import grails.boot.config.GrailsAutoConfiguration
import grails.util.Holders
import org.springframework.boot.CommandLineRunner
import org.springframework.context.EnvironmentAware
import org.springframework.core.env.Environment
import org.springframework.core.env.MapPropertySource

import java.time.LocalDateTime
import java.time.format.DateTimeFormatter

class Application extends GrailsAutoConfiguration implements EnvironmentAware {

    static void main(String[] args) {
        GrailsApp.run(Application, args)
    }

    @Override
    void setEnvironment(Environment environment) {

        String configFileName = System.getenv('nsl_services_config') ?: "/etc/nsl/services-config-g3.groovy"
        File configBase = new File(configFileName)

        if (configBase.exists()) {
            DateTimeFormatter dtf = DateTimeFormatter.ofPattern("yyyy/MM/dd HH:mm:ss.SSS");
            println "Loading external configuration from Groovy: ${configBase.absolutePath} ${LocalDateTime.now().format(dtf)}"
            def config = new ConfigSlurper().parse(configBase.toURI().toURL())
            println "load database url: ${config.dataSource.url} ${LocalDateTime.now().format(dtf)}"
            environment.propertySources.addFirst(new MapPropertySource("configurationProperties", config))
        } else {
            println "External config could not be found, checked ${configBase.absolutePath}"
        }
    }
}

In other words, it uses the EnvironmentAware interface so that it can append extra configuration from its own config file. This used to work fine up until at least grails 3, I think also 4 but it's definitely broken in grails 5.0 onwards to 5.2

Oh, one other thing, the problem doesn't happen when using grails run-app, it only happens when running a standalone jar, or in other words when running java -jar application.war. This is baffling that the result differs between these two scenarios.

Actual Behaviour

If you for example, have a built-in config in grails-app/conf/application.groovy then it uses the values in there even though it has successfully loaded the priority config in setEnvironment.

I've spend days trying to trace what happens internally in grails and spring but it's all so complicated. However what I have noticed is that the code was getting properties ( e.g. ConfigurationPropertySourcesPropertyResolver.getProperty) is called many times before setEnvironment is even run. For example server.port property is read before setEnvironment is called, therefore whatever is reading that value can't possibly have access to the correct value.

The net result of this is it starts up on the wrong port and connects to the wrong database because it doesn't read from the correct property sources.

Now if there is some other recommended way to insert one's code into the start up sequence to add one's own property sources, I'm happy to do that, but as far as I know, this is how you are supposed to do it, and it used to work, and there's no other way to do it. The exact same code works in grails 3, but despite successfully loading the config in grails 5 in setEnvironment, it has no effect on the running program.

My best guess is that the startup sequence when running as a standalone jar is slightly different to running grails run-app, and the order is wrong between calling setEnvironment in the jar scenario.

Steps To Reproduce

1. add a setEnvironment method to Application, load your own config file or config values using a method such as the code above.
2. Override in that configuration the values in conf/application.groovy or conf/application.yml
3. Notice that when running as a war / stand alone jar (java -jar application.war), the values aren't picked up.

Environment Information

tried on Linux and Windows.
Tried grails 5.0 and grails 5.2.5
Using java 8.

Example Application

https://gitlab.com/xpusostomos/myappbug2

Version

5.2.5

[chrisbitmead]

I made the above example app. You will see in Application.setEnvironment, if you just include the first 3 lines of a manually constructed Map of values, (and comment out the rest of the function) that server.port is correctly picked up and it starts on 8083. However... if you include all of Application.setEnvironment as in the example app, it fails.

If you create a config file (set the environment variable nsl_services_config to the location of an config.groovy file), and in that file you put:

server.port=8086

then the porty: print statement now prints null, and the application starts on 8080 instead.

So it seems that it works if you pass in a regular map, but ConfigSlurper returns a ConfigObject which is not a simple Map, but some kind of hierarchial map, and this doesn't seem to work anymore under grails5, even though it used to work.

[chrisbitmead]

I'm closing this because I've narrowed down the problem more and will reopen with a more focused bug report

[philboeselager]

@chrisbitmead Can you please link to your bug report? Thanks!

[philboeselager]

@chrisbitmead And can you please add a working example? I'm not sure I understand how exactly "include the first 3 lines of a manually constructed Map of values" is meant.

[xpusostomos]

@philboeselager I found that the issue is that ConfigSlurper generates a hierarchy of values. i.e. if you had a property a.b.c = d, it would return [a: [b: [c: 'd']]].

Something has changed in grails5 it seems that:

environment.propertySources.addFirst(new MapPropertySource("externalGroovyConfig", config))

isn't happy with a hierarchy of values anymore. Now it wants ['a.b.c' = 'd']

The easiest way to achieve this is to change the line:

def config = new ConfigSlurper().parse(configBase.toURI().toURL())

to

def config = new ConfigSlurper().parse(configBase.toURI().toURL()).flatten()

That means that I think documentation around on how to do this is wrong.

I'm not sure if that means you lose the branch nodes from being able to be retrieved. i.e. if you can still retrieve 'a.b' from properties, or if you can only retrieve leaf nodes. As I've said before, moving away from this ability is a terrible stupid brain dead idea, as I've documented here: #12429

When I said "include the first 3 lines of a manually constructed Map of values", I just meant that it's easier to document a test case with a map you hard code: [a: [b: [c: 'd']]], than having a separate yml file or whatever.