Docker Environment Variables not in emqx.conf

[N0tronic]

Environment:

• EMQ X version 4.3-rc.3
• Docker Engine v20.10.5

What happened and what you expected to happen:

I expect the variable values in config file "etc/emqx.conf" which I'd set as environment variables, but I seen only default values in the config file.

How to reproduce it (as minimally and precisely as possible):

Start the docker container with a environment variable.

Anything else we need to know?:
Seems that the values works that as environments variables set, but can't see that in the config file. Tested by the log.level variable.

[k32]

Could you please show the full command you're using to start the container?

[zmstone]

Hi @N0tronic
Starting from 4.3, the environment variable overrides are not translated as replacements in .conf files.
Instead, the overrides are applied at runtime.

[zmstone]

You can test it with for example this flag: -e EMQX_LOG__LEVEL=debug in docker run command.

[N0tronic]

Thx for prompt reply :-)

@zmstone
Bug or Feature? I know that the variables, set over environment, applied at runtime (look at my description).

@k32
The specific command is not helpful. It´s a general behavior by setting environment variables...

Hope it´s a bug. I need to see the right configs in the configuration file. Otherwise it´s realy difficult to make configuration work.

[zmstone]

Hi @N0tronic This is a feature not a bug.
The difficulty to make configuration work, is maybe from the lack of trust in our new config parser ? ;)

We are aware of the fact that there is currently a lack of means to inspect configuration overrides.
Our proposal is to add logs.

1. before boot, when generating data/app.<time>.config, overrides are printed to console
2. When loading plugin configs after booted, overrides are logged as INFO level logs

[N0tronic]

Ok thx, I understand. The new config parsing make sense. Nevertheless it would be nice to have a view of the actual system config in the config file. What speak against it? It´s confusing if the actual system config differ from config file...

[zmstone]

The old sed replacements only worked for docker.
we wanted a generic solution for both emqx when running for tests, as a systemd daemon, and in docker.

The sed replacements were error prone and hard to maintain.
it will be even harder to maintain when we move to HOCON format in version 5.0,
the values (when there are special chars on the right hand side of =) will have to be quoted,
using sed replacements to fiddle with the escape hell is not fun.

The override rule is simple, environment variable is always the top layer which overrides in-file overrides includes etc.
In this regard I do not think it's confusing at all.

To make the overrides more visible we will log the overrides to console and to emqx logs.
however it's worth mentioning that in-file overrides are not logged, e.g. a later key=2 overrides a prior key=1 in conf file

[N0tronic]

Thx, you´re right. Maybe only confusing at the beginning of config mechanism change ;-)