Add config/runtime.exs

[josevalim]

Full proposal here: https://groups.google.com/forum/#!topic/elixir-lang-core/jWB-uC7_8UU

"config/runtime.exs" will be loaded both by Mix and Releases, closing the gap between them.

For Mix, "config/runtime.exs" will load after the code is compiled and before the application starts, this allows "config/runtime.exs" to rely on any available code - they can even start any necessary dependency - as long as you keep in mind that any application that is started during "config/runtime.exs" cannot be configured by "config/runtime.exs" itself. Furthermore, given "config/runtime.exs" works at runtime, changing it won't require the whole application to be recompiled.

For Releases, it will work precisely the same as "config/releases.exs". If both are available, "config/runtime.exs" is executed first, followed by "config/releases.exs".

There are a couple pitfalls to be aware though:

1. Since "config/runtime.exs" is used by both Mix and releases, it cannot configure :kernel, :stdlib, :elixir, and :mix themselves. Attempting to configure those will emit an error. For those rare scenarios, you will need to use "config/releases.exs" - but "config/releases.exs" will remain simple, which will reduce the odds of syntax errors.

2. Since "config/runtime.exs" is used by both Mix and releases, it cannot invoke "Mix" directly. Therefore, for conditional environment compilation, we will add a env/2 macro to Config that will be available for all config files. For example, someone could do:

   import Config
   
   env :prod do
     config :my_app, :secret_key, System.get_env!("SECRET_KEY")
   end

One may argue that "config/runtime.exs" should eventually replace "config/config.exs" as the default file for application configuration. We will certainly evaluate this option in the future but it is important to take baby steps. And the first step is to support "config/runtime.exs". :)

Implementation

Although the feature is relatively small, it requires many improvements to Mix and the underlying config engine. The tasks are:

• Make sure all apps are consistently loaded before compilation - especially with the new app tracer in master (suggestion: add a app.load that will be invoked before compilation)
• Add a feature to Config.Reader that allows a warning to be emitted if an undesired module is used (for example, Mix)
• Add the env/2 macro to Config (we will also need a way to configure multiple environments and also say all environments but this)
• Raise if "import_config" is used in "config/runtime.exs" and "config/releases.exs" - providing proper guidance to users
• Load config/runtime.exs inside Mix - we should warn if a "forbidden" module is used
• Copy config/runtime.exs inside escripts and load them when the escript runs
• Copy config/runtime.exs inside releases (similar to config/releases.exs)

One aspect to consider is exactly when runtime config should be loaded inside Mix. We need to choose between doing it at the end of the "compile" task or before "app.start". The issue is that many projects have tasks that only need the application to be compiled but not started. For example, Ecto Repo management tasks or Phoenix routes tasks. Those tasks today simply run Mix.Task.run("compile"). However, if we were to introduce "config/runtime.exs" and load it before "app.start", those tasks will now run without "config/runtime.exs" and behave incorrectly.

Our proposal is to do it at the end of the "compile" task, however, given commands like "release" and "escript.build" need to compile code but does not want to run its runtime config, we will need a flag to skip loading the runtime configuration.

[josevalim]

Instead of the following:

env :prod do
  config :my_app, :secret_key, System.get_env!("SECRET_KEY")
end

I propose we simply introduce a env() macro instead:

if config_env() == :prod do
  config :my_app, :secret_key, System.get_env!("SECRET_KEY")
end

The env will be passed to the configuration file, retaining functionality while using Mix while being compatible with releases.

[josevalim]

Another change on the proposal above: we don't need both config/runtime.exs and config/releases.exs, only one of them. Therefore we will respect config/releases.exs for now but it will be marked for deprecation in the future in favor of the more general purpose config/runtime.exs.

[josevalim]

We may need to support config_target/0 as well for Nerves. We also need to check if relying exclusively on config/runtime.exs will be ok for Nerves. /cc @mobileoverlord. :)

[mobileoverlord]

The default project generation today does not rely on config/releases.exs due to the the danger it presents that this proposal fixes, which I am very excited about! :)

For Nerves, we find that runtime configuration is most valuable when the configuration occurs after the VM is loaded and before any dependencies have started. Relying on config/runtime.exs is okay, we would definitely need support for config_target/0.

[devonestes]

Would building support for .env and .env.test files into this new runtime configuration be something that would make sense, or would that be better done in a library?

[josevalim]

Definitely done in a library. And it can be done more reliably now, since libraries will actually be available by the time config/runtime.exs runs.

[Adzz]

Clearly a bit late for this, but is config/runtime.exs really config/boot_time.exs ?

[mmalter]

Clearly a bit late for this, but is config/runtime.exs really config/boot_time.exs ?

It's just mean. I have been got caught by this too.