Adding deps to included_applications raises on release

[mobileoverlord]

Environment

• Elixir & Erlang/OTP versions (elixir --version): Elixir 1.10.0-dev (2a3a38e)
• Operating system: *

Current behavior

Adding an application's dependency to the included_applications list raises an error when calling mix release.

** (Mix) :{dependency} is listed both as a regular application and as an included application

Expected behavior

Adding applications to included_applications should not raise an error when building the release.

Here is a sample project to use for reference: https://github.com/fhunleth/included_apps_release_question

In this example, a depends on b and c, but the application a wants b and c to be loaded, but not started. Therefore, b and c are added to the included_applications list of a.

If we were to compile a and call mix run we can see that b and c are loaded, but not started.

iex(1)> Application.loaded_applications
[
  {:mix, 'mix', '1.10.0-dev'},
  {:a, 'a', '0.1.0'},
  {:c, 'c', '0.1.0'},
  {:compiler, 'ERTS  CXC 138 10', '7.4.9'},
  {:b, 'b', '0.1.0'},
  {:kernel, 'ERTS  CXC 138 10', '6.5'},
  {:elixir, 'elixir', '1.10.0-dev'},
  {:logger, 'logger', '1.10.0-dev'},
  {:stdlib, 'ERTS  CXC 138 10', '3.10'},
  {:iex, 'iex', '1.10.0-dev'}
]
iex(2)> Application.started_applications
[
  {:a, 'a', '0.1.0'},
  {:logger, 'logger', '1.10.0-dev'},
  {:mix, 'mix', '1.10.0-dev'},
  {:iex, 'iex', '1.10.0-dev'},
  {:elixir, 'elixir', '1.10.0-dev'},
  {:compiler, 'ERTS  CXC 138 10', '7.4.9'},
  {:stdlib, 'ERTS  CXC 138 10', '3.10'},
  {:kernel, 'ERTS  CXC 138 10', '6.5'}
]

This would be the behavior that I expect from the release runtime. I've commented out this check and always called the put_in and it worked as expected. The question is why the choice was made to raise on this case?

[josevalim]

In this example, a depends on b and c, but the application a wants b and c to be loaded, but not started. Therefore, b and c are added to the included_applications list of a.

I would say the error is correct. If B depends on C and A depends on C as an included application, they have conflicting requirements on how C should be started.

In particular, I don't believe this is the correct usage of included applications, as included applications still need to be started by the includer. However, I understand this may be the only way to achieve what you want with Mix. Can you talk about the use cases? Why do you need something to only be loaded and not started? Would it be ok if they were not started nor loaded?

[fhunleth]

The permutation of deps/included_applications in the linked repository was one of the tries. Another try was to have A depend on B as an included application and not list C at all in A's mix.exs. Since included applications aren't transitive, C was started even though it was only depended on by B and that wasn't wanted.

This is trying to mirror a scenario in a real Nerves application. In that application, B and C directly or indirectly may use hardware that hasn't been set up. This is mainly a first use concern since after the hardware has been set up, it should be good on all future boots. The way this was implemented with Distillery was that B and C were listed as included applications and then started in A after a sanity check of the hardware.

Also, in the real application, A really should only depend on B. I'm not sure how we ended up with C in our dependency list when we were using Distillery. It might have been the only way to get C to be an included_application. I'm not sure.

[josevalim]

@fhunleth thanks for the info. I still think included_applications is not the mechanism to get the behaviour that you want. Here are some alternatives I can think of:

1. Move the check if the application should be started or not to B and C start/2 callback. In case it should not, you can start an empty supervision tree.

2. Instead of listing those as included applications, do not add B and C as dependencies of A. You can do that by passing runtime: false to B and C. Then, in a release, you explicitly include them with applications: [b: :load, c: :load].

I think 2 is the simplest approach. We may be able to make it even leaner by supporting something like runtime: :load too. Have you tried it already? Is there anything that made it fail?

[fhunleth]

Thanks. Option 2 seems to work so far in my testing. (Only hedging since implementing in the real app will take a little more effort.)

I think this issue can be closed (at least my immediate problem is addressed).

Fwiw, I'm finding that a common pattern in Nerves projects is to want sets of OTP applications started or stopped depending on what mode the device is in. I think this is less common in other contexts, and for normal embedded Linux, it's orchestrated using systemd or systemd-like things. The solution for Nerves projects seems to be sprinkling Application.start/stops around which is what the included_applications/load solutions had been promoting. I'm not that happy with this, so I have mixed feelings towards encouraging anyone to spend time on runtime: :load.

Thanks for your help!

[josevalim]

@fhunleth I see. If that's the case, then I think 1 is the best approach. Basically it needs to be a feature of the application to start only parts of it, or not at all, based on some configuration or some runtime check. After all, wouldn't the application know better the requirements it needs to run?

Anyway, I will close this as the original issue doesn't hold but I will be glad to discuss solutions to the general problem (which seems to not have been fully addressed).

[DavidRawk-Blake]

The only issue with 1. What if you don't maintain the application. If it's an application you want to be able to start based on your apps config. If ApplicationX only needs brod (for instance) if the config says brod is enabled, then we're trapped.

[josevalim]

@DavidRawk-Blake you can set brod: :load in your release (and probably as runtime: falsein your deps. Then call Application.start(:brod) as necessary in your application start callback.