Expand the capability of the :runtime option in Mix

[bitwalker]

Per my conversation with @josevalim earlier today in IRC, this fix is intended to address a discrepancy between how projects run under Mix versus releases.

Previously, the purpose of runtime: true | false was to specify whether a given application should be included in the runtime applications list. However, this presents problems when running projects under Mix versus under releases. When run via Mix, setting runtime: false would result in the application not being started (because it's not in the runtime applications list), but would allow you to still load/start it at runtime (because Mix allows you to dynamically load applications which are on the code path). When in a release, the application is not even available to load as it was excluded from the release due to setting runtime: false. In other words, true | false is not granular enough to express the intermediate state, where you want the dependency included at runtime, but not started.

This change adds two additional :runtime option values, :load and :start. false behaves like it did before, and true is supported for backwards compatibility here (it has the same meaning as :start), :start takes it's place as the new default value. We also introduce :load which is effectively the same as false when running a project under Mix, but allows release tools (such as Distillery) to specify the start type of the application to use when compiling the release.

[josevalim]

Thanks @bitwalker. The concern is: should we do something in Mix if :runtime is set to :load? Also, how can we propagate this information to the .app file to make sure a dependency with loaded application is also properly loaded+started?

[fishcakez]

This does not provide suitable guarantees because a runtime: :load dep is not guaranteed to be loaded when Application.ensure_all_started returns :ok. Similarly the application is not guaranteed to be loaded by mix. We can not provide this guarantee without it being built into :application in some way. I think this requires a PR to OTP otherwise release tools need to handle their own configuration.

[bitwalker]

@josevalim Mix doesn't need to do anything with :load, as running Application.ensure_all_started/1 will just dynamically load the application and start it (along with whatever it needs loaded/started). Nothing needs to go in the .app either, as start types are not stored there, rather they are stored in the .rel (and used when generating the .script and .boot files used by a release).

I'm not sure I follow @fishcakez's concern here, at least as far as what "guarantees" need to be provided here - my understanding is that when running under Mix, if some code executes which references a module which isn't in a loaded application, we attempt to load that module and then proceed. For example, if I open up a shell and run :crypto.supports(), it just works, even though :crypto isn't in Application.loaded_applications. If I reference some application not on the code path, I get the expected error that it doesn't exist. The two use cases of :load are either including an application you want to use code from, but don't want to start it's supervisor tree (probably less common), or you want to start it manually at some later point. As long as the application is on the code path, both of those use cases "just work" under Mix.

With releases it's a different situation:

1. Distillery generates the list of applications to include in the release and their start types by getting the list of applications from Mix, and extending the list/overriding start types via the release config file rel/config.exs. The default start type is :permanent. Anything with runtime: false is excluded (as this was the entire point of introducing runtime: false as far as I recall from my conversations with José)
2. If a project is using the new "inferred runtime applications list" behaviour, but has a dependency that they want to lazily start (or maybe conditionally start), it seems as if people are currently accomplishing this by tagging the dependency with runtime: false to exclude it from being automatically started, but then relying on Mix's behaviour to load and start it later. This obviously doesn't work with releases because if the application is excluded from the release with runtime: false, then they won't be able to load it at all, and additionally, releases run in embedded mode, so even if the application was on the code path, it couldn't be dynamically loaded at runtime. Prior to this, people just omitted the application from the applications list, and added it to the applications list in rel/config.exs with a start type of :load, they had no need to set runtime: false unless they truly wanted to exclude the dependency from the release. Now, setting runtime: false is the only way to achieve the "load-only" behaviour with inferred applications - it's not what it was intended for, but is the only way to achieve it. This sets people up for a surprise when they switch from local development to running the release. The only workaround right now is to switch back to using an explicit applications list.

The primary issue here is that there is no way for anyone to express that they want to load an application at runtime, but not automatically start it. The facilities for this already exist in OTP, in the release tooling - the problem is that there is nowhere in the Mix project file to express it. In my opinion, this PR is the cleanest fix which doesn't actually impact current behaviour with Mix, but provides release tools with enough information to correctly generate a release package with the desired properties.

[fishcakez]

In mix, or anytime outside a release, Application.load won't be called on the dep when runtime: :load is used on that dep, which is misleading. For example it wont be loaded when Application.ensure_all_started is called on project app. This means default env(and other specs) for that application won't be available and would be different in dev/test and releases. This could only be added by patching :application.

It is also fragile because while the code for that dep might be available the modules are not guaranteed to work correctly after changing even patch version because it may start requiring supervision tree and that's not part of public API, it's an implementation detail. Also it is unclear what dependencies of a :load application should do, the dep may require any of its (transient) deps to be loaded or started, and this requirement can again change on patch version because it's an implementation detail. This makes load only deps an antipattern in mix.

The way to handle this situation in mix is to use configuration and/or add functionality to start parts of tree later in the dep. This ensures all deps are available and working as intended. This may requiring patching deps.

I understand that reltool (and relx) support this feature and it is convenient in short term to be available to a user but in my opinion this does not belong in mix and should be actively discouraged because the semantics are not good and behaviour would still be different in mix and releases.

[fishcakez]

@tsloughter your expert opinion would be valued on load only applications, and why rebar3/relx behaves as it does.

[tsloughter]

To be clear, runtime: false simply means it is not included in the generated .app list of applications?

The load or none has to be done at the top level release config because there is no OTP defined precedence for when one app has load, another none, another normal, etc.

[fishcakez]

To be clear, runtime: false simply means it is not included in the generated .app list of applications?

In mix yes. The implication is ofc that the dep is not used at runtime.

[bitwalker]

@fishcakez Distillery and relx are separate tools, just FYI.

In mix, or anytime outside a release, Application.load won't be called on the dep when runtime: :load is used on that dep, which is misleading.

Seems to me, that if this is a gap we wish to close, we could do so by ensuring that Mix runs Application.load on such applications, loading any of their dependencies which are not already loaded or started. I don't think that's necessary though, and I'll get into that below.

This means default env(and other specs) for that application won't be available and would be different in dev/test and releases. This could only be added by patching :application.

I think you're missing the fact that developers are already doing this. Omitting applications from the applications list and dynamically loading them later (under Mix) or using a start type of :load with releases is a thing I see very often. The only issue at this point is the fact that when building a release, we're unable to distinguish when a runtime: false dep is being excluded because it's only needed at compile-time (the original purpose) or because someone is trying to prevent the application from being inferred as a runtime dependency and being automatically started. Just supporting runtime: :load would close the gap between Mix and Distillery (and release tools in general) in this regard. To be clear this only presents itself as a problem with inferred application lists, not explicitly managed application lists - because in the former everything is added by default and in the latter, nothing is added by default.

It is also fragile because while the code for that dep might be available the modules are not guaranteed to work correctly after changing even patch version because it may start requiring supervision tree and that's not part of public API, it's an implementation detail. Also it is unclear what dependencies of a :load application should do, the dep may require any of its (transient) deps to be loaded or started, and this requirement can again change on patch version because it's an implementation detail. This makes load only deps an antipattern in mix.

This is a separate issue - and apparently not one which is causing anyone problems at this point. I agree with what you are saying here though. The thing is that almost universally, developers are using "load-only" deps to lazily start some application (i.e. via Application.ensure_all_started/1) - for that use case, the problems you've mentioned are a non-issue.

[fishcakez]

Distillery and relx are separate tools, just FYI.

I asked tristan's opinion because he has the most experience with build/release tools on BEAM. Always worth asking a knowledgable person :).

Seems to me, that if this is a gap we wish to close, we could do so by ensuring that Mix runs Application.load on such applications, loading any of their dependencies which are not already loaded or started. I don't think that's necessary though, and I'll get into that below.

Unfortunately we can not ensure that mix loads these applications because it would not work consistently. For example in ExUnit with no start where a user manually starts applications. For this to work it would need to be built into :application, whereby the dep was loaded, and its deps started/loaded, when the application was started with ensure all started. I am not even sure how that would work. Perhaps could have :loaded_applications, that was similar to :included_applications. However it would mean we could not support it until we are OTP 21+, which is long way off, so likely inadequate.

I think you're missing the fact that developers are already doing this. Omitting applications from the applications list and dynamically loading them later (under Mix) or using a start type of :load with releases is a thing I see very often. The only issue at this point is the fact that when building a release, we're unable to distinguish when a runtime: false dep is being excluded because it's only needed at compile-time (the original purpose) or because someone is trying to prevent the application from being inferred as a runtime dependency and being automatically started. Just supporting runtime: :load would close the gap between Mix and Distillery (and release tools in general) in this regard. To be clear this only presents itself as a problem with inferred application lists, not explicitly managed application lists - because in the former everything is added by default and in the latter, nothing is added by default.

I think we should be discouraging people from doing this. Having a load only dep is fragile (for reasons previously stated) and ambiguous because it is uncertain what the intent is for deps of that dep. Do they need to be started? Loaded? Not even in path? One would assume loaded I guess.

This is a separate issue - and apparently not one which is causing anyone problems at this point. I agree with what you are saying here though. The thing is that almost universally, developers are using "load-only" deps to lazily start some application (i.e. via Application.ensure_all_started/1) - for that use case, the problems you've mentioned are a non-issue.

It is not a separate issue because mix becomes forced to support it and we would want to prevent people getting into a situation where people are open to these bugs. Mix does not have the boot of a release so is open to many more issues. If that is the main use case then it sounds like we need to improve education on start phases. That already works in mix.

[bitwalker]

I asked tristan's opinion because he has the most experience with build/release tools on BEAM. Always worth asking a knowledgable person :).

Oh sure, no worries, I just thought I would mention it in case you thought Distillery was built on relx like exrm was.

Unfortunately we can not ensure that mix loads these applications because it would not work consistently. For example in ExUnit with no start where a user manually starts applications. For this to work it would need to be built into :application, whereby the dep was loaded, and its deps started/loaded, when the application was started with ensure all started. I am not even sure how that would work. Perhaps could have :loaded_applications, that was similar to :included_applications. However it would mean we could not support it until we are OTP 21+, which is long way off, so likely inadequate.

I'm not sure I follow - when loading an application, you can simply load all of it's dependent applications as well, which is information that you can get via Application.spec/2 (which I imagine you are already aware of), I'm not sure what you mean by "it would not work consistently" here. But that's beside the point from my perspective - the way things work in Mix today is fine, I'm not proposing any changes to it's behaviour. I am proposing that we redefine the :runtime option to describe runtime semantics, since it's already being abused to get "load-only" behaviour - this way release tooling can "do the right thing", even if we don't agree that it's the "right thing", it should still be consistent between Mix and releases.

Using included_applications is of course an option which covers at least some of the reasons "load-only" applications are desired - but it carries caveats and requires more thought to use properly. It isn't the right solution for all such cases though.

I think another point of confusion here is what application start types are used for in the first place - Erlang only uses them in releases, where they are used to guide the generation of instructions in the boot script (the .script source and the resulting .boot file). Outside of that use case, you can just load/start on demand as long as they are on the code path (and assuming the VM is not running in embedded mode). Given that, I'm not sure what the purpose of a :loaded_applications list would serve, or indeed what the VM would even do with that information.

I think we should be discouraging people from doing this. Having a load only dep is fragile (for reasons previously stated) and ambiguous because it is uncertain what the intent is for deps of that dep. Do they need to be started? Loaded? Not even in path? One would assume loaded I guess.

Well, I'm not so sure, maybe, except there isn't any alternative which allows you to lazily start applications (such as in cases where you need to do some configuration at runtime before you start an application). You can use start phases for that, along with included_applications in some cases, but that's not an acceptable option for libraries.

As for what to do with deps of that dep, it's pretty clear to me that anything the application depends on which isn't already started or loaded by something else gets an effective start type of :load as well - this doesn't change what happens when you run Application.ensure_all_started, just how applications are started on boot - this is how it works already today in Distillery. I don't think there is any ambiguity there.

It is not a separate issue because mix becomes forced to support it and we would want to prevent people getting into a situation where people are open to these bugs. Mix does not have the boot of a release so is open to many more issues. If that is the main use case then it sounds like we need to improve education on start phases. That already works in mix.

There is already a problem! As it stands now, I'm left helping people figure out why things work in Mix but not in releases, because of Mix not guiding users in the right direction or in this case, essentially forcing them to abuse an option for something other than it's intended purpose to achieve something that was effectively not an issue in previous versions of Elixir:

• Mix allows you to "exclude" an application, but still load/start it at runtime, releases do not (and for good reason!)
• Mix's new behaviour with inferring the applications list now requires someone to add runtime: false if they don't want it started when their application starts, usually because they need to do some preliminary work to set up the environment/configuration for that dependency. They then abuse Mix's ability to dynamically load/start applications. There is no workaround for this.
• A variant of this was present prior to inferred applications, except that they simply omitted the dependency from the applications list, and then loaded/started it at runtime.
• Releases support only loading an application on boot so that it can be started some time later, but because Mix has no support for configuring applications this way (it just "magically" works), release tools have no way of identifying such applications without separate configuration - the need for such extra/external configuration is confusing to users because from their perspective things already were setup properly with Mix, so "why doesn't the release work the same way". The answer I'm hearing is that, officially, Mix doesn't support loading applications and starting them later - even though it totally works and many projects do exactly that.

If the opinion is "we would want to prevent people getting into a situation where people are open to these bugs", then I would like to point out that you are already failing in this regard, people are already open to this, and Mix is doing nothing to help. Of course, Mix can't do too much because applications with runtime: false still have to be on the code path in order for "build-only" tools to work - but it can provide some granularity to the :runtime option so that tools integrating with Mix can do the right thing, which will still let people get themselves into trouble (like they already can!) but without the additional confusion of having things work under Mix and then not work under releases.

Just want to apologize if I'm coming across as rude - I really hope I'm not, I have the utmost respect for you all - I'm just a little confused why we're looking at this PR as if it opens us up to the issue that it's trying to address. At worst, all it does is reduce the cognitive dissonance when switching between Mix and releases - all of the problems you've raised are already an issue today, it's just that I get to shoulder the support burden ;)

[fishcakez]

At worst, all it does is reduce the cognitive dissonance when switching between Mix and releases - all of the problems you've raised are already an issue today, it's just that I get to shoulder the support burden ;)

The problem is exactly this, the change does not work in mix but does work in releases. This PR is trying to move the problem from releases to applications but OTP does not support the feature at the applications level, only at the releases level. That means there is very little we can do at the mix level unless applications also support this. Therefore I don't think this approach is good for mix because we would be shipping a feature that doesn't work in mix so it can work in another tool. This is quite an uncompromising position I think.

However, it is clear there is a problem to solve, I did not mean to imply there wasn't, and we do need to find some way to solve this. My opinion about the way to solve this, assuming it is delaying starting of applications, is to provide examples and material on start phases and included applications. Using a combination of these provides delayed startup, whilst also guaranteeing dependency ordering and works in mix and releases because it is at the application level.

[tsloughter]

This PR is trying to move the problem from releases to applications but OTP does not support the feature at the applications level, only at the releases level.

Yes, this is my point as well.

[bitwalker]

With that justification I'm curious why we introduced the :runtime option in the first place - my understanding at the time was that it was specifically to indicate to release tooling that a given dependency should be excluded because it was a compile-time only tool. I'm not aware of any case where runtime: false does anything useful when run under Mix, though please correct me if that's the case. If :runtime is there to assist with release tooling, then it seems natural to me that if there are problems with it, we would start there.

My opinion about the way to solve this, assuming it is delaying starting of applications, is to provide examples and material on start phases and included applications

I agree with this - I've got some documentation on this already, and am in the middle of writing a bunch of new material, including a bunch of examples of using these and how to solve a variety of use cases people have written me about. There are caveats though - in releases you cannot include an application which is already included by something else, but Mix will apparently let you do this no problem, but even if it didn't, you have to know to solve it by including the other application instead; you have to find out the application module for the application you are including and set up an unusual supervisor spec (i.e. supervisor(SomeApp.Application, [:normal, []], function: :start) for example), which is not exactly an onerous requirement of course, but is definitely not intuitive or even discoverable for most. The other thing which I think represents an issue is that if you are using both included_applications and start_phases, but one of your included applications doesn't define start_phase/3, things explode - which is fine if you control the application you are including, but that's not often the case. This could probably be solved by defining start_phase/3 in the Application behaviour as overridable - but I don't know if that's the right approach for sure or not.

If we want to require people to use included_applications for this, I'm thumbs up on that, but from my perspective we then need to do the following, or some approximation:

• Ensure that runtime: false dependencies are unavailable to running applications, it seems to me that if running iex -S mix or mix run --no-halt, it should be feasible to omit such dependencies from the code path. Mix tasks would work as usual. This will make it clear that setting this option really means "not available at runtime", rather than "don't start this application" which appears to be how people read this currently
• Ensure that Mix raises an error if you include an application already included by a dependency
• Probably implement a default start_phase/3 callback in the Application behavior so that you aren't forced to fork the repo of one of your dependencies just so that you can include it.
• Add explicit warnings to Application.ensure_all_started, Application.ensure_started and Application.start warning against using them at runtime to dynamically load/start applications, and to refer to the documentation on included_applications instead.
• Provide much better documentation on the use of included_applications and start_phases.

I'd be happy to tackle these items, but I want to make sure we're on the same page.

[tsloughter]

When getting into the idea that mix raises an error regarding included applications or keeps runtime: false apps out of the code path when running a shell is asking for trouble in my opinion.

Unless mix projects only ever have 1 release this may be fine, but that shouldn't necessarily be a restriction. So there may be project dependencies with an included application that aren't included in the release with another dependency that includes the application... if that makes sense.

It can also be nice to have apps that you don't want in a release available for testing with the shell.

[fishcakez]

I thought runtime: true | false was to dedup the deps and applications entries, and make it clearer the intentions for each dependency - which ofcourse has implications for releases. One particular reason for this was to make it clear that every dep is an application whether it has a supervision tree or not, and to add all those to the applications list - very useful for release mindset.

Ensure that runtime: false dependencies are unavailable to running applications, it seems to me that if running iex -S mix or mix run --no-halt, it should be feasible to omit such dependencies from the code path. Mix tasks would work as usual. This will make it clear that setting this option really means "not available at runtime", rather than "don't start this application" which appears to be how people read this currently

I'm unsure how we can achieve this without breaking backwards compatibility, and many people might use a custom task, such as with phoenix. Also it would limit mix's dynamic nature, which is much more flexible than releases - we wouldn't want to lose that.

Ensure that Mix raises an error if you include an application already included by a dependency

As @tsloughter mentions it is tricky for mix to warn about included_applications. This would fit nicely if it was handled better/more strictly by :application, which could error and Application/mix could handle the error. I think this would make a good contribution to OTP as it is easy for us to support as soon as available.

Probably implement a default start_phase/3 callback in the Application behavior so that you aren't forced to fork the repo of one of your dependencies just so that you can include it.

If there are no start phases then it should never be called? I think it might be extremely rare for an application to include multiple applications and for a subset to require phases and others not. Even in that most complex of situations I guess one would have to use an intermediate app with start_phases set to []. In other case :application_starter should handle things nicely when requiring phases.

Add explicit warnings to Application.ensure_all_started, Application.ensure_started and Application.start warning against using them at runtime to dynamically load/start applications, and to refer to the documentation on included_applications instead.

Provide much better documentation on the use of included_applications and start_phases.

Given the most common behaviour suggested earlier it seems like only included applications would only usually be required and not phases, as people aren't doing that. I didn't check the docs before posting this but I'm sure there is room for improvement on both counts.

That all being said using included_applications has some serious caveats. Configuration and calling functions in an already running application might be simpler. Likely it is simpler to patch these apps. Is it possible to show some examples when this has caused trouble? I realise I'm not being very helpful, but these release issues are difficult but I'm unsure what more can be done at the application/mix level except docs.

[bitwalker]

If there are no start phases then it should never be called?

Right, it would be there to cover the case of when there are start phases.

I think it might be extremely rare for an application to include multiple applications and for a subset to require phases and others not.

I'm not sure that's true, I can think of an example pretty easily - you are building a release from an application in an umbrella where you include one of the other umbrella applications so that you can start a subset of the supervisor tree, but you also need to include one of your dependencies so that you can configure it at runtime - I have an application at work right now that does the former but not the latter, but all it would take to need the latter is if we happened to be using a dependency which needed configuration prior to starting, that's a super low bar to be considered "extremely rare" in my opinion.

Even in that most complex of situations I guess one would have to use an intermediate app with start_phases set to [].

Sure, that's an option, but I can't help but feel like it's a dirty hack which would not be at all obvious to most Elixir devs. I would hope that the language would provide options which make this stuff feel well integrated and less like you are doing something you aren't supposed to be doing.

Likely it is simpler to patch these apps.

I don't think I'd ever say patching some other library is "simple", especially for newcomers to the language just trying to get something into production. It's certainly doable, and you always have the option of maintaining your own fork, but it's horrible user experience. People will feel like they are doing something wrong because the "official" tooling doesn't prevent them from doing the wrong thing, but some random third-party (me) is telling them to do a bunch of other stuff in order to get their application working, when it "worked just fine when run with Mix" (I've heard that phrase sooo many times).

Is it possible to show some examples when this has caused trouble?

Examples of when using included_applications has caused trouble? I can see if there is anything in the distillery/exrm/relx trackers but I can't recall anything specific off the top of my head. If you are talking about an example of when the runtime: false thing has caused problems, I do have some examples from distillery's tracker.

I realise I'm not being very helpful, but these release issues are difficult but I'm unsure what more can be done at the application/mix level except docs.

No worries, this kind of discussion is important :)

I can tell you definitively that while docs are great for telling users to RTFM, there are a very large number of people who don't even bother. Distillery's docs are as rich as ever, but it doesn't prevent people from almost daily opening new issues on the tracker trying to figure out why something they thought worked just fine is now broken when trying to run their release. Then I go read in blog posts or on the Elixir forum or Twitter that "deployment in Elixir is an obstacle to growth" because of the poor impression they have of how well integrated releases are with Elixir. Regardless of the fact that the tooling is well documented, very flexible, and is easy to get started with, these kind of pain points leave a huge negative impression that is very difficult to overcome. Sometimes making things harder on the tooling maintainers in order to improve the experience for users is a tradeoff worth making. We've certainly identified what the "right way" is in this discussion, but I want to emphasize that maybe we should consider the experience for users in the equation too. I can provide workarounds on the Distillery side so that people have an easy out, but it was my impression that supporting releases was an important goal for Mix - and because of that I generally avoid doing anything that widens the gap between how things are done in Mix and how they are done with releases - as it stands now, Mix doesn't really "support releases", it just doesn't not support them.

The average release is more or less a 1:1 mapping with a Mix project - there are exceptions to that rule, including in my own projects, but I would say that it's easily the majority case. Those times where it's not the case, you have to specify the applications to include in the release manually anyway. Given that, I consider it the "happy path", where you should more or less be able to just run mix release and be ready to go - but what's going to happen a lot now, is people are going to lean on runtime: false and Application.ensure_all_started (because they didn't read the theoretical docs we discussed here), things will work just fine under Mix (because it lets you do whatever basically), and then they will run mix release and because Distillery has no way of knowing via the Mix project configuration that some dependency should have been included as "load-only", things will be broken. I will then have to point to the docs and explain for like the 1000th time why things work one way under Mix and another under releases. In theory that's fine - the user failed to do their job and read the docs, and things do work differently under Mix and under releases, but because there is really no way the vast majority of devs are going to intuitively understand that, or even be warned that they might see these differences, they are left confused and frustrated - I can't think of any other language which has such an odd discrepancy between how things work in dev and in production. I think we can do a lot better, or at least should try to do better.

I'm going to close this PR for now - it seems like at this point it's not an acceptable solution to the team, which is fine - I'm just disappointed there doesn't seem to be a good way to close the gap. Unfortunately it means I'll have to paper over it in Distillery somehow. If you have suggestions on how you'd like to approach unifying the tooling, at least as far as making the experience feel tighter, I'm all ears and glad to help. Thanks for taking the time to talk it through!

[fishcakez]

Distillery has no way of knowing via the Mix project configuration that some dependency should have been included as "load-only", things will be broken.

This is a release only configuration/feature, and it can't be handled at the application level because its not supported there in OTP. Mix can support something equivalent to: :none, can that be utilised to close the gap?

[fishcakez]

I wasn't aware of the :none app type for releases. I think a good compromise here would be to add :none to this PR, and explain that :load works like :none in mix. This allows releases to upgrade to a more advanced behaviour, and allow mix a way to describe the behaviour without shipping a subtle broken feature. Mix would need a warning that release tools would behave most similar with :none, rather than :load.

Edit: Perhaps just start with :none, to keep he API simpler, and consider :load in future?

[josevalim]

It seems we have four states:

1. don't include it in a release
2. include it but don't load it
3. include it and load it but don't start it
4. include it and start it

Unfortunately we cannot support option 3 right now because it wouldn't work in Mix nor across dependencies.

I would say that Mix does not support 1, because if you have a dependency, it is always included, even with runtime: false. And I would say that's the disconnection, because releases treat runtime: false to mean 1, Mix treats it to mean 2.

@bitwalker in your experience, which one is more likely to happen for users? 1 or 2? The only case one would expect 1 is with distillery itself? What if you default your behaviour to be the same as in Mix and then provide a mechanism for people to explicitly exclude applications (similar to how they explicitly need to set :load)?

Because I think that, regardless if we add more modes to Mix, we need to make runtime: false mean the same thing in both tools, and that's already not true today.

[OvermindDL1]

I have some code generation libraries that do not nor should not really exist at runtime. Option 1 makes sense for me. In fact if there was an option to only load a library in at 'compile-time' (macro's and so forth) but not exist at mix runtime I would be quite happy, especially if mix itself could be in that list somehow (I know it can't, just bikeshedding there) so people quit accidentally using it then trying to Release while making Mix calls.

[bitwalker]

I would say that Mix does not support 1, because if you have a dependency, it is always included, even with runtime: false. And I would say that's the disconnection, because releases treat runtime: false to mean 1, Mix treats it to mean 2.

From #5473: "Actually, :start is not good because you can have an application that you want included in a release but not started. However, this option is doing both: it is not including it in the release and not starting it.". That's @josevalim explaining why the choice to use :runtime as the option name was made - and it seems pretty clear to me that the desired semantics follow 1 from the list, even if practically it's not possible because of the way Mix works, they are the semantics I followed when deciding how to interpret it.

@bitwalker in your experience, which one is more likely to happen for users? 1 or 2? The only case one would expect 1 is with distillery itself?

There are two different reasons users use :runtime - firstly, to exclude build-time tooling from the release, and secondly, to achieve "load-only" semantics with Mix (and thus releases as well, even though this isn't the recommended approach). So the answer to your question, users want 1 for the former and 3 for the latter (because releases run in embedded mode, the application must be loaded, but from the user's perspective 2 and 3 are effectively the same thing). If we supported :none like @fishcakez proposed, then the implementation in Distillery would be to only include it and load it (the same as :load).

What if you default your behaviour to be the same as in Mix and then provide a mechanism for people to explicitly exclude applications (similar to how they explicitly need to set :load)?

While that's an option, the first thing I'm going to hear from users is "I set runtime: false but the application was still included in the release - it's a build tool, I don't want it in the release!" - to which I will explain that they should set some setting in rel/config.exs to exclude those applications, and they will be frustrated that they are having to maintain effectively the same configuration in two different places. It would affect everyone using releases rather than a solution which only impacts the users trying to get "load-only" semantics using runtime: false.

Because I think that, regardless if we add more modes to Mix, we need to make runtime: false mean the same thing in both tools, and that's already not true today.

My plan at this point is to let users override runtime: false by adding it to the applications list in rel/config.exs. Based on this new definition of what runtime: false is supposed to imply, it's not useful to the release tooling at all - most devs using it will want to simply exclude things, and for the remaining users I'll just point them to the docs for using included_applications, but provide them the workaround via rel/config.exs as an escape hatch. This should keep the ambiguity around what it means to a minimum.

[josevalim]

Thanks @bitwalker!

Something I also noticed that is that the confusion between 1 and 2 also happens with OTP and Elixir, where you can load at them any time inside Mix, even with not listing them, but not inside releases.

Given this symptom is wide-spread and, per your feedback, 2 and 3 would be the same, we are back to the :load discussion, which unfortunately I don't think can be supported until we add loaded_applications or similar to OTP. :(

[fishcakez]

to achieve "load-only" semantics with Mix

Mix does not have "load-only" semantics and can not support it without OTP support. We have been using the term load where it strictly means Application.load, rather than :none which is having modules/app in code path, perhaps there has been some confusion on that point.

2 and 3 are very similar but definitely not the same. I think it would be a mistake to change a :none (in mix) to a :load (in distillery) or call it a :load in mix but behaviour like a :none because the semantics are slightly different. I think it would confused people in a much more likely to be misunderstood way that code not being available because the "wrong" or different config could be applied.

Also I think it would only make sense to make this change (:start | :none) in Elixir if Distillery were to include all those :none by default as :none so behaviour is the same. As its better for distillery not to include those and require non-started apps to be explicitly opt in, and mix can not support :load then we can't close the gap.

[bitwalker]

Mix does not have "load-only" semantics and can not support it without OTP support.

To be clear, I know that, it's why I'm air-quoting it :P. People are setting runtime: false and then using the fact that Mix can dynamically load/start at runtime via Application.ensure_all_started/1 - to them it feels like "load-only" semantics, even though that's not what is happening at all.

2 and 3 are very similar but definitely not the same. I think it would be a mistake to change a :none (in mix) to a :load (in distillery) or call it a :load in mix but behaviour like a :none because the semantics are slightly different. I think it would confused people in a much more likely to be misunderstood way that code not being available because the "wrong" or different config could be applied.

I don't think it would confuse people nearly as much as the current situation.

Also I think it would only make sense to make this change (:start | :none) in Elixir if Distillery were to include all those :none by default as :none so behaviour is the same. As its better for distillery not to include those and require non-started apps to be explicitly opt in, and mix can not support :load then we can't close the gap.

Distillery can certainly do that, except releases run in embedded mode, so :none is either the same as excluding it entirely or means we'll load the code but do nothing with it. You cannot load applications in a release once it's booted in embedded mode. We could run releases in interactive mode, but if I recall there are caveats with that.

It's unfortunate we'll have to continue with this awkward difference between Mix and releases, but I'll see about putting in a PR to OTP for something along the lines of loaded_applications. We can revisit this when that becomes a thing, if it does.

[fishcakez]

I don't think it would confuse people nearly as much as the current situation.

The confusion is worse because bugs can be hidden and non obvious as it code may work but have wrong configuration, whereas the current situation would lead to a crash.

Distillery can certainly do that, except releases run in embedded mode, so :none is either the same as excluding it entirely or means we'll load the code but do nothing with it. You cannot load applications in a release once it's booted in embedded mode.

Are you sure this is true?

[fishcakez]

I just tested a few releases and can't find how :none doesn't work exactly the same as mix runtime: false, it seems to close the gap nicely, what am I missing?