New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why use applications: for run-time dependencies? #190

Closed
tonyvanriet opened this Issue Jan 20, 2017 · 7 comments

Comments

Projects
None yet
3 participants
@tonyvanriet
Contributor

tonyvanriet commented Jan 20, 2017

I understand that all run-time dependencies need to be declared in the mixfile applications: list in order to be included in the release package. I see some people that have been tripped up by not adding simple library applications to this list. I'm hoping to better understand why Distillery uses the mixfile applications: as opposed to some of the alternatives. Sorry if this question has already been well explained. I dug around the issues here and exrm and couldn't find anything explaining the reasoning.

My understanding had been that this list is essentially passed through to the applications value in the Erlang .app file that is generated which declares "all applications that must be started before this application is allowed to be started".

On the surface, this makes it seem like Distillery is borrowing this mechanism to determine what to include in the release package. Is there some underlying Erlang detail that forces the OTP application startup to be coupled to the run-time module dependencies?

Distillery already inspects the deps in order to warn the user that something may be missing from applications:. What prevents Distillery from just using deps to determine what to include based on the chosen environment? This means the run-time dependency list would be coupled to the build-time dependency list so it's not much of an improvement, but it seems the deps is at least a closer fit to what Distillery needs.

Another alternative would be for Distillery to get this from its own rel/config.exs. Granted, this would be annoying from a maintenance standpoint, but is there any technical reason why it would not work to use this and only putting start/stoppable applications in the mixfile applications:?

@bitwalker

This comment has been minimized.

Show comment
Hide comment
@bitwalker

bitwalker Jan 20, 2017

Owner

Up until Elixir 1.4, we couldn't use the dependencies list because some things which we don't want in the release package must be dependencies in the environment used to build the release - basically, any build-time tools (distillery itself is such an example, it has to be present when MIX_ENV=prod, even though it's not a runtime dependency). As of Elixir 1.4, there is now the :runtime option which can be used in the dependency specification, which tells us to exclude a given dependency from the applications to be included.

So to answer your question more directly, the way it works today is due to legacy behaviour. Beyond that though, there has always been a hesitation to infer meaning from the dependencies list, when there isn't a 1:1 correlation with the applications list - they don't mean the same thing, even though you effectively have to put everything your application depends on at runtime in the applications list.

Distillery will be moving to support the new :runtime option, and thus relying less on everything being properly declared in the applications list, but that hasn't been done yet. Once implemented, it won't require any changes on the user's part, things should just work as expected.

Owner

bitwalker commented Jan 20, 2017

Up until Elixir 1.4, we couldn't use the dependencies list because some things which we don't want in the release package must be dependencies in the environment used to build the release - basically, any build-time tools (distillery itself is such an example, it has to be present when MIX_ENV=prod, even though it's not a runtime dependency). As of Elixir 1.4, there is now the :runtime option which can be used in the dependency specification, which tells us to exclude a given dependency from the applications to be included.

So to answer your question more directly, the way it works today is due to legacy behaviour. Beyond that though, there has always been a hesitation to infer meaning from the dependencies list, when there isn't a 1:1 correlation with the applications list - they don't mean the same thing, even though you effectively have to put everything your application depends on at runtime in the applications list.

Distillery will be moving to support the new :runtime option, and thus relying less on everything being properly declared in the applications list, but that hasn't been done yet. Once implemented, it won't require any changes on the user's part, things should just work as expected.

@tonyvanriet

This comment has been minimized.

Show comment
Hide comment
@tonyvanriet

tonyvanriet Jan 20, 2017

Contributor

I can understand the reluctance to use the build-time dependencies list.

For the sake of argument, say my app has some non-start/stoppable library application in its deps that is not included in applications:. If Distillery did use deps, thereby including this in the release package, would it cause some other runtime problem down the line because the library is not listed in the .app file?

Contributor

tonyvanriet commented Jan 20, 2017

I can understand the reluctance to use the build-time dependencies list.

For the sake of argument, say my app has some non-start/stoppable library application in its deps that is not included in applications:. If Distillery did use deps, thereby including this in the release package, would it cause some other runtime problem down the line because the library is not listed in the .app file?

@tonyvanriet

This comment has been minimized.

Show comment
Hide comment
@tonyvanriet

tonyvanriet Jan 20, 2017

Contributor

This SO answer seems to be alluding to Erlang using this applications list to drive module loading in addition to OTP application startup, though I haven't found anything yet to back that up in the docs.

Contributor

tonyvanriet commented Jan 20, 2017

This SO answer seems to be alluding to Erlang using this applications list to drive module loading in addition to OTP application startup, though I haven't found anything yet to back that up in the docs.

@bitwalker

This comment has been minimized.

Show comment
Hide comment
@bitwalker

bitwalker Jan 20, 2017

Owner

Distillery when it supports adding things from the deps list, will add them to the applications list with a start type of :load. In other words, it will be in the .app file, just not with a type that will cause it to be started.

A better reference for how releases are constructed can be found here. The .app file is used to determine which applications need to be loaded/started, and is recursive. However the file we use to build the release is actually the .rel file, but much of the data is duplicated between them. In the olden days, one had to construct the .rel by hand. Tools like Distillery handle that for you now, but derive the information for the .rel from mix.exs (or .app in the case of Erlang apps).

Owner

bitwalker commented Jan 20, 2017

Distillery when it supports adding things from the deps list, will add them to the applications list with a start type of :load. In other words, it will be in the .app file, just not with a type that will cause it to be started.

A better reference for how releases are constructed can be found here. The .app file is used to determine which applications need to be loaded/started, and is recursive. However the file we use to build the release is actually the .rel file, but much of the data is duplicated between them. In the olden days, one had to construct the .rel by hand. Tools like Distillery handle that for you now, but derive the information for the .rel from mix.exs (or .app in the case of Erlang apps).

@skosch

This comment has been minimized.

Show comment
Hide comment
@skosch

skosch Aug 18, 2017

@bitwalker I'm using Elixir 1.5 and it seems that I still have to add the set applications: [...] line. Is the inference from the dependency list supported now, and if yes, what do I need to do to make it work?

skosch commented Aug 18, 2017

@bitwalker I'm using Elixir 1.5 and it seems that I still have to add the set applications: [...] line. Is the inference from the dependency list supported now, and if yes, what do I need to do to make it work?

@bitwalker

This comment has been minimized.

Show comment
Hide comment
@bitwalker

bitwalker Aug 18, 2017

Owner

@skosch You shouldn't need to, the applications are still inferred (Distillery relies on Mix for this, but also does it's own checks to see if applications were missed). What are you seeing?

Owner

bitwalker commented Aug 18, 2017

@skosch You shouldn't need to, the applications are still inferred (Distillery relies on Mix for this, but also does it's own checks to see if applications were missed). What are you seeing?

@skosch

This comment has been minimized.

Show comment
Hide comment
@skosch

skosch Aug 19, 2017

Thank you Paul. It seems like I misunderstood the purposes of releases vs. environments, and it's all good now.

skosch commented Aug 19, 2017

Thank you Paul. It seems like I misunderstood the purposes of releases vs. environments, and it's all good now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment