Package dependencies are always treated as global

[kaharlichenko]

Given a quote from Dependencies: Declaring dependencies

Dependencies can be declared in a top-level rebar.config file, and inspected with the rebar3 deps command.
...
All dependencies are to be project-local. This is usually a good choice in order avoid the common problems of global libraries having version conflicts.

It is not obvious that:

• global package dependencies are the only ones that are supported;
• in fact you can list package dependencies in {apps,lib}/*/rebar.config even though it is not documented, but these would be treated as if they were specified in the top-level rebar.config anyways.

Consider an example:

%% apps/myapp/rebar.config
{deps, [{foo, "1.2.3"}]}.

%% lib/mylib/rebar.config
{deps, [{bar, "0.2.0"}]}.

%% rebar.config
{deps, [{baz, "0.3.0"}]}.

One would expect that:

• foo is a package dependency that is used only in myapp;
• bar is a package dependency that is used only in mylib;
• baz is a global package dependency and is used in both myapp and mylib.

This behavior is intuitive, but wrong (at least in rebar 3.0.0). This can be confirmed by running rebar3 deps:

foo (locked package 1.2.3)
bar (locked package 0.2.0)
baz (locked package 0.3.0)

The output of the command above suggests that package dependencies aren't grouped per application. The other commands treat package dependencies the same way, at least escriptize would put bar into myapp script even though it doesn't seem to depend on it.

The situation is worsened by the fact that source dependencies are treated in a different way.

It would be nice if this behavior would be described in more details in the documentation. It would be even better if rebar would issue a warning that deps found in {apps,lib}/*/rebar.config are treated as global ones.

Alternatively per app dependencies could be implemented instead. To my mind this would be the ideal scenario.

Either way or another I think a little more explanation is needed in the docs.

[tsloughter]

There are no "per-app dependencies" because this is Erlang.

Sub-app's like apps/A/rebar.config list of dependencies are treated like a dependency in the top level rebar.config because A itself is basically a "dependency" of the project. Same with apps/B or lib/C. There is no meaningful way or purpose of somehow splitting them instead of all going in _build/<profile>/lib since that is how the VM works.

[ferd]

When we mention project-local, it isn't app-local. It is for the whole project. The idea of a 'global' dependency would refer to cross-project dependencies (much like plugins can be global -- they work everywhere on your OS).

If the dependency is declared for myapp and mylib and both reside under the same project (i.e. both are in apps/ or lib/), all dependencies there should be for the project.

You should call rebar3 from the top-level of the project and get a single _build directory for everything.

You should not call rebar3 individually for each application within a project. That's not how it is intended to work.

So when you get:

foo (locked package 1.2.3)
bar (locked package 0.2.0)
baz (locked package 0.3.0)

The behaviour is correct because that's the union of all rebar.config files in the project's applications. There is no possible 'grouping per application' since there is no way to have duplicate or disjoint dependencies running at once in a VM (aside from hot code loading, but we can't use that as a feature).

[talentdeficit]

declaring dependencies in the individual apps as opposed to the root config is important for being able to publish individual apps to package managers like hex

[kaharlichenko]

Maybe this was the behavior by design and I'm just trying to map my previous experience from different languages, but I still can't wrap my head around it.

Consider an example use case. I build a web service and a command line tool that prepares some assets for this web service. Since they both share a common library for reading/writing the assets it is natural to keep them all in a single repository/source tree.

So I would end up with 3 applications:

• mylib, with no dependencies;
• myweb, with dependencies on mylib and cowboy;
• mycli, with dependencies on mylib and getopt.

Naturally I don't want unnecessary deps hanging around.

When building a release for myweb with relx I get exactly what I want: only the necessary apps are copied to _build/default/rel/myweb/lib/.

But when I build an escript for mycli suddenly it doesn't include mylib in even though I need it, but it does include cowboy even though I don't need it.

There is no meaningful way or purpose of somehow splitting them instead of all going in _build//lib since that is how the VM works.

JVM doesn't tolerate different versions of the same libraries being in the CLASSPATH, but somehow it is possible to achieve separate dependencies for separate sub-apps with maven.

---

As I said maybe I am mapping my previous experience from different languages to Erlang and I try to do something stupid. But for other people like me, it would be nice to have a line or two explaining why it happens this way and what is the Erlang way of doing what I want.

[ferd]

The dependencies of applications for purposes of inclusion inside releases and scripts of all kinds should be defined by what is declared within the .app.src file's {applications, ...} tuple. The rebar3 deps are instructions to fetch dependencies, not necessarily to represent which app depends on them.

[kaharlichenko]

The rebar3 deps are instructions to fetch dependencies, not necessarily to represent which app depends on them.

Ok, point taken. Thanks for explanations. May I suggest to add a couple of lines to the docs making it more explicit to others coming from different backgrounds? I suppose that would help understanding the way rebar works.

---

rebar3 release works the way you described. As for rebar3 escriptize I'm not sure it does behave the same way: for some reason it includes all the {deps, [...]} dependencies listed in rebar.config and ignores all the {applications, ...} listed in .app.src.

Please, see the example project in rebar-1137.tar.gz.

[h8]

I have exactly the same situation and I'm a little bit confused about how to configure my project properly to include needed dependencies for CLI. In my case escriptize ignores dependencies from app.src, but includes all the deps from all rebar config files.

[ferd]

That may be a bug with the escriptize command then.

[kaharlichenko]

The rebar3 deps are instructions to fetch dependencies, not necessarily to represent which app depends on them.

Looks like this is indeed not the case with escriptize:

1. in rebar_prv_escriptize:escriptize it can be seen here that the list of all the package dependencies is passed to escript:create;
2. all_deps fetches the package dependencies from the State here;
3. the State is populated with the list of all the package dependencies in rebar_prv_install_deps.erl;
4. rebar_prv_install_deps:do seems to be called before rebar_prv_escriptize:do due to plugin dependency chain: escript -> compile -> lock -> install_deps.

This way all the {deps, [...]} specified in rebar.config (be it top-level or application-level one) get unconditionally included into the final escript instead of relying on {applications, ...} property of app.src. Should I fire a separate issue for this?

[ferd]

Yeah I think it would be less confusing to open up an issue about escriptize not fetching the right dependency chains rather than in here.

[kaharlichenko]

Thank you for the response, I've created a separate issue.

What about this one? Do you consider the docs incomplete? If so, could you please label it as a documentation bug. This way I could refer to the documentation in other issues and claim that the declared and actual behaviors differ :)

[ferd]

Added to http://www.rebar3.org/v3/docs/dependencies