Prob in elixirc paths used by `:path` dependencies in Elixir 1.2/1.3

[myronmarston]

Environment

• Elixir version (elixir -v):

Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Elixir 1.3.0-rc.1 (9eb4781)

• Operating system:

El Capitan

Current behavior

There's an example app for this in josevalim/umbrella_sample#6.

Running against that branch, if I use a :path dependency (set via the BAR_DEP env var) and run mix test, everything passes:

$ export BAR_DEP=path && elixir -v && rm -rf _build && mix test
Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Elixir 1.3.0-rc.1 (9eb4781)
"/Users/myron/code/umbrella_sample/apps/bar"
"/Users/myron/code/umbrella_sample/apps/bar"
"/Users/myron/code/umbrella_sample/apps/bar"
==> bar
Compiling 2 files (.ex)
Generated bar app
"/Users/myron/code/umbrella_sample/apps/bar"
==> foo
Compiling 1 file (.ex)
Generated foo app
==> bar
"/Users/myron/code/umbrella_sample/apps/bar"
.

Finished in 0.04 seconds
1 test, 0 failures

Randomized with seed 220497
"/Users/myron/code/umbrella_sample/apps/bar"
==> foo
.

Finished in 0.03 seconds
1 test, 0 failures

Randomized with seed 300412

(BTW, you can ignore the printed "/Users/myron/code/umbrella_sample/apps/bar" bits: I added that to confirm that the path was in fact correct regardless of which directory is the current working dir).

But, if I use a :path dependency, cd into apps/foo, and run mix test, if fails -- a module defined in the dependency's test/support directory (which has been added to elixirc_paths), it fails:

$ export BAR_DEP=path && elixir -v && rm -rf _build && (cd apps/foo && mix test)
Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Elixir 1.3.0-rc.1 (9eb4781)
"/Users/myron/code/umbrella_sample/apps/bar"
"/Users/myron/code/umbrella_sample/apps/bar"
==> bar
Compiling 1 file (.ex)
Generated bar app
==> foo
Compiling 1 file (.ex)
Generated foo app
** (CompileError) test/foo_test.exs:3: module Bar.TestHelper is not loaded and could not be found
    (stdlib) erl_eval.erl:669: :erl_eval.do_apply/6
    (elixir) lib/code.ex:363: Code.require_file/2
    (elixir) lib/kernel/parallel_require.ex:56: anonymous fn/2 in Kernel.ParallelRequire.spawn_requires/5

If I instead use in_umbrella: true for the bar dependency, it works in both cases:

$ export BAR_DEP=umbrella && elixir -v && rm -rf _build && mix test
Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Elixir 1.3.0-rc.1 (9eb4781)
==> bar
Compiling 2 files (.ex)
Generated bar app
==> foo
Compiling 1 file (.ex)
Generated foo app
==> bar
.

Finished in 0.05 seconds
1 test, 0 failures

Randomized with seed 709339
==> foo
.

Finished in 0.02 seconds
1 test, 0 failures

Randomized with seed 792854

$ export BAR_DEP=umbrella && elixir -v && rm -rf _build && (cd apps/foo && mix test)
Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Elixir 1.3.0-rc.1 (9eb4781)
==> bar
Compiling 2 files (.ex)
Generated bar app
==> foo
Compiling 1 file (.ex)
Generated foo app
.

Finished in 0.02 seconds
1 test, 0 failures

Randomized with seed 590057

On 1.2, the same behavior I've described exists:

$ export BAR_DEP=path && elixir -v && rm -rf _build && mix test
Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Elixir 1.2.1
"/Users/myron/code/umbrella_sample/apps/bar"
"/Users/myron/code/umbrella_sample/apps/bar"
"/Users/myron/code/umbrella_sample/apps/bar"
==> bar
Compiled test/support/bar_helper.ex
Compiled lib/bar.ex
Generated bar app
"/Users/myron/code/umbrella_sample/apps/bar"
==> foo
Compiled lib/foo.ex
Generated foo app
Consolidated List.Chars
Consolidated Collectable
Consolidated String.Chars
Consolidated Enumerable
Consolidated IEx.Info
Consolidated Inspect
==> bar
"/Users/myron/code/umbrella_sample/apps/bar"
.

Finished in 0.07 seconds (0.07s on load, 0.00s on tests)
1 test, 0 failures

Randomized with seed 767345
"/Users/myron/code/umbrella_sample/apps/bar"
==> foo
.

Finished in 0.00 seconds
1 test, 0 failures

Randomized with seed 803551

$ export BAR_DEP=path && elixir -v && rm -rf _build && (cd apps/foo && mix test)
Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Elixir 1.2.1
"/Users/myron/code/umbrella_sample/apps/bar"
"/Users/myron/code/umbrella_sample/apps/bar"
==> bar
Compiled lib/bar.ex
Generated bar app
==> foo
Compiled lib/foo.ex
Generated foo app
Consolidated List.Chars
Consolidated Collectable
Consolidated String.Chars
Consolidated Enumerable
Consolidated IEx.Info
Consolidated Inspect
** (CompileError) test/foo_test.exs:3: module Bar.TestHelper is not loaded and could not be found

$ export BAR_DEP=umbrella && elixir -v && rm -rf _build && mix test
Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Elixir 1.2.1
==> bar
Compiled test/support/bar_helper.ex
Compiled lib/bar.ex
Generated bar app
==> foo
Compiled lib/foo.ex
Generated foo app
Consolidated List.Chars
Consolidated Collectable
Consolidated String.Chars
Consolidated Enumerable
Consolidated IEx.Info
Consolidated Inspect
==> bar
.

Finished in 0.06 seconds (0.06s on load, 0.00s on tests)
1 test, 0 failures

Randomized with seed 170481
==> foo
.

Finished in 0.00 seconds
1 test, 0 failures

Randomized with seed 205215

$ export BAR_DEP=umbrella && elixir -v && rm -rf _build && (cd apps/foo && mix test)

Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Elixir 1.2.1
==> bar
Compiled test/support/bar_helper.ex
Compiled lib/bar.ex
Generated bar app
==> foo
Compiled lib/foo.ex
Generated foo app
Consolidated List.Chars
Consolidated String.Chars
Consolidated Collectable
Consolidated Enumerable
Consolidated IEx.Info
Consolidated Inspect
.

Finished in 0.05 seconds (0.05s on load, 0.00s on tests)
1 test, 0 failures

Randomized with seed 953446

However, there is a noticeable difference in behavior between 1.2 and 1.3. In 1.2, if you do mix test and then do cd apps/foo && mix test, it works:

xport BAR_DEP=path && elixir -v && rm -rf _build && mix test && (cd apps/foo && mix test)
Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Elixir 1.2.1
"/Users/myron/code/umbrella_sample/apps/bar"
"/Users/myron/code/umbrella_sample/apps/bar"
"/Users/myron/code/umbrella_sample/apps/bar"
==> bar
Compiled test/support/bar_helper.ex
Compiled lib/bar.ex
Generated bar app
"/Users/myron/code/umbrella_sample/apps/bar"
==> foo
Compiled lib/foo.ex
Generated foo app
Consolidated List.Chars
Consolidated String.Chars
Consolidated Collectable
Consolidated Enumerable
Consolidated IEx.Info
Consolidated Inspect
==> bar
"/Users/myron/code/umbrella_sample/apps/bar"
.

Finished in 0.07 seconds (0.07s on load, 0.00s on tests)
1 test, 0 failures

Randomized with seed 576670
"/Users/myron/code/umbrella_sample/apps/bar"
==> foo
.

Finished in 0.00 seconds
1 test, 0 failures

Randomized with seed 608613
"/Users/myron/code/umbrella_sample/apps/bar"
"/Users/myron/code/umbrella_sample/apps/bar"
Compiled lib/foo.ex
Generated foo app
Consolidated List.Chars
Consolidated Collectable
Consolidated String.Chars
Consolidated Enumerable
Consolidated IEx.Info
Consolidated Inspect
.

Finished in 0.05 seconds (0.05s on load, 0.00s on tests)
1 test, 0 failures

Randomized with seed 525439

...so the act of first running mix test at the umbrella root fixes it. But when I do that on 1.3, it still does not work when I'm in the child app directory:

$ export BAR_DEP=path && elixir -v && rm -rf _build && mix test && (cd apps/foo && mix test)
Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Elixir 1.3.0-rc.1 (9eb4781)
"/Users/myron/code/umbrella_sample/apps/bar"
"/Users/myron/code/umbrella_sample/apps/bar"
"/Users/myron/code/umbrella_sample/apps/bar"
==> bar
Compiling 2 files (.ex)
Generated bar app
"/Users/myron/code/umbrella_sample/apps/bar"
==> foo
Compiling 1 file (.ex)
Generated foo app
==> bar
"/Users/myron/code/umbrella_sample/apps/bar"
.

Finished in 0.04 seconds
1 test, 0 failures

Randomized with seed 886356
"/Users/myron/code/umbrella_sample/apps/bar"
==> foo
.

Finished in 0.02 seconds
1 test, 0 failures

Randomized with seed 951032
"/Users/myron/code/umbrella_sample/apps/bar"
"/Users/myron/code/umbrella_sample/apps/bar"
==> bar
Generated bar app
==> foo
** (CompileError) test/foo_test.exs:3: module Bar.TestHelper is not loaded and could not be found
    (stdlib) erl_eval.erl:669: :erl_eval.do_apply/6
    (elixir) lib/code.ex:363: Code.require_file/2
    (elixir) lib/kernel/parallel_require.ex:56: anonymous fn/2 in Kernel.ParallelRequire.spawn_ruires/5

Sorry if that's too much detail; I spent many hours trimming my large umbrella project into something small and reproducible here, and only figured out that our use of :path dependencies was causing the problem. As it so happens, we're only using :path as a way to work around an issue with :in_umbrella that I brought up on the mailing list and that, from what I understand, is fixed in Elixir 1.3, so we should be able to switch back to :in_umbrella to work around this problem.

Still, the behavior of :path dependencies is surprising; it seems like :elixirc_paths in them should work fine.

Expected behavior

The :elixirc_paths option should work on dependencies regardless of if the dependency is specified as an :in_umbrella dependency or a :path dependency. It should work both from the root project directory and from the directory of one of the child apps.

[ericentin]

:path dependencies will always be built with :prod env, while :in_umbrella deps will share the same environment as the current application, so I believe this is expected behavior. I'm not sure why it changed from 1.2 to 1.3, though.

[ericentin]

I'm presuming you only add test/support to :elixirc_paths in :test, btw.

[myronmarston]

:path dependencies will always be built with :prod env, while :in_umbrella deps will share the same environment as the current application,

This is useful information I did not know. Thanks! It's documented pretty clearly but I somehow missed that.

I'm presuming you only add test/support to :elixirc_paths in :test, btw.

Yep:

https://github.com/josevalim/umbrella_sample/pull/6/files#diff-69eb10ba0ae7102ccb907c0b64f6a64bR12

I'm not sure why it changed from 1.2 to 1.3, though.

I decided to try this command:

$ export BAR_DEP=path && elixir -v && rm -rf _build && mix test && tree _build/test/lib && (cd apps/foo && mix test); tree _build/test/lib

Basically, do mix test and then cd apps/foo mix test with a tree _build/test/lib call after each mix test. The results are interesting.

On 1.2, this is what gets printed:

_build/test/lib
├── bar
│   └── ebin
│       ├── Elixir.Bar.TestHelper.beam
│       ├── Elixir.Bar.beam
│       └── bar.app
└── foo
    └── ebin
        ├── Elixir.Foo.beam
        └── foo.app

4 directories, 5 files
# ...
_build/test/lib
├── bar
│   └── ebin
│       ├── Elixir.Bar.TestHelper.beam
│       ├── Elixir.Bar.beam
│       └── bar.app
└── foo
    └── ebin
        ├── Elixir.Foo.beam
        └── foo.app
4 directories, 5 files

So it looks like Elixir.Bar.TestHelper.beam is compiled when mix test is called form the project root, and it remains to be used when I run mix test in apps/foo.

In 1.3 it's different:

_build/test/lib
├── bar
│   └── ebin
│       ├── Elixir.Bar.TestHelper.beam
│       ├── Elixir.Bar.beam
│       └── bar.app
└── foo
    └── ebin
        ├── Elixir.Foo.beam
        └── foo.app

4 directories, 5 files
# ...
_build/test/lib
├── bar
│   └── ebin
│       ├── Elixir.Bar.beam
│       └── bar.app
└── foo
    └── ebin
        ├── Elixir.Foo.beam
        └── foo.app

4 directories, 4 files

Notice that Elixir.Bar.TestHelper.beam is gone when tree is run the 2nd time. So it appears that on 1.3, mix test triggers a "compile reconciliation" (for lack of a better term) even when no compilation is happening. That in turn causes beam files that would not be compiled in the current environment to be purged. 1.2 did not do this.

The 1.3 behavior seems better and more consistent, as it ensures mix test in apps/foo always produces the same result -- where as on 1.2 the results varied based on if you first run mix test in the project root or not.

In my project, the :path dependency option was actually being set incorrectly by some logic that I'm fixing so we're not going to have this issue anymore.

That said, I think there are potentially some improvements to be made here. I spent multiple debugging sessions across multiple days trying to track down the inconsistent behavior I was seeing and found it very confusing. Ideally, running mix test in the project root would result in an error or warning about the conflict between foo having a :path dependency on :bar while having tests that depend upon a module in bar that is only available when compiled in the test environment. I'm not sure how feasible that is, though.

Regardless -- thank you for helping me understand what is going on, @antipax!

[josevalim]

Ok, so the bug here is really that you are trying to run bar in two conflicting environments and we were not reporting it. The reason we were not reporting it is because we were by passing some checks if a dependency comes from the umbrella. I have fix for this in master and I am working on a test next.

Thanks for the fantastic report @myronmarston!

[myronmarston]

Thanks for the fix, @josevalim! Will the fix be included in 1.3 final?

[josevalim]

@myronmarston I am not sure yet. :) I am running master (which is basically v1.3.0 with this fix right now) and you are welcome to do the same. If I don't detect any issues, I may include it but I would like to avoid doing changes to the rc unless we absolutely have to.