Compiler loses track of existing modules

[fabientownsend]

Environment

Erlang/OTP 19 [erts-8.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Elixir 1.4.2

• Operating system: OS X El Capitan

Current behavior

1. Duplicate a module
2. Remove the methods from the duplication but keep the module name
3. Run mix test, it will find the duplication without the method and complain that the methods don't exist
4. Delete the duplication of the module
5. Run mix test, it doesn’t find original module anymore and still complaining that it can't find the method

I assume there are some leftover .beam files floating about… and it doesn't pick up that there is another original module to compile now

To solve this problem I had to:

1. rename the original module
2. save
3. rename it to its original name
4. save

after that the compilation work

Expected behavior

To not loose track of the existing module if they still exist?

[whatyouhide]

Hey @fabientownsend, I couldn't reproduce the issue. If I put the module with the same name in the same file as the original module, recompilation works fine. If I put it in a different file, Mix complains that "cannot define module ... because it is currently being defined in ...".

Steps I used to reproduce:

1. mix new compiler_bug
2. Add a test to test/compiler_bug_test.exs that tests assert CompilerBug.hello() == :world
3. mix test, all tests pass
4. Added defmodule CompilerBug do; end at the bottom of lib/compiler_bug.ex
5. mix test, tests fail as expected since hello/0 cannot be found in CompilerBug
6. Removed defmodule CompilerBug do; end at the bottom of lib/compiler_bug.ex
7. mix test, all tests pass since lib/compiler_bug.ex is recompiled

Can you provide detailed steps to reproduce? :)

[chathaway-codes]

I can confirm this on Elixir 1.4 (Windows 10, NTFS filesystem if that matters).

I put the code here (https://github.com/chuck211991/elixir/tree/test/elixir5983); each step is commented. At each commit, run mix test.

[chathaway-codes]

Here is the output of running mix test after creating the duplicate module (sorry about spam!)

C:\Users\Charles\Programming\elixir5983>elixir --version
Erlang/OTP 19 [erts-8.1] [64-bit] [smp:6:6] [async-threads:10]

Elixir 1.4.0

C:\Users\Charles\Programming\elixir5983>mix --version
Erlang/OTP 19 [erts-8.1] [64-bit] [smp:6:6] [async-threads:10]

Mix 1.4.0

C:\Users\Charles\Programming\elixir5983>mix test
Compiling 1 file (.ex)
warning: redefining module Elixir5983 (current version loaded from _build/test/lib/elixir5983/ebin/Elixir.Elixir5983.beam)
  lib/t/elixir5983.ex:1



  1) test test hello (Elixir5983Test)
     test/elixir5983_test.exs:5
     ** (UndefinedFunctionError) function Elixir5983.hello/0 is undefined or private
     stacktrace:
       (elixir5983) Elixir5983.hello()
       test/elixir5983_test.exs:6: (test)



Finished in 0.04 seconds
1 test, 1 failure

Randomized with seed 807000

C:\Users\Charles\Programming\elixir5983>rm -r lib/t

C:\Users\Charles\Programming\elixir5983>mix test
Generated elixir5983 app
** (CompileError) test/elixir5983_test.exs:3: module Elixir5983 is not loaded and could not be found
    (ex_unit) expanding macro: ExUnit.DocTest.doctest/1
    test/elixir5983_test.exs:3: Elixir5983Test (module)
    (elixir) lib/code.ex:370: Code.require_file/2
    (elixir) lib/kernel/parallel_require.ex:57: anonymous fn/2 in Kernel.ParallelRequire.spawn_requires/5

[chathaway-codes]

Confirmed this on a dev build of 1.5; output below. Different, but not surprising.

So one solution is to do a mix clean; is that sufficient to mark this as "solved"?

C:\Users\Charles\Programming\elixir5983>mix test
Generated elixir5983 app
** (CompileError) test/elixir5983_test.exs:3: module Elixir5983 is not loaded and could not be found
    (ex_unit) expanding macro: ExUnit.DocTest.doctest/1
    test/elixir5983_test.exs:3: Elixir5983Test (module)
    (elixir) lib/code.ex:376: Code.require_file/2
    (elixir) lib/kernel/parallel_require.ex:57: anonymous fn/2 in Kernel.ParallelRequire.spawn_requires/5


22:48:09.330 [error] GenServer #PID<0.112.0> terminating
** (CompileError) test/elixir5983_test.exs:3: module Elixir5983 is not loaded and could not be found
    (elixir) src/elixir_expand.erl:86: :elixir_expand.expand/2
    (elixir) src/elixir_expand.erl:540: :elixir_expand.expand_arg/2
    (elixir) src/elixir_expand.erl:479: :elixir_expand.expand_list/4
    (elixir) src/elixir_expand.erl:418: :elixir_expand.expand/2
    (ex_unit) expanding macro: ExUnit.DocTest.doctest/1
    test/elixir5983_test.exs:3: Elixir5983Test (module)
    (elixir) src/elixir.erl:221: :elixir.eval_forms/4
    (elixir) src/elixir_compiler.erl:61: :elixir_compiler.eval_forms/3
    (elixir) src/elixir_lexical.erl:17: :elixir_lexical.run/3
    (elixir) src/elixir_compiler.erl:27: :elixir_compiler.quoted/3
    (elixir) lib/code.ex:376: Code.require_file/2
    (elixir) lib/kernel/parallel_require.ex:57: anonymous fn/2 in Kernel.ParallelRequire.spawn_requires/5
Last message: {:EXIT, #PID<0.109.0>, {%CompileError{description: "module Elixir5983 is not loaded and could not be found", file: "c:/Users/Charles/Programming/elixir5983/test/elixir5983_test.exs", line: 3}, [{:elixir_expand, :expand, 2, [file: 'src/elixir_expand.erl', line: 86]}, {:elixir_expand, :expand_arg, 2, [file: 'src/elixir_expand.erl', line: 540]}, {:elixir_expand, :expand_list, 4, [file: 'src/elixir_expand.erl', line: 479]}, {:elixir_expand, :expand, 2, [file: 'src/elixir_expand.erl', line: 418]}, {ExUnit.DocTest, :doctest, 1, [file: 'expanding macro']}, {Elixir5983Test, :__MODULE__, 0, [file: 'test/elixir5983_test.exs', line: 3]}, {:elixir, :eval_forms, 4, [file: 'src/elixir.erl', line: 221]}, {:elixir_compiler, :eval_forms, 3, [file: 'src/elixir_compiler.erl', line: 61]}, {:elixir_lexical, :run, 3, [file: 'src/elixir_lexical.erl', line: 17]}, {:elixir_compiler, :quoted, 3, [file: 'src/elixir_compiler.erl', line: 27]}, {Code, :require_file, 2, [file: 'lib/code.ex', line: 376]}, {Kernel.ParallelRequire, :"-spawn_requires/5-fun-1-", 2, [file: 'lib/kernel/parallel_require.ex', line: 57]}]}}
State: {:state, {#PID<0.112.0>, Supervisor.Default}, :simple_one_for_one, [{:child, :undefined, GenServer, {GenServer, :start_link, []}, :temporary, 5000, :worker, [GenServer]}], {:set, {:set, 2, 16, 16, 8, 80, 48, {[], [], [], [], [], [], [], [], [], [], [], [], [], [], [], []}, {{[], [], [], [#PID<0.115.0>], [#PID<0.116.0>], [], [], [], [], [], [], [], [], [], [], []}}}}, 3, 5, [], 0, Supervisor.Default, {:ok, {{:simple_one_for_one, 3, 5}, [{GenServer, {GenServer, :start_link, []}, :temporary, 5000, :worker, [GenServer]}]}}}

C:\Users\Charles\Programming\elixir5983>iex
Eshell V8.1  (abort with ^G)
Interactive Elixir (1.5.0-dev) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> Terminate batch job (Y/N)? y

C:\Users\Charles\Programming\elixir5983>

[chathaway-codes]

I submitted a fix which should address the issue in a more "new user friendly" way; please see pull request #6089

[chathaway-codes]

Should the correct behavior be to:

A. Prevent the user from compiling the duplicate module
B. Do as described above, and simple pick up on the "old" when the "new" one goes away

In the case of B, we also need to address the question of how we decide which module to pick up; should it really just be the most recently modified one? Or should it be the one at the highest subdirectory? How do we know that a file contains a module without compiling it?

I suspect we actually want behavior A; does that seem reasonable?

Thanks!

[whatyouhide]

@chuck211991 A. is not necessary desirable but in any case it's backwards incompatible so we can't do it.