Issues with interaction of meck with cover + eunit

[sjplatt]

Mocking a covered module multiple times in unit tests cause the tests to fail.

Reproduction Steps

Sample code:

-module(my_repro_module).
-include_lib("eunit/include/eunit.hrl").

first_test() ->
    cover:compile_beam_directory("PATH_TO_DIRECTORY which contains test_module"),
    meck:new(test_module),
    meck:expect(test_module, read, fun(_, _) -> ok end),
    meck:unload(test_module).

second_test() ->
    meck:new(test_module),
    meck:expect(test_module, read, fun(_, _) -> ok end),
    meck:unload(test_module).

Result:

eunit:test(my_repro_module, [verbose]).

module 'my_repro_module'
  my_repro_module: first_test...[0.365 s] ok
  my_repro_module: second_test...*skipped*
undefined
*unexpected termination of test process*
::{terminated,[{io,format,
                   [<0.5771.0>,
                    "WARNING: Deleting data for module ~w imported from~n~tp~n",
                    [test_module,
                     ["PATH_...test_module.1438497.coverdata",
                      "PATH_...test_module_meck_original.1438497.coverdata"]]],
                   []},
               {cover,remove_imported,2,[{file,"cover.erl"},{line,1362}]},
               {cover,fix_state_and_result,3,[{file,"cover.erl"},{line,1496}]},
               {cover,main_process_loop,1,[{file,"cover.erl"},{line,664}]}]}

=======================================================

Observed behavior

If I comment out meck:unload(test_module). in second_test(), the test will pass.
If I add c:l(test_module) at the start of second_test(), the test will pass.

Versions

• Meck version: 0.8.12
• Erlang version: OTP 21

[Add any other environment or test framework related information here, such as
what OS you are using and how Erlang is installed]

[eproxus]

@sjplatt Finally had some time to look at this issue. Wow, what a ride 😅

So, I can trigger some weird behavior which I think is the same, just not as visible (I think Meck exposes this by having another linked process running on the side, which shows some more stack traces and exceptions).

The repro_test module below can trigger a killed error. It is placed in an example project under ./test. In ./test/modules/test_module.erl I have an example test module.

-module(repro_test).
-include_lib("eunit/include/eunit.hrl").

first_test() ->
    erlang:display(cover:compile_beam_directory("_build/test/lib/meck_repro/test")),
    erlang:display(cover:compile_beam_directory("_build/test/lib/meck_repro/test")).

Running this I get:

$ rebar3 eunit
===> Verifying dependencies...
===> Compiling meck
===> Compiling meck_repro
===> Performing EUnit tests...
[{ok,repro_test},{ok,test_module}]

Pending:
  undefined
    %% Related process exited with reason: killed


Finished in ? seconds
2 tests, 0 failures, 2 cancelled
===> Error running tests

(Notice how EUnit can't even 😛 )

If I change the test case lines to point to the directory where only the example module is:

    erlang:display(cover:compile_beam_directory("_build/test/lib/meck_repro/test/modules")),
    erlang:display(cover:compile_beam_directory("_build/test/lib/meck_repro/test/modules")).

It works:

$ rebar3 eunit
===> Verifying dependencies...
===> Compiling meck
===> Compiling meck_repro
===> Performing EUnit tests...
[]
[]
.

Top 1 slowest tests (0.004 seconds, 17.4% of total time):
  repro_test:first_test/0: module 'repro_test'
    0.004 seconds

Finished in 0.023 seconds
1 tests, 0 failures

What's happening is that you're recompiling the test case module itself with cover, while running inside the test case. This means Erlang is killing the process running the test case (because the module is recompiled twice, removing the oldest running version and killing the process inside that code). What you see from Meck and EUnit is just the linked errors eventually cascading through the other linked processes.

Thus, you can fix your test problem by moving the test_module into it's own directory that doesn't contain the test code itself, and only cover compile that.

I'm closing this as invalid for now, but please re-open if this is not the case. 😉

[sjplatt]

Hi,

Thanks for digging into this. Unfortunately, this doesn't seem to be the issue I am running into.

In my situation the two modules are in distinct directories:
EX: ./test/a/test_module.erl and ./test/b/my_repro_module.erl

They are compiled into two separate directories:
EX: ./test/a/ebin/test_module.beam and ./test/b/ebin/my_repro_module.beam

Then in the test we are only calling cover:compile_beam_directory() for the path ./test/a/ebin (as an example).

(Sorry if this is vague, we have a lot of machinery/makefiles for compilation so it is difficult to get a super clean example).

[sjplatt]

@eproxus I don't think I can re-open the issue bc you closed it.

I will also see if I can get a better/more concise repro although might be difficult.

[eproxus]

Are any of the modules used in processes that are linked to the test case?

[sjplatt]

Are any of the modules used in processes that are linked to the test case?

As far as I know no (the test case also doesn't do anything, it simply mecks and unmecks a file).

One other way to potentially repro this (this is how we originally found it, the above example was just a simpler way to repro) would be to start a node, cover compile everything, and then run the above test (without the cover:compile_beam_directory line).