`{:error, {:no_abstract_code, ...}}` with Elixir 1.5.0-rc.0 and Erlang 20.0 #179

paulswartz · 2017-07-04T11:55:07Z

After updating to the latest RC of Elixir, I started seeing errors when running my test with coverage enabled. It only appears to happen on 1.5.0-rc.0 /and/ 20.0: Elixir 1.4.5 or Erlang 19.3 are okay.

Repo: https://github.com/paulswartz/meck_test
Travis CI builds: https://travis-ci.org/paulswartz/meck_test

Reproduction Steps

git clone https://github.com/paulswartz/meck_test.git
cd meck_test
mix deps.get
mix test --cover

Expected behavior

$ asdf local erlang 20.0
$ asdf local elixir 1.4.5
$ mix test --cover
==> mock
Compiling 1 file (.ex)
Generated mock app
==> meck_test
Compiling 1 file (.ex)
Generated meck_test app
Cover compiling modules ...
..

Finished in 0.3 seconds
2 tests, 0 failures

Randomized with seed 539434

Generating cover results ...
Analysis includes data from imported files
["/Users/paulswartz/Projects/github/meck_test/Elixir.MeckTest.coverdata",
 "/Users/paulswartz/Projects/github/meck_test/Elixir.MeckTest_meck_original.coverdata"]

Observed behavior

$ asdf local erlang 20.0
$ asdf local elixir 1.5.0-rc.0
$ mix test --cover
==> mock
Compiling 1 file (.ex)
Generated mock app
==> meck_test
Compiling 1 file (.ex)
Generated meck_test app
Cover compiling modules ...
.

  1) test can be mocked (MeckTestTest)
     test/meck_test_test.exs:9
     ** (EXIT from #PID<0.219.0>) an exception was raised:
         ** (MatchError) no match of right hand side value: [error: {:no_abstract_code, <<70, 79, 82, 49, 0, 0, 2, 124, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 0, 123, 0, 0, 0, 10, 29, 69, 108, 105, 120, 105, 114, 46, 77, 101, 99, 107, 84, 101, 115, 116, 95, 109, 101, 99, 107, 95, 111, 114, 105, ...>>}]
             (meck) /Users/paulswartz/Projects/github/meck_test/deps/meck/src/meck_cover.erl:32: :meck_cover.compile_beam/2
             (meck) /Users/paulswartz/Projects/github/meck_test/deps/meck/src/meck_proc.erl:387: :meck_proc.backup_original/3
             (meck) /Users/paulswartz/Projects/github/meck_test/deps/meck/src/meck_proc.erl:206: :meck_proc.init/1
             (stdlib) gen_server.erl:365: :gen_server.init_it/2
             (stdlib) gen_server.erl:333: :gen_server.init_it/6
             (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3



Finished in 0.2 seconds
2 tests, 1 failure

Randomized with seed 374996

Generating cover results ...

Versions

Meck version: 0.8.7
Erlang version: 20.0
Elixir version: 1.5.0-rc.0

The text was updated successfully, but these errors were encountered:

eproxus · 2017-07-04T11:58:43Z

Thanks for the detailed bug report!

It seems Mock is running with an older version of Meck, before Erlang 20+ compatibility fixes were added. Can you try to override Meck to version 0.8.7 on Hex.pm?

paulswartz · 2017-07-04T12:00:08Z

Sorry, I wrote the wrong version (I was using meck via the Elixir mock library which was 0.2.1). I've simplified the example to only use meck and that's on 0.8.7.

paulswartz · 2017-07-04T12:04:45Z

Possibly related: a Elixir.MeckTest.coverdata file gets dropped at the root of the project when the tests are run and not cleaned up after.

eproxus · 2017-07-04T12:21:20Z

Preliminary analysis points to that the module Elixir.MeckTest is not compiled with debug_info... Not sure why though.

eproxus · 2017-07-04T12:37:13Z

$ iex -pa _build/test/lib/meck_test/ebin -e 'IO.inspect MeckTest.module_info' -e System.halt
Erlang/OTP 20 [erts-9.0] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]

[module: MeckTest,
 exports: [__info__: 1, hello: 0, module_info: 0, module_info: 1],
 attributes: [vsn: [97807801047081573425803547081250957748]],
 compile: [options: [], version: '7.1',
  source: '/private/tmp/meck_test/lib/meck_test.ex'], native: false,
 md5: <<73, 149, 24, 231, 139, 253, 152, 246, 234, 96, 23, 219, 28, 103, 233,
   180>>]

It should be options: [:debug_info] here

eproxus · 2017-07-04T12:47:10Z

Tried to use the option elixirc_options: [debug_info: true] in the Mix file as documented here (based on this example) but it didn't do anything.

@josevalim @ericmj Any ideas? Did something change in Elixir 1.5.0 that modifies compiler options?

josevalim · 2017-07-04T12:51:15Z

@eproxus on OTP 20, we use the long {debug_info, Backend, Metadata} option and that is not stored in the compile options because it contains a large data structure. Are you explicitly checking for that option instead of checking if the chunk exists?

josevalim · 2017-07-04T12:56:48Z

To be more precise, what do beam_lib:chunks(BeamFile, [abstract_code]) and beam_lib:chunks(BeamFile, [debug_info]) return?

josevalim · 2017-07-04T13:04:42Z

Ok, since @paulswartz was awesome and provided all mechanism to reproduce this issue, I decided to give it a try. The chunk is definitely there:

$ elixir -pa _build/test/lib/meck_test/ebin -e "IO.inspect :beam_lib.chunks :code.which(MeckTest), [:abstract_code]"
{:ok,
 {MeckTest,
  [abstract_code: {:raw_abstract_v1,

Now to find the misbehaving parts.

eproxus · 2017-07-04T14:32:38Z

@josevalim Thanks for looking into it so quickly! Didn't know about the new extended debug_info option, interesting. For some reason, I cannot run your example:

$ elixir -pa _build/test/lib/meck_test/ebin -e "IO.inspect(:beam_lib.chunks(:code.which(MeckTest), [:abstract_code]))"
** (UndefinedFunctionError) function :elixir_erl.debug_info/4 is undefined (module :elixir_erl is not available)
    :elixir_erl.debug_info(:erlang_v1, MeckTest, {:elixir_v1, %{attributes: [], compile_opts: [], definitions: [{{:hello, 0}, :def, [line: 15], [{[line: 15], [], [], :world}]}], file: "/private/tmp/meck_test/lib/meck_test.ex", line: 1, module: MeckTest, unreachable: []}, []}, [])
    (stdlib) beam_lib.erl:652: :beam_lib.chunks_to_data/7
    (stdlib) beam_lib.erl:521: :beam_lib.read_chunk_data/3
    (stdlib) beam_lib.erl:509: :beam_lib.read_chunk_data/2
    (stdlib) erl_eval.erl:670: :erl_eval.do_apply/6
    (stdlib) erl_eval.erl:878: :erl_eval.expr_list/6
    (stdlib) erl_eval.erl:404: :erl_eval.expr/5

In this bug, I think the error comes from :cover.compile_beam/1 so I'm not sure where the problem lies. The relevant snippet from cover:

%% Beam is a binary or a .beam file name
do_compile_beam1(Module,Beam,UserOptions) ->
    %% Clear database
    do_clear(Module),
    
    %% Extract the abstract format and insert calls to bump/6 at
    %% every executable line and, as a side effect, initiate
    %% the database
    
    case get_abstract_code(Module, Beam) of
	no_abstract_code=E ->
	    {error,E};
	encrypted_abstract_code=E ->
	    {error,E};
	{raw_abstract_v1,Code} ->
            Forms0 = epp:interpret_file_attribute(Code),
	    case find_main_filename(Forms0) of
		{ok,MainFile} ->
		    do_compile_beam2(Module,Beam,UserOptions,Forms0,MainFile);
		Error ->
		    Error
	    end;
	{_VSN,_Code} ->
	    %% Wrong version of abstract code. Just report that there
	    %% is no abstract code.
	    {error,no_abstract_code}
    end.

get_abstract_code(Module, Beam) ->
    case beam_lib:chunks(Beam, [abstract_code]) of
	{ok, {Module, [{abstract_code, AbstractCode}]}} ->
	    AbstractCode;
	{error,beam_lib,{key_missing_or_invalid,_,_}} ->
	    encrypted_abstract_code;
	Error -> Error
    end.

eproxus · 2017-07-04T14:34:17Z

Nevermind, PATH problems. Your example works fine.

eproxus · 2017-07-04T15:18:39Z

Figured out what happens, I think. Meck just takes proplists:get_value(options, Module:module_info(compile)) and assumes that is enough. With the new "hidden" debug_info, that option is lost in the process.

josevalim · 2017-07-04T15:30:18Z

Yes, from OTP 20, the debug_info option is just one way of attaching debug_info. Strictly speaking, that was also true in previous versions, since a chunk can be attached at any time, but I guess nobody relied on that.

eproxus · 2017-07-04T15:57:29Z

@josevalim I'm having a bit of trouble understanding how to reliably detect if debug_info has been used or not. Compare:

Erlang

Without options:

$ erl -pa _build/dev/lib/meck/ebin -eval 'io:format("~p~n", [beam_lib:chunks(code:which(foo), [compile_info, debug_info])])'
Erlang/OTP 20 [erts-9.0] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V9.0  (abort with ^G)
1> {ok,{foo,[{compile_info,[{options,[]},
                         {version,"7.1"},
                         {source,"/private/tmp/meck_test/foo.erl"}]},
          {debug_info,{debug_info_v1,erl_abstract_code,{none,[]}}}]}}

With debug_info:

$ erl -pa _build/dev/lib/meck/ebin -eval 'io:format("~p~n", [beam_lib:chunks(code:which(foo), [compile_info, debug_info])])'
Erlang/OTP 20 [erts-9.0] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V9.0  (abort with ^G)
1> {ok,{foo,
        [{compile_info,
             [{options,[debug_info]},
              {version,"7.1"},
              {source,"/private/tmp/meck_test/foo.erl"}]},
         {debug_info,
             {debug_info_v1,erl_abstract_code,
                 {[{attribute,1,file,{"foo.erl",1}},
                   {attribute,1,module,foo},
                   {attribute,4,export,[{f,1}]},
                   {function,8,f,1,
                       [{clause,8,[{var,8,'_'}],[],[{atom,9,ok}]}]},
                   {eof,10}],
                  [debug_info]}}}]}}

Elixir

Without options:

$ erl -pa _build/dev/lib/meck_test/ebin -eval 'io:format("~p~n", [beam_lib:chunks(code:which(''Elixir.MeckTest''), [compile_info, debug_info])])'
Erlang/OTP 20 [erts-9.0] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V9.0  (abort with ^G)
1> {ok,{'Elixir.MeckTest',
        [{compile_info,
             [{options,[]},
              {version,"7.1"},
              {source,"/private/tmp/meck_test/lib/meck_test.ex"}]},
         {debug_info,{debug_info_v1,elixir_erl,none}}]}}

With --debug-info to mix compile:

$ erl -pa _build/dev/lib/meck_test/ebin -eval 'io:format("~p~n", [beam_lib:chunks(code:which(''Elixir.MeckTest''), [compile_info, debug_info])])'
Erlang/OTP 20 [erts-9.0] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V9.0  (abort with ^G)
1> {ok,{'Elixir.MeckTest',
        [{compile_info,
             [{options,[]},
              {version,"7.1"},
              {source,"/private/tmp/meck_test/lib/meck_test.ex"}]},
         {debug_info,
             {debug_info_v1,elixir_erl,
                 {elixir_v1,
                     #{attributes => [],compile_opts => [],
                       definitions =>
                           [{{hello,0},
                             def,
                             [{line,15}],
                             [{[{line,15}],[],[],world}]}],
                       file => <<"/private/tmp/meck_test/lib/meck_test.ex">>,
                       line => 1,module => 'Elixir.MeckTest',
                       unreachable => []},
                     []}}}]}}

The only difference I see for Elixir is the Data section of the debug info, which is considered proprietary (see beam_lib). How am I supposed to detect if debug_info has been used for Elixir code?

josevalim · 2017-07-04T16:32:17Z

@eproxus let's take a step back. Why do you need to detect if debug_info has been used? Asking for the abstract_code should work just fine and be completely transparent for you.

eproxus · 2017-07-05T09:37:41Z

Good point, but I'm not sure. In this case it is cover that doesn't find the abstract code. In this scenario what happens is:

Meck renames the original module
Compiles it with the original options
Enables cover on it because it was originally cover compiled
Cover breaks because in step (2) debug_info is not there

The procedure so far has been to copy the compilation options from the original module, assuming that's the "only" way to get it compiled as originally intended. Not sure how to treat the new debug_info way of doing things. With Erlang, it doesn't break as described above, but with Elixir it seems to do. Is there a way for Elixir to add the debug_info atom to the compilation options just as Erlang does? Another workaround for Meck might be to always add debug_info in the case of cover (because the abstract code is always needed there).

josevalim · 2017-07-05T10:38:50Z

Another workaround for Meck might be to always add debug_info in the case of cover (because the abstract code is always needed there).

That's what cover does and that would be the preferred solution, in my opinion. I don't even think it should be conditional, just always add the debug_info option.

eproxus · 2017-07-05T10:47:41Z

Yeah, makes sense. Thanks for all your input, it has been very helpful!

eproxus/meck#179 was fixed in the most recent meck release, this should unbreak mock for users with elixir 1.5 / OTP 20 meck PR here: eproxus/meck#180

ericentin mentioned this issue Jul 25, 2017

Always add debug_info to compile opts for mocks #180

Merged

eproxus closed this as completed in #180 Aug 7, 2017

connorjacobsen mentioned this issue Aug 29, 2017

upgrade meck to 0.8.8 jjh42/mock#70

Merged

Olshansk pushed a commit to jjh42/mock that referenced this issue Aug 30, 2017

upgrade meck to 0.8.8 (#70)

f2a3b6e

eproxus/meck#179 was fixed in the most recent meck release, this should unbreak mock for users with elixir 1.5 / OTP 20 meck PR here: eproxus/meck#180

gmile mentioned this issue Jan 28, 2018

Bump meck to 0.8.9 parroty/excoveralls#129

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`{:error, {:no_abstract_code, ...}}` with Elixir 1.5.0-rc.0 and Erlang 20.0 #179

`{:error, {:no_abstract_code, ...}}` with Elixir 1.5.0-rc.0 and Erlang 20.0 #179

paulswartz commented Jul 4, 2017 •

edited

Loading

eproxus commented Jul 4, 2017

paulswartz commented Jul 4, 2017

paulswartz commented Jul 4, 2017 •

edited

Loading

eproxus commented Jul 4, 2017

eproxus commented Jul 4, 2017

eproxus commented Jul 4, 2017

josevalim commented Jul 4, 2017 •

edited

Loading

josevalim commented Jul 4, 2017

josevalim commented Jul 4, 2017

eproxus commented Jul 4, 2017

eproxus commented Jul 4, 2017

eproxus commented Jul 4, 2017

josevalim commented Jul 4, 2017

eproxus commented Jul 4, 2017

josevalim commented Jul 4, 2017

eproxus commented Jul 5, 2017

josevalim commented Jul 5, 2017

eproxus commented Jul 5, 2017

{:error, {:no_abstract_code, ...}} with Elixir 1.5.0-rc.0 and Erlang 20.0 #179

{:error, {:no_abstract_code, ...}} with Elixir 1.5.0-rc.0 and Erlang 20.0 #179

Comments

paulswartz commented Jul 4, 2017 • edited Loading

Reproduction Steps

Expected behavior

Observed behavior

Versions

eproxus commented Jul 4, 2017

paulswartz commented Jul 4, 2017

paulswartz commented Jul 4, 2017 • edited Loading

eproxus commented Jul 4, 2017

eproxus commented Jul 4, 2017

eproxus commented Jul 4, 2017

josevalim commented Jul 4, 2017 • edited Loading

josevalim commented Jul 4, 2017

josevalim commented Jul 4, 2017

eproxus commented Jul 4, 2017

eproxus commented Jul 4, 2017

eproxus commented Jul 4, 2017

josevalim commented Jul 4, 2017

eproxus commented Jul 4, 2017

Erlang

Elixir

josevalim commented Jul 4, 2017

eproxus commented Jul 5, 2017

josevalim commented Jul 5, 2017

eproxus commented Jul 5, 2017

`{:error, {:no_abstract_code, ...}}` with Elixir 1.5.0-rc.0 and Erlang 20.0 #179

`{:error, {:no_abstract_code, ...}}` with Elixir 1.5.0-rc.0 and Erlang 20.0 #179

paulswartz commented Jul 4, 2017 •

edited

Loading

paulswartz commented Jul 4, 2017 •

edited

Loading

josevalim commented Jul 4, 2017 •

edited

Loading