New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Triq plugin doesn’t output errors in generators #699

Closed
KrzysiekJ opened this Issue May 11, 2017 · 19 comments

Comments

Projects
None yet
3 participants
@KrzysiekJ
Copy link
Contributor

KrzysiekJ commented May 11, 2017

Let’s say we have the following module:

-module(foo).

-include_lib("triq/include/triq.hrl").

crash_gen() ->
    ?LET(
       {X, _}, %% This won’t match.
       binary(),
       X).

prop_crash() ->
    ?FORALL(
       _,
       crash_gen(),
       true).

When running triq:module(foo), we get something like:

Testing foo:prop_crash/0
** exception error: no function clause matching foo:'-crash_gen/0-fun-0-'(<<"£">>) (src/foo.erl, line 8)
     in function  triq_dom:bind_pick/2 (src/triq_dom.erl, line 955)
     in call from triq:check_forall/6 (src/triq.erl, line 277)
     in call from triq:check/3 (src/triq.erl, line 382)
     in call from triq:all/2 (src/triq.erl, line 299)

However, make triq gives us only:

Testing foo:prop_crash/0
{"init terminating in do_boot",{function_clause,[{erl_eval,expr,3,[]}]}}
init terminating in do_boot ()

Crash dump is being written to: erl_crash.dump...done
@essen

This comment has been minimized.

Copy link
Member

essen commented May 11, 2017

Interesting. I'll check it tomorrow. Thanks!

@essen

This comment has been minimized.

Copy link
Member

essen commented May 12, 2017

Yeah. So if I catch the exception I get this:

Exception error:function_clause in
[{erl_eval,match1,4,[{file,"erl_eval.erl"},{line,1088}]},
 {erl_eval,match_tuple,5,[{file,"erl_eval.erl"},{line,1181}]},
 {erl_eval,match,4,[{file,"erl_eval.erl"},{line,1074}]},
 {erl_eval,match_list,4,[{file,"erl_eval.erl"},{line,1207}]},
 {erl_eval,match_clause,5,[{file,"erl_eval.erl"},{line,983}]},
 {erl_eval,try_clauses,8,[{file,"erl_eval.erl"},{line,924}]},
 {init,start_it,1,[]},
 {init,start_em,1,[]}]

Not quite helpful to say the least. Obviously the shell gets a different exception, but I do not seem to be able to get that one. Open to suggestions.

@KrzysiekJ

This comment has been minimized.

Copy link
Contributor

KrzysiekJ commented Aug 15, 2017

I offer a small bounty of 125 BLK (worth about $29 at the time of writing) to whoever fixes this bug. Rules:

  1. To claim the bounty, include payment address in the commit message, submit a pull request to this repository and reference this issue.
  2. Bounty expires on September 5th 2017 (UTC).
  3. Repository’s contributing guidelines prevail.
  4. Bounty will be paid in seven days.
@essen

This comment has been minimized.

Copy link
Member

essen commented Aug 23, 2017

The problem seems to be in erl_eval, FYI. I'm looking into it.

@essen

This comment has been minimized.

Copy link
Member

essen commented Aug 23, 2017

It seems the stacktrace is eaten by a variable '_'. The question is why would there be this variable (from parsing perhaps?) and why doesn't erl_eval do anything to keep the stacktrace around.

@essen

This comment has been minimized.

Copy link
Member

essen commented Aug 23, 2017

Yes it comes from parsing. So the only question then is why isn't erl_eval handling this case.

1> erl_parse:parse_exprs(element(2, erl_scan:string("try abc catch C:E -> erlang:get_stacktrace() end."))).
{ok,[{'try',1,
            [{atom,1,abc}],
            [],
            [{clause,1,
                     [{tuple,1,[{var,1,'C'},{var,1,'E'},{var,1,'_'}]}],
                     [],
                     [{call,1,
                            {remote,1,{atom,1,erlang},{atom,1,get_stacktrace}},
                            []}]}],
            []}]}
@essen

This comment has been minimized.

Copy link
Member

essen commented Aug 23, 2017

Testing test_triq_generator_error:prop_crash/0
Error error function_clause
[{test_triq_generator_error,'-crash_gen/0-fun-0-',
                            [<<"">>],
                            [{file,"src/test_triq_generator_error.erl"},
                             {line,6}]},
 {triq_dom,bind_pick,2,[{file,"src/triq_dom.erl"},{line,954}]},
 {triq,check_forall,6,[{file,"src/triq.erl"},{line,251}]},
 {triq,check,3,[{file,"src/triq.erl"},{line,356}]},
 {triq,all,2,[{file,"src/triq.erl"},{line,273}]},
 {erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,672}]},
 {erl_eval,eval_lc1,6,[{file,"erl_eval.erl"},{line,698}]},
 {erl_eval,eval_generate,7,[{file,"erl_eval.erl"},{line,727}]}]

So the question now is what a proper patch for this should look like. I will open a ticket in OTP and ask. Might take a while.

@essen

This comment has been minimized.

Copy link
Member

essen commented Aug 23, 2017

Opened PR for discussion at erlang/otp#1540

@essen

This comment has been minimized.

Copy link
Member

essen commented Aug 23, 2017

By the way if you really need to you can apply the patch to erl_eval, recompile it and replace the beam file in your installation (or any other method of overriding the module, but this is what I used for testing quickly) and you'll get a better idea of what your triq issues are.

@essen

This comment has been minimized.

Copy link
Member

essen commented Aug 28, 2017

Saving here the start of a test I wrote for this ticket.

diff --git a/test/plugin_triq.mk b/test/plugin_triq.mk
index 908ce47..84ecc29 100644
--- a/test/plugin_triq.mk
+++ b/test/plugin_triq.mk
@@ -7,7 +7,34 @@ TRIQ_TARGETS = $(addprefix triq-,$(TRIQ_CASES))
 
 triq: $(TRIQ_TARGETS)
 
+triq-generator-error: build clean
+
+   $i "Bootstrap a new OTP application named $(APP)"
+   $t mkdir $(APP)/
+   $t cp ../erlang.mk $(APP)/
+   $t $(MAKE) -C $(APP) -f erlang.mk bootstrap $v
+
+   $i "Add Triq to the list of dependencies"
+   $t perl -ni.bak -e 'print;if ($$.==1) {print "DEPS = triq\n"}' $(APP)/Makefile
+
+   $i "Generate a module containing faulty Triq properties"
+   $t printf "%s\n" \
+       "-module($(APP))." \
+       "-ifdef(TEST)." \
+       "-include_lib(\"triq/include/triq.hrl\")." \
+       "crash_gen() ->" \
+       "   ?LET({X, _}, %% This won’t match." \
+       "       binary(), X)." \
+       "prop_crash() -> ?FORALL(_, crash_gen(), true)." \
+       "-endif." > $(APP)/src/$(APP).erl
+
+   $i "Run the Triq plugin"
+   $t $(MAKE) -C $(APP) triq $v
+
+# @todo And then?
@KrzysiekJ

This comment has been minimized.

Copy link
Contributor

KrzysiekJ commented Aug 29, 2017

Instead of recompiling Erlang, it is also possible to run Triq from the shell. Stack traces are visible then (this issue has been opened as using the plugin is more convenient).

@essen

This comment has been minimized.

Copy link
Member

essen commented Aug 29, 2017

Right. But an actual fix to this ticket will require patching Erlang. Let's hope they look at my PR soon-ish. :-)

@KrzysiekJ

This comment has been minimized.

Copy link
Contributor

KrzysiekJ commented Sep 8, 2017

I hereby extend the bounty until November 1st, giving @essen precedence as he debugged the issue. :)

@KrzysiekJ

This comment has been minimized.

Copy link
Contributor

KrzysiekJ commented Jan 17, 2018

The bounty has expired, but I’m willing to grant part of it anyway. Let it be:

(1 BLK is worth about $0.45 at the time of writing).

To claim the bounty, either paste a BlackCoin payment address here or send me an email (krzysztof.jurewicz@gmail.com, fingerprint of PGP key: 49B8 FA87 441C CF12 268D A7B1 2483 F9A8 5ECD 3D48). In the latter case, put a mean of authentication (like email address) in a comment — in case it’s not already deducible from GitHub profile.

This expires on February 20th.

@essen

This comment has been minimized.

Copy link
Member

essen commented Jan 17, 2018

OK I'll get back to you, thanks!

@uabboli

This comment has been minimized.

Copy link

uabboli commented Jan 19, 2018

I do not intend to claim the bounty. Just doing my job. :)

@essen

This comment has been minimized.

Copy link
Member

essen commented Jan 26, 2018

@KrzysiekJ Let's try this: BMzcciiU2QBZN35X4dbueWg6X7ziJLeMw5

Hope it works!

@KrzysiekJ

This comment has been minimized.

Copy link
Contributor

KrzysiekJ commented Jan 26, 2018

@essen

This comment has been minimized.

Copy link
Member

essen commented Jan 26, 2018

Things happened! Thanks.

Guess we can close this then. Enjoy!

@essen essen closed this Jan 26, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment