Add 'generator' option to eunit to run a single generator

[ostinelli]

Eunit allows to specify individual test generators to run, instead of all the tests or a specific suite (Ref). This is very useful when you are working on a single test and don't necessarily want to run all tests every time you are writing one.

You can for instance run just the single test myapp_mymod_tests:myfunc_test_/0 by running:

eunit:test({generator, myapp_mymod_tests, myfunc_test_}).

This patch allows to specify a single generator test like this:

rebar eunit generator=myapp_mymod_tests:myfunc_test_

Output of make check:

$ make check
[...]
Command 'debug' not understood or not applicable
Congratulations! You now have a self-contained script called "rebar" in
your current working directory. Place this script anywhere in your path
and you can use rebar to build OTP-compliant apps.
==> rebar (xref)
make: [dialyzer_warnings] Error 2 (ignored)

Three new tests added, partial output of rebar eunit -v:

Ensure EUnit runs with tests in a 'test' dir and a defined generator
  rebar_eunit_tests:68: eunit_with_generator_test_ (Specific module is found and run)...ok
  rebar_eunit_tests:71: eunit_with_generator_test_ (Specific test is found and run)...ok
  rebar_eunit_tests:74: eunit_with_generator_test_ (Single test is run)...ok

[dizzyd]

+1 for selecting generators/tests with "tests" parameter.

[ghost]

@richcarl thoughts?

[richcarl]

I think that if you write suite=a,b,... test=x,y,..., then rebar should for each m in a,b,... check module m (and m_tests) for test functions (..._test() and ...test()), and for each t in x,y,..., pick the shortest named test function (if any) such that t is a prefix of the name, and run that as a test if it is named ..._test() or as a test generator if it is named ...test(). Alternatively, don't allow t to be an arbitrary prefix, but only an exact match of the whole name before _test() or test().

[ostinelli]

fair enough.

however, according to the documentation I don't see an option to run a specific test, just a generator.

@richcarl can you please point me in the right direction?

[richcarl]

Oops, you're right, I never implemented {test, M, F}. I think I should do that. (Unofficially, {M,F} works right now but should be removed, since it's ambiguous.) Meanwhile, you can use eunit_test:mf_wrapper(M,F) to turn a module and function name into a 0-arity test fun. This gives better error reporting than erlang:make_fun(M,F,0) in case the module or function is missing.

[ostinelli]

@richcarl the two functions you are referring to eunit_test:mf_wrapper(M,F) and erlang:make_fun(M,F,0) seem to be undocumented (or at least, I wasn't able to find any docs for them).

I'm a little doubtful on how to proceed.

Wouldn't it be better to implement only the generators selection for now, and leave the implementation of the tests once these are properly supported in eunit? Or is there a way to implement those in a clean and maintainable way (i.e. without undocumented functions)? @Tuncer?

[richcarl]

Don't be such a wuss, Roberto! ;-)

Anyhow, you can trust that eunit_test:mf_wrapper(M,F) won't go away anytime soon, even though it's not documented. And erlang:make_fun(M,F,0) isn't documented either, but in practice it's currently the only way of dynamically making a symbolic fun object from atoms+arity, and it's used by several projects; it's not likely to be removed or renamed.

[ostinelli]

No problem in going this way, if that's what it takes. ^^_

But before I dig into this I'd still want to hear from @Tuncer, as I'd rather avoid losing dev time on this if this route isn't viable in the first place.

[ghost]

Works for me as @richcarl made it clear and promised that the undocumented functions won't be changed or removed. @richcarl would it make sense to document the two functions?

[richcarl]

I don't think eunit:mf_wrapper/2 should be documented. (In my dev repo at github, I have added {test,M,F} which has the same effect, but that might not be included in the official OTP distro soon enough for you. But you can change your implementation to use this later on when it's generally available.) The erlang:make_fun/3 function shouldn't need to be documented either - actually, if you're only targeting R15 and later, you can simply use the syntax "fun M:F/A" where M, F, A, are variables (only constant atoms and integers are allowed in R14 and earlier). From the R15A relase notes: "OTP-9643 Variables are now now allowed in 'fun M:F/A'".

[ostinelli]

I've opened a new pull request #282 implementing what has been discussed above.