Test function registry #1

Closed
Ovid opened this Issue Nov 18, 2009 · 3 comments

3 participants

@Ovid

Require people to "register" test functions when writing test modules with TB2.

The rationale is that test modules often have many functions which are not test functions. If you want to hook tests (say, to trap output or do something more exotic), it might be useful to write:

foreach my $test (Test::Module->test-functions) { ... }

Cheers,
Ovid

@AndyA

Got a use case? :)

I'd have thought you'd mostly want to do any hooking at the lowest possible level - i.e. at TB level.

And, unless I'm missing the point, this list would be the same as @Test::Module::EXPORT for most test modules wouldn't it?

@Ovid

Hi Andy,

Because if you register something as a test function, you can presumably trap/alter the behavior in it. For example, "die on fail" becomes really easy because you wrap all test functions in sub which dies and you don't worry about diagnostics getting lost (which would happen now because $test->ok dieing immediately would mean the following diag gets lost.

Take a look at the can_ok code. You should see why you can't die the second that the ok in it fails, but have to die when that test function exits. Hence, the desire to "register" test functions :)

And it's not the same as the EXPORT level because some of those functions might return false but not be test functions. Thus, you wouldn't want to die with them.

Cheers,
Ovid

@schwern
Test-More member

The registration is already happening, though for different reasons and it is not currently recorded. See install_test() in Test::Builder2::Module. Adding a method to the TB2::Module role to list a module's test methods would be fine.

@schwern schwern closed this Mar 16, 2014
@exodist exodist added a commit that referenced this issue Aug 18, 2014
@exodist exodist Perfomance enhancements
Stuff was running unreasonably slow in perl blead. This commit tightens
things up. I used NYTProf to find the most-run, and most time consuming
code.

Util::accessor generated accessors were #1, simply because they were
called so much, I was able to make a minor change that gave a huge
boost.

Builder::Trace was the next biggest offender, this was more complicated
to handle:

1) Instead of tracing multiple times, ok, and similar functions locally
cache a trace to re-use if needed. This is by far the biggest
performance boost.

2) In an independant branch I worked on code to avoid 'building' a
trace until necessary. This means it simply grabs all the call stack,
and only calculates what must be done now (encoding, and todo).
Everything else is generated once needed, if at all. In most cases a
build only needs to happen during a test failure. This provided a big
boost, but not as big as #1.

Branches for #1 and #2 were combined and together provided extra
performance boosts.

I also updated a couple places that used trace to simply look at caller
themselves, trace was overkill.

In 2 places Test::Builder does string evals to pretend to be the caller,
this is done to make sure warning go to the correct place, I updated
these to only build the trace if warnings actually occur.

All of this together resulted in huge improvements with the test script
I wrote for profiling.
Original run time: 160 seconds.
Final run time: 15 seconds.

The script was a loop that ran 1000 times running both passing and
failing tests using ok, like, is, is_deeply both inside and outside a
subtest.

This still is longer than the Test::Builder before the alpha, which took
5 seconds, but I believe the added benefit from the alphas is worth
this.
af47c93
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment