Finding the "top" test function #12

Closed
schwern opened this Issue May 27, 2010 · 1 comment

1 participant

@schwern
Test-More member
  • GC-ID: 6
  • GC-Labels: Type-Enhancement, Priority-High, Milestone-Test-Builder2
  • GC-Status: Started
  • GC-Attachments: No

One of the major problems with Test::Builder is that it does not know which
function is the user-level test function. Here's a simple example
reimplementing is()

sub is {
    my($have, $want, $name) = @_;

    my $tb = Test::Builder->new;
    return $tb->ok( $have eq $want, $name ) or
        $tb->diag( <<"END" );
have: $have
want: $want
END

}

$tb->ok() must look up the stack in order to print out the file and line
number diagnostics. It also needs this to find the $TODO variable. It
needs to know that is() is the user-level called function. Currently it
does this by maintaining how many levels up the stack the "top" is,
$Test::Builder::Level. This is complicated and buggy and requires careful
tweaking by the test module author.

Test::Builder2 needs a better way to find the top function.

One option is to have test modules declare their test functions to
Test::Builder2. Something like...

install_test( is => sub {
my($have, $want, $name) = @_;

return $TB->ok( $have eq $want, $name ) or
    $TB->diag( <<"END" );

have: $have
want: $want
END

});

This gives Test::Builder total control over entering and leaving each test
routine. It can remember the top level, eliminating the need for $Level.
It can perform start and end of test functions, such as die on fail. And
it can pre-declare a localized Test::Builder object for use in the function
($TB above).

This scheme would seem to solve a number of problems. The drawback is that
it subverts the normal way to write a Perl module. Having alternative ways
of declaring a test function may help mitigate this. Possibilities include:

# Using attributes
sub is : Test_Function {
    ...
}

# Using an already declared routine
install_test( "is" );

It's worth considering how this would interact with an autoloader.

A nice bonus is that this allows test functions to use other test
functions. For example, there is a natural tendency to wrap around
Test::More rather than use Test::Builder. This will work if the default
for diagnostics is to use the highest test function in the stack.

use Test::More;

install_test(is => sub {
    my($have, $want, $name);

    return ok($have eq $want, $name)
      or diag(...);
});


Test::Builder2::Module implements install_test() and TB2 has a stack containing the
top of the test. t/Builder2/top.t shows that it can keep track of the top user level
even with multiple wrapped calls.


A way to work towards this type of thing in Test::Builder without breaking backwards
compat:

Encourage predicate function authors to inform Test::Builder when predicate functions
are entered or left via code in the predicate function:

sub is {
my $predicate_handle = $TB->enter_predicate;

}

enter_predicate() would return a blessed object who's destructor deals with the
"inform T::B when you leave a predicate" part.

enter_predicate() records the predicate caller's stack frame details in the
Test::Builder object, removing the need to maintain $Level within T::B. Predicates
that use enter_predicate() are free to call other predicates without worrying about
$Level. enter_predicate() takes account of $Level when called, so calling a
new-style predicate that uses enter_predicate() from an old-style predicate that
increments $Level would work correctly.

If the builder already has predicate caller stack frame details stored, then
enter_predicate() does nothing. This would be the case when new-style predicates are
called from within other new-style predicates.

Each entry preidcate entry point into Test::Builder would need to check that
enter_predicate() has been called, and call it on behalf of its caller if not.

I think that gives total backward compat, freedom from "local $Level = $Level + 1"
within Test::Builder, and hoops like the ones I've just been jumping through in
http://github.com/schwern/test-more/commit/396afe27401099c3dd12f66ccaaba69cffccb28d
would go away.

Let me know what you think, maybe I'll have a go at this in a branch.


Good effort. The Test::Builder issue isn't really about breaking backwards compat,
its about making all the existing Test::Builder based modules Just Work without all
of them having to change their code.

If Test modules are going to change their code then we might as well just add TB2's
methods of tracking to TB1. install_test() to Test::Builder::Module and
test_start/test_end to Test::Builder. This is fully backwards compatible and keeps
TB1's interface the same as TB2 in this respect.

What I hadn't thought about is the TB1 -> TB2 upgrade path and backporting TB2
features into TB1. It would be great if you implemented install_test() in
Test::Builder::Module and converted some Test::More functions to use it as a trial.
Give it a shot in the Test-Builder2 branch please.

PS A sentinel object is not ideal. An explicit "end of test" trigger takes the
result of the test so that end-of-test actions can act on the result. See
t/Builder2/assert.t for an example of use.


@schwern
Test-More member

I'm going to close this up as TB2 has already solved this. There's smaller problems still open, but they have their own issues (like assert stack issues).

@exodist exodist added a commit that referenced this issue Apr 21, 2016
@exodist exodist v0.000031
    - Regenerate README files
    - Apply spelling fixes (aquire->acquire) #11
    - Improve error message for missing hubs #12
e510fde
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment