A failing subtest within a subtest within a TODO will emit the failures on prove

[petdance]

Originally submitted to https://rt.cpan.org/Ticket/Display.html?id=115522 and I was referred over here.

A failing subtest within a subtest within a TODO will emit the failures on prove. I would expect all the diagnostics for the failures to be hidden, but the nested ones make their way out to the user. The test passes overall, but has messy diagnostics that we don't want to see. The horse/sheep mismatch is suppressed correctly, but the cow/pig mismatch goes to the screen.

$ cat foo.t
use warnings;
use strict;
use Test::More 'no_plan';

diag "Perl $], Test::More $Test::More::VERSION\n";
TODO: {
    local $TODO = 'Have not changed laws of biology';

    subtest 'Farm animals' => sub {
        is( 'horse', 'sheep', 'Horses and sheep are the same' );

        subtest 'Barn animals' => sub {
            is( 'cow', 'pig', 'Cows and pigs are the same' );
        };

        is( 'duck', 'goose', 'Duck and goose are the same' );
    };
}


$ prove foo.t
[16:04:05] foo.t .. # Perl 5.020003, Test::More 1.302015


        #   Failed test 'Cows and pigs are the same'
        #   at foo.t line 13.
        #          got: 'cow'
        #     expected: 'pig'
        # Looks like you failed 1 test of 1.


[16:04:05] foo.t .. ok       63 ms ( 0.00 usr  0.00 sys +  0.06 cusr  0.00 csys =  0.06 CPU)
[16:04:05]
All tests successful.
Files=1, Tests=1,  0 wallclock secs ( 0.03 usr  0.00 sys +  0.06 cusr  0.00 csys =  0.09 CPU)
Result: PASS


$ prove --version
TAP::Harness v3.36 and Perl v5.20.3

[petdance]

I tested this on an old version of Test::More and it passes just fine.

$ prove --version
TAP::Harness v3.29 and Perl v5.10.1

$ prove foo.t
[21:11:26] foo.t .. # Perl 5.010001, Test::More 1.001002
[21:11:26] foo.t .. ok       33 ms
[21:11:26]
All tests successful.
Files=1, Tests=1,  0 wallclock secs ( 0.02 usr  0.00 sys +  0.01 cusr  0.00 csys =  0.03 CPU)
Result: PASS

[exodist]

I have a fix for this, but not a good test (yet) once I write the test I will get the fix out.

[exodist]

this was caused by a different recent bugfix.

[exodist]

fixed