Test::Builder::Tester is case sensitive, causing tests to fail with 1.5

[barefootcoder]

I recently started co-maint on Test::File. It contains some code like this:

{
local $^O = 'Win32';

my @subs = qw(
    file_mode_is file_mode_isnt
    file_executable_ok file_not_executable_ok
    );

foreach my $sub ( @subs )
    {
    no strict 'refs';

    test_out("ok 1 # skip $sub doesn't work on Windows!");
    &{$sub}();
    test_test();
    }

}        

which works fine on Test::More prior to 1.x, but fails for CPAN Testers who have upgraded to 1.5 (specifically 1.005000_002). The failure looks like:

#   Failed test at t/win32.t line 35.
# STDOUT is:
# ok 1 # SKIP file_not_executable_ok doesn't work on Windows!
#
# not:
# ok 1 # skip file_not_executable_ok doesn't work on Windows!
#
# as expected

i.e. the capitalization of "skip" is different. I asked on perl-qa and Ovid advised me that TAP should be case-insensitve, so perhaps Test::Builder::Tester should be changed to ignore case differences ... ? Or maybe there's a better way to do this than calling test_out directly as brian d foy is doing there, but I didn't see anything in TBT's POD.

This may or may not be related to #254.

[schwern]

This is similar to #254 and has a similar response; TBT was changed to use a formatter which matches how TB1 behaved, but it hasn't been released yet. Please try the Test-Builder1.5 branch and let us know if that solves the problem.

We're moving away from testing the formatted output and towards a more direct test of the result and test events.

You can use Test::Tester which does not rely on the formatted output.

[barefootcoder]

Please try the Test-Builder1.5 branch and let us know if that solves the problem.

But, since the tests are failing for certain CPAN Testers, I don't have any way to really do that ... do I? That is, they would have to upgrade, right?

You can use Test::Tester which does not rely on the formatted output.

Yeah, I thought about that, but I'm hesitant to rewrite the entire test suite (especially since it's not my module), and I'm not sure it's a great idea to have some tests using TBT and others using TT ...

Isn't this a problem that other users of TBT will be having?

[schwern]

On 2012.2.18 2:14 AM, Buddy Burden wrote:

Please try the Test-Builder1.5 branch and let us know if that solves the problem.

But, since the tests are failing for certain CPAN Testers, I don't have any way to
really do that ... do I? That is, they would have to upgrade, right?

You do. You can download the Test-Builder1.5 branch, try it with Test::File,
and let us know the results.

Eventually we'll push out an alpha with the fix, and CPAN Testers will
upgrade, but for right now we need to know if it does fix it. I could do it
myself, but it's better for the project if I delegate small tasks.

If all goes to plan you should have to make no changes to your Test::File tests.

You can use Test::Tester which does not rely on the formatted output.

Yeah, I thought about that, but I'm hesitant to rewrite the entire test suite
(especially since it's not my module), and I'm not sure it's a great idea to
have some tests using TBT and others using TT ...

I'd recommend a complete move to TT because TBT is fundamentally flawed. It
uses the formatted output to do the testing, and small changes to the
formatting have and will continue to cause breakage.

Isn't this a problem that other users of TBT will be having?

Yes. I've taken the approach to fix it across the board by having TBT use a
slightly different formatter which matches the existing quirks of
Test::Builder 0.98. See #242 and #215. Otherwise we'd be playing
whack-a-mole with formatting incompatibilities forever.

[schwern]

1.005000_004 fixed this problem.