Plack test failure on Test-Builder2 #273

Closed
masochist opened this Issue Mar 31, 2012 · 7 comments

Projects

None yet

3 participants

@masochist
PERL_DL_NONLAZY=1 /Users/apeiron/perl5/perlbrew/perls/testbuilder2/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'inc', 'blib/lib', 'blib/arch')" t/*.t t/*/*.t
t/00_compile.t ................................. ok
t/HTTP-Message-PSGI/content_length.t ........... ok
t/HTTP-Message-PSGI/path_info.t ................ ok
t/HTTP-Message-PSGI/utf8_req.t ................. ok
t/HTTP-Server-PSGI/post.t ...................... ok
t/Plack-Builder/builder.t ...................... ok
t/Plack-Builder/mount.t ........................ ok
t/Plack-Builder/oo_interface.t ................. ok
t/Plack-Handler/apache1.t ...................... skipped: TEST_APACHE1 is not set
t/Plack-Handler/apache2-registry.t ............. skipped: TEST_APACHE2 is not set
t/Plack-Handler/apache2.t ...................... skipped: TEST_APACHE2 is not set
t/Plack-Handler/cgi.t .......................... skipped: Test requires module 'HTTP::Request::AsCGI' but it's not found
t/Plack-Handler/fcgi.t ......................... skipped: Test requires module 'FCGI' but it's not found
t/Plack-Handler/http_server_simple.t ........... skipped: Test requires module 'HTTP::Server::Simple::PSGI' but it's not found
t/Plack-Handler/output_encoding.t .............. ok
t/Plack-Handler/standalone.t ................... All 102 subtests passed
t/Plack-HTTPParser-PP/simple.t ................. ok
t/Plack-Loader/auto.t .......................... ok
t/Plack-Loader/auto_fallback.t ................. ok
t/Plack-Loader/delayed.t ....................... ok
t/Plack-Loader/restarter.t ..................... skipped: Dev test
t/Plack-Loader/shotgun.t ....................... 49/? # 55 tests ran, but no plan was declared.
t/Plack-Loader/shotgun.t ....................... All 102 subtests passed
t/Plack-Middleware/access_log.t ................ ok
t/Plack-Middleware/access_log_timed.t .......... ok
t/Plack-Middleware/access_log_value_zero.t ..... ok
t/Plack-Middleware/auth_basic.t ................ ok
t/Plack-Middleware/auth_basic_env.t ............ ok
t/Plack-Middleware/auth_basic_simple.t ......... skipped: Test requires module 'Authen::Simple::Passwd' but it's not found
t/Plack-Middleware/bufferedstreaming.t ......... ok
t/Plack-Middleware/cgibin.t .................... skipped: Test requires module 'CGI::Emulate::PSGI' but it's not found
t/Plack-Middleware/cgibin_exec.t ............... skipped: Test requires module 'CGI::Emulate::PSGI' but it's not found
t/Plack-Middleware/chunked.t ................... skipped: Test requires module 'IO::Handle::Util' but it's not found
t/Plack-Middleware/component-leak.t ............ ok
t/Plack-Middleware/component.t ................. skipped: Test requires module 'IO::Handle::Util' but it's not found
t/Plack-Middleware/conditional.t ............... ok
t/Plack-Middleware/conditionalget.t ............ ok
t/Plack-Middleware/conditionalget_writer.t ..... ok
t/Plack-Middleware/content_length.t ............ ok
t/Plack-Middleware/directory.t ................. ok
t/Plack-Middleware/error_document.t ............ ok
t/Plack-Middleware/file.t ...................... ok
t/Plack-Middleware/head.t ...................... ok
t/Plack-Middleware/httpexceptions.t ............ ok
t/Plack-Middleware/httpexceptions_streaming.t .. ok
t/Plack-Middleware/iis6_script_name_fix.t ...... ok
t/Plack-Middleware/jsonp.t ..................... ok
t/Plack-Middleware/lint.t ...................... ok
t/Plack-Middleware/lint_utf8_false_alarm.t ..... ok
t/Plack-Middleware/log4perl.t .................. ok
t/Plack-Middleware/log_dispatch.t .............. ok
t/Plack-Middleware/order.t ..................... ok
t/Plack-Middleware/prefix.t .................... ok
t/Plack-Middleware/psgibin.t ................... ok
t/Plack-Middleware/rearrange_headers.t ......... ok
t/Plack-Middleware/refresh-init.t .............. ok
t/Plack-Middleware/runtime.t ................... ok
t/Plack-Middleware/simple_content_filter.t ..... ok
t/Plack-Middleware/simple_logger.t ............. ok
t/Plack-Middleware/static.t .................... ok
t/Plack-Middleware/static_env.t ................ ok
t/Plack-Middleware/urlmap.t .................... ok
t/Plack-Middleware/urlmap_builder.t ............ ok
t/Plack-Middleware/urlmap_env.t ................ ok
t/Plack-Middleware/urlmap_ports.t .............. ok
t/Plack-Middleware/wrapcgi.t ................... skipped: Test requires module 'CGI::Emulate::PSGI' but it's not found
t/Plack-Middleware/wrapcgi_exec.t .............. skipped: Test requires module 'CGI::Emulate::PSGI' but it's not found
t/Plack-Middleware/xframework.t ................ ok
t/Plack-Middleware/xsendfile.t ................. ok
t/Plack-MIME/add_type.t ........................ ok
t/Plack-MIME/basic.t ........................... ok
t/Plack-MIME/fallback.t ........................ skipped: Test requires module 'MIME::Types' but it's not found
t/Plack-Request/base.t ......................... ok
t/Plack-Request/body.t ......................... ok
t/Plack-Request/content-on-get.t ............... ok
t/Plack-Request/content.t ...................... ok
t/Plack-Request/cookie.t ....................... ok
t/Plack-Request/double_port.t .................. ok
t/Plack-Request/hostname.t ..................... ok
t/Plack-Request/many_upload.t .................. ok
t/Plack-Request/multi_read.t ................... ok
t/Plack-Request/new.t .......................... ok
t/Plack-Request/parameters.t ................... ok
t/Plack-Request/params.t ....................... ok
t/Plack-Request/path_info.t .................... ok
t/Plack-Request/path_info_escaped.t ............ ok
t/Plack-Request/readbody.t ..................... ok
t/Plack-Request/request_uri.t .................. ok
t/Plack-Request/upload-basename.t .............. ok
t/Plack-Request/upload-large.t ................. All 6 subtests passed
t/Plack-Request/upload.t ....................... ok
t/Plack-Request/uri.t .......................... ok
t/Plack-Request/uri_utf8.t ..................... ok
t/Plack-Response/body.t ........................ ok
t/Plack-Response/compatible.t .................. ok
t/Plack-Response/cookie.t ...................... ok
t/Plack-Response/new.t ......................... ok
t/Plack-Response/redirect.t .................... ok
t/Plack-Response/response.t .................... ok
t/Plack-Runner/options.t ....................... ok
t/Plack-Server/compat.t ........................ skipped: Test requires module 'HTTP::Request::AsCGI' but it's not found
t/Plack-TempBuffer/print.t ..................... ok
t/Plack-Test/2args.t ........................... ok
t/Plack-Test/cookie.t .......................... ok
t/Plack-Test/hello.t ........................... ok
t/Plack-Test/hello_server.t .................... ok
t/Plack-Test/suite.t ........................... ok
t/Plack-Util/foreach.t ......................... ok
t/Plack-Util/headers.t ......................... ok
t/Plack-Util/headers_obj.t ..................... ok
t/Plack-Util/inline_object.t ................... ok
t/Plack-Util/io_with_path.t .................... ok
t/Plack-Util/is_real_fh.t ...................... ok
t/Plack-Util/load.t ............................ ok
t/Plack-Util/response_cb.t ..................... ok

Test Summary Report
-------------------
t/Plack-Handler/standalone.t                 (Wstat: 0 Tests: 102 Failed: 0)
  Parse errors: If TAP version is present it must be the first line of output
t/Plack-Loader/shotgun.t                     (Wstat: 0 Tests: 102 Failed: 0)
  Parse errors: If TAP version is present it must be the first line of output
t/Plack-Request/upload-large.t               (Wstat: 0 Tests: 6 Failed: 0)
  Parse errors: If TAP version is present it must be the first line of output
Files=114, Tests=764, 23 wallclock secs ( 0.54 usr  0.27 sys + 15.68 cusr  1.69 csys = 18.18 CPU)
Result: FAIL
Failed 3/114 test programs. 0/764 subtests failed.
make: *** [test_dynamic] Error 255
FAIL
Testing Plack-0.9986 failed.
@celogeek
celogeek commented Apr 3, 2012

Plack depends on Test::SharedFork which make bad stuff about fork. So it failed.
I'm working since the hackathon with Schwern on it here :

https://github.com/schwern/test-sharedfork

I will purpose a patch as soon as possible.

The only part missing actually is subtest.

@celogeek

This is good for subtest now, but they is a new bug I think.


PERL_DL_NONLAZY=1 /Users/geistteufel/perl5/perlbrew/perls/perl-tb2/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'inc', 'blib/lib', 'blib/arch')" t/*.t t/*/*.t t/*/*/*.t
t/00_compile.t ........... 1/? # Test::Builder::VERSION: 1.005000004
t/00_compile.t ........... ok   
t/01_simple.t ............ ok    
t/02_fork_method.t ....... ok    
t/03_toomany_run.t ....... ok    
t/04_test_before_fork.t .. ok   
t/05_nest.t .............. ok   
t/06_fail_lineno.t ....... ok   
t/07_lazy_load.t ......... ok   
t/08_threads.t ........... skipped: Not implemented yet
t/09_very_simple.t ....... ok   
t/10_subtest.t ........... ok   
t/11_failure_location.t .. 2/? Saw a test_start, but testing has already started at /Users/geistteufel/perl5/perlbrew/perls/perl-tb2/lib/5.14.2/TB2/EventHandler.pm line 85
# 1 test of 2 failed.
t/11_failure_location.t .. Dubious, test returned 25 (wstat 6400, 0x1900)
Failed 1/2 subtests 
t/store/00_simple.t ...... ok   
t/store/01_tie_scalar.t .. ok   
t/store/02_tie_array.t ... ok   

Test Summary Report
-------------------
t/11_failure_location.t (Wstat: 6400 Tests: 1 Failed: 0)
  Non-zero exit status: 25
  Parse errors: Tests out of sequence.  Found (2) but expected (1)
                Bad plan.  You planned 2 tests but ran 1.
Files=15, Tests=135,  4 wallclock secs ( 0.16 usr  0.05 sys +  3.51 cusr  0.47 csys =  4.19 CPU)
Result: FAIL
Failed 1/15 test programs. 0/135 subtests failed.
make: *** [test_dynamic] Error 255

The var $testing isn't keep between each test. I try to findout the issue.

@celogeek

actually test_test reset the $testing value. Then we start another test, and

    $state->post_event( TB2::Event::TestStart->new );

is fired again.

so this kind of test doesn't work anymore :

test_out 'not ok 1';
test_fail +3;
test_err "#          got: '0'";
test_err "#     expected: '1'";
is 0, 1;
test_test 'Failure locations should be correct';


test_out 'not ok 1';
test_fail +3;
test_err "#          got: '0'";
test_err "#     expected: '1'";
is 0, 1;
test_test 'Failure locations should be correct';
@celogeek

the first is ok, then we do another, and it crash.

@celogeek

we saw the issue if I try Plack with the patched test-sharedfork :


t/Plack-Handler/standalone.t ................... 1/? Saw a test_start, but testing has already started at /Users/geistteufel/perl5/perlbrew/perls/perl-tb2/lib/5.14.2/TB2/EventHandler.pm line 85

@schwern
Member
schwern commented Apr 13, 2012

Yes, our fix to Test::SharedFork must consider which EventCoordinator it is storing the history for using the $ec->object_id. This likely necessitates using a real Store object to store the different history files.

@schwern
Member
schwern commented Apr 14, 2013

Relies on #64

@schwernbot schwernbot referenced this issue in Test-More/TB2 Mar 16, 2014
Open

Plack test failure on Test-Builder2 #46

@schwern schwern closed this Mar 16, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment