zef install readline fails with --tap-harness

[jaguart]

Issue

zef install Readline  
# fails - --verbose shows
# [Readline] t/02-base.t (Wstat: 0 Tests: 17 Failed: 0)
# [Readline]   Parse errors: Subtest 5 expected 3 but contains 2 tests

zef install Readline --/tap-harness # works

If I download the source, and comment out this line: https://github.com/lathropd/perl6-readline/blob/1e2f1f00c56c8cc1762b997e083ea460ea7ee1ae/t/02-base.t#L58 the tests pass.

The -verbose output from the test shows that terminal actually runs three tests, but then reports expected 3 but contains 2 tests

Environment:

• Cygwin MINTTY 3.6.1-1 terminal 'xterm' connecting via ssh to:
• Linux 5.10.0-20-amd64 #1 SMP Debian 5.10.158-2 (2022-12-13) x86_64 GNU/Linux
• TERM=xterm

Please let me know if you need any further information. I can also test if you don't have a similar environment.

:)

[JJ]

Thanks for the report, but I'm not totally sure it belongs to this repo. Sure, this is what fails, but wouldn't it be convenient to, at least, raise a similar issue in Readline?

[Leont]

TAP::Harness is a bit stricter on valid subtests than most other parsers. I suspect that the verbose output of the tests would reveal the real issue.

[jaguart]

I have this gist: https://gist.github.com/jaguart/8ace1402c02d85a9d0e5144704f30cd1

Which probably doesn't show what you want to actually see. When it hit's the subtest that is failing, you can see that there are 3 x ok messages. At the end though, it fails with:

# [Readline] t/02-base.t (Wstat: 0 Tests: 17 Failed: 0)
# [Readline]   Parse errors: Subtest 5 expected 3 but contains 2 tests

So at the end TAP think's it ran 2 tests, even though the terminal shows 3 ok's

Somehow the call to VT100 .prepterminal causes TAP to miss one of the OKs.

[jaguart]

ok, so this test emits control chars:

^[[?2004h    ok 1 - prep-terminal lives
^[[?2004l^M    ok 2 - deprep-terminal lives
    ok 3 - reset-terminal
    1..3
ok 5 - terminal

Interestingly it's the only test that does this.

[JJ]

Some PR coming from the non-usual computer what uses Windows, maybe. It's invisible in the review, so there you go... At any rate, again it's probably best if you raise the issue there, they'll be able to help you with that.

[jaguart]

Not sure what you mean? I guess the point is that the tests have all actually passed - but the TAP harness thinks something is screwy.

[JJ]

Not sure what you mean? I guess the point is that the tests have all actually passed - but the TAP harness thinks something is screwy.

Well, the tests are emitting characters that TAP can't parse. And that's something that can be fixed only in that distro, I'm afraid. As @Leont says, strict TAP parsing is something that can be done and that's the technical decision taken here; other test runners might be more lenient. Baseline is, it's not something we can fix here. You can either try and have Readline fix it, or else tell zef to opt for a different test runner, which IIRC is something that you can do.

[jaguart]

Ok thanks for explaining. :)