nixos/lib/test*: remove perl test driver

[flokli]

This has been deprecated in 20.03, and all tests have been migrated to
the python framework, effectively making this dead code.

Motivation for this change

#72828

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

[worldofpeace]

I think that's my only requested change. Everything else looks good.

[aszlig]

LGTM, however could we please address #72964 before merging this? Most of the tests in nixpkgs are rather simple, but as soon as you use Nix to parameterize tests you'll get random failures with black.

[flokli]

@aszlig I don't see how #72964 should block merging this in. I agree this would be nice, but I don't know if finding a linter ignoring line lengths is too hard, or if just noone has picked up the task yet. All of nixpkgs managed to somewhat work around these issues too :-)

This perl driver has been deprecated for a whole release now, and removing it from nixpkgs for the 20.09 release has been announced as well.

[khumba]

LGTM, however could we please address #72964 before merging this? Most of the tests in nixpkgs are rather simple, but as soon as you use Nix to parameterize tests you'll get random failures with black.

Or if you disable linting on your test suite wholesale, then an evaluation of all your packages produces a screenful of Linting is disabled! lines. I've created #96515 to hopefully start a discussion.

Before removing the Perl driver, I'm more concerned about #86889. It's pretty easy to hang the Python test driver with a test that produces a lot of output on stderr. In my experience it isn't even intermittently; if the test writes thousands of lines, it's going to hang. This is a regression compared to the Perl driver.

[flokli]

Before removing the Perl driver, I'm more concerned about #86889. It's pretty easy to hang the Python test driver with a test that produces a lot of output on stderr. In my experience it isn't even intermittently; if the test writes thousands of lines, it's going to hang. This is a regression compared to the Perl driver.

This has been fixed by #96254. I updated #86889.

[flokli]

Or if you disable linting on your test suite wholesale, then an evaluation of all your packages produces a screenful of Linting is disabled! lines. I've created #96515 to hopefully start a discussion.

For all of 20.03, if you still have been using the non-python test driver, there were 3 lines of warnings:

Perl VM tests are deprecated and will be removed for 20.09.
Please update your tests to use the python test driver.
See https://github.com/NixOS/nixpkgs/pull/71684 for details.

I don't see why getting it down to 1 line per test (after migrating to the python test and setting skipLint to true) is so bad we need to introduce another option in #96515.

[Ma27]

@aszlig when hacking on tests you probably want to use skipLint = true;, IMHO those linting "errors" become rather annoying otherwise :)

[dasJ]

@flokli For my private tests I simply use skipLint = true; but otherwise I'd have fixed that ;)

[flokli]

Let's talk about how to make the python linter less annoying in #72964.

Thanks for merging this PR - I don't really see how this is related to removing a piece of code unused in nixpkgs, and announced to be removed half a year ago.

[Ma27]

I don't really see how this is related to removing a piece of code unused in nixpkgs, and announced to be removed half a year ago.

Well, there are people who also use one of the test frameworks for their own stuff and dislike this "feature" ;-)