harness: take a test pattern through %ENV

[mkende]

This is adding a HARNESS_RE environment variable used by harness to make it simpler to run specific tests of the core distribution without manually calling harness, which requires a more complex command line.

[jkeenan]

I don't see the need for this new functionality. We already have very similar functionality, and your patch failed for me in a surprising way.

The existing functionality -- the TEST_ARGS switch to make test_harness -- is already documented in pod/perlhack.pod:

Under the Unix build process you may use the TEST_ARGS and 
TEST_FILES parameters to pass arguments through to the 
underlying harness call. This means that for instance you could do

    make test_harness TEST_ARGS="-v -re pat"

which would make, and then run the test harness in verbose 
mode over files which contain "pat". ...

Configuring and building in blead (while also supplying TEST_JOBS=1 to facilitate copying from the terminal), I get:

$ TEST_JOBS=1 TEST_ARGS="-re porting" make test_harness
....
All tests successful.
Files=35, Tests=47577, 27 wallclock secs ( 0.71 usr  0.04 sys + 28.12 cusr  1.54 csys = 30.41 CPU)
Result: PASS
Finished test run at Thu Jan 25 17:53:56 2024.

Similarly configuring and building in your branch, I get:

$ TEST_JOBS=1 HARNESS_RE="porting" make test_harness
...
porting/test_bootstrap.t .. ok     
porting/test_testlist.t ... 1/3 # Failed test 2 - t/TEST will not test aything that t/harness does not at porting/test_testlist.t line 39
#      got "2856"
# expected "0"
# tTEST: cpan/Archive-Tar/t/01_use.t
# tTEST: cpan/Archive-Tar/t/02_methods.t
...
# tTEST: t/uni/variables.t
# tTEST: t/uni/write.t
porting/test_testlist.t ... Failed 1/3 subtests 
porting/update_authors.t .. ok    
porting/utils.t ........... ok    

Test Summary Report
-------------------
porting/test_testlist.t (Wstat: 0 Tests: 3 Failed: 1)
  Failed test:  2
Files=35, Tests=47576, 29 wallclock secs ( 0.72 usr  0.05 sys + 29.50 cusr  1.68 csys = 31.95 CPU)
Result: FAIL
Finished test run at Thu Jan 25 17:59:46 2024.
make: *** [makefile:866: test_harness] Error 1

Now, I haven't delved into t/porting/test_testlist.t to see why it's failing. I've never had to do that, because it has always passed for me up until now. So even if you fixed this, if we accepted your pull request, we would be duplicating functionality but also adding a little to our maintenance burden.

[mkende]

Thanks a lot for the detailed answer. I actually had missed the TEST_ARGS variable in the documentation (for some reason I saw only the HARNESS_OPTIONS one that supports only a subset of the options).

Anyway, this can indeed be rejected.

P.S. the porting failure is really bad luck in selecting the test to run, because it’s the one test that checks that test and harness run the same test files (and test did not know about the new variable obviously).