WIP: Add parallel test execution to jcpan (-j flag)

dev/modules/datetime_format_ical.md

@@ -0,0 +1,89 @@
+# DateTime::Format::ICal — Dependency Chain Fixes
+
+## Status: WORKING (all 134 tests pass)
+
+```bash
+./jcpan -j 4 -t DateTime::Format::ICal   # All 5 test files pass (134/134 subtests)
+```
+
+## Dependency Chain
+
+```
+DateTime::Format::ICal  (Module::Build)
+├── DateTime::Set       (Module::Build, pure Perl)
+│   ├── Params::Validate  (Module::Build, XS + PP)
+│   │   ├── Devel::Peek     [test_requires] ← NEW STUB
+│   │   ├── Module::Implementation [requires]
+│   │   └── (XS compilation)  ← FIXED: auto-pureperl
+│   └── DateTime           (already working via jcpan)
+├── Params::Validate     (see above)
+└── DateTime             (already working)
+```
+
+## Issues Fixed (this branch)
+
+### 1. Parallel test execution for jcpan (`-j N` flag)
+**Files:** `jcpan`, `jcpan.bat`, `TAP/Parser/Iterator/Process.pm`
+
+`jcpan -t` ran CPAN module tests sequentially.  Added `-j N` flag that sets
+`HARNESS_OPTIONS=jN`, which flows through `Test::Harness` →
+`TAP::Harness(jobs=N)` → `_aggregate_parallel()`.
+
+The parallel path uses `IO::Select` via `TAP::Parser::Multiplexer`, but
+PerlOnJava's `TAP::Parser::Iterator::Process` fell back to pipe-opens
+without registering handles for `IO::Select` (because `d_fork` is not set).
+Fixed by trying `IO::Select` on pipe handles in the non-fork fallback path.
+
+**Benchmark (Path::Tiny, 30 test files):**
+- Sequential: 36s → Parallel `-j 4`: 18.5s (~2x speedup)
+
+### 2. Module::Build `./Build` chained shebang fix
+**File:** `CPAN/Distribution.pm` (`_build_command()` + install path)
+
+`jperl` is a bash wrapper script.  Unix kernels don't support chained
+shebangs (`#!/path/to/jperl` where jperl has `#!/bin/bash`), so
+`./Build test` was executed by bash instead of jperl.
+
+Fixed `_build_command()` to return `"$^X Build"` when `archname` contains
+`java`.  Also fixed the install path which bypasses `_build_command()` and
+uses `$CPAN::Config->{mbuild_install_build_command}` (hardcoded to
+`./Build`).
+
+### 3. Devel::Peek stub
+**File:** `src/main/perl/lib/Devel/Peek.pm`
+
+Params::Validate lists `Devel::Peek` as `test_requires`.  CPAN treats this
+as a hard dependency.  Since `Devel::Peek` is a core XS module (can't be
+installed separately), the dependency chain was blocked.
+
+Created a stub that provides `SvREFCNT()` (returns 1 — JVM uses tracing GC,
+not refcounting) and `Dump()` (prints a placeholder).
+
+### 4. Module::Build auto-pureperl for XS modules
+**File:** `src/main/perl/lib/Module/Build/Base.pm`
+
+Params::Validate sets `allow_pureperl => 1` in Build.PL, but Module::Build
+only skips XS when **both** `pureperl_only` AND `allow_pureperl` are true.
+Since `pureperl_only` defaults to 0, XS compilation was always attempted
+and died ("no compiler detected").
+
+Fixed by overriding `process_xs_files` to auto-set `pureperl_only` when
+the module declares `allow_pureperl` and no C compiler is available.
+
+## Remaining Issues
+
+### Params::Validate test failures (12/38 programs fail)
+Not blocking — module installs and works with `jcpan -f -i`.  Failures are:
+- **Glob type detection**: `*FH` detected as `scalar` instead of `glob`
+- **Taint mode**: PerlOnJava doesn't fully support taint checking
+- **Callbacks**: Error message format differences
+- **Case sensitivity**: Some case-related validation mismatches
+
+These are PerlOnJava runtime issues, not dependency/build issues.
+
+### Modules that could benefit from this work
+Any Module::Build module with `allow_pureperl => 1` should now build
+correctly on PerlOnJava.  Examples from CPAN:
+- `Params::Validate` ✅ (tested)
+- `Class::XSAccessor` (has PP fallback)
+- `Package::Stash` (has PP fallback via Package::Stash::PP)

jcpan

@@ -3,8 +3,35 @@
 # jcpan - CPAN Client for PerlOnJava (Unix wrapper)
 # Runs the standard cpan script with jperl
 #
+# Supports -j N for parallel test execution:
+#   jcpan -j 4 -t Module::Name
+#
 SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
 
+# Parse -j option for parallel test jobs (consumed here, not passed to cpan)
+ARGS=()
+while [[ $# -gt 0 ]]; do
+    case "$1" in
+        -j)
+            if [[ -n "$2" && "$2" =~ ^[0-9]+$ ]]; then
+                export HARNESS_OPTIONS="j${2}"
+                shift 2
+            else
+                echo "Error: -j requires a numeric argument" >&2
+                exit 1
+            fi
+            ;;
+        -j[0-9]*)
+            export HARNESS_OPTIONS="j${1#-j}"
+            shift
+            ;;
+        *)
+            ARGS+=("$1")
+            shift
+            ;;
+    esac
+done
+
 # Find cpan script - check development path first, then installed path
 if [ -f "$SCRIPT_DIR/src/main/perl/bin/cpan" ]; then
     CPAN_SCRIPT="$SCRIPT_DIR/src/main/perl/bin/cpan"
@@ -15,4 +42,4 @@ else
     exit 1
 fi
 
-exec "$SCRIPT_DIR/jperl" "$CPAN_SCRIPT" "$@"
+exec "$SCRIPT_DIR/jperl" "$CPAN_SCRIPT" "${ARGS[@]}"

jcpan.bat

@@ -1,5 +1,23 @@
 @echo off
 rem jcpan - CPAN Client for PerlOnJava (Windows wrapper)
 rem Runs the standard cpan script with jperl
+rem Supports -j N for parallel test execution:
+rem   jcpan -j 4 -t Module::Name
 set SCRIPT_DIR=%~dp0
-"%SCRIPT_DIR%jperl.bat" "%SCRIPT_DIR%src\main\perl\bin\cpan" %*
+
+rem Parse -j option for parallel test jobs
+set "JCPAN_ARGS="
+:parse_args
+if "%~1"=="" goto run
+if "%~1"=="-j" (
+    set "HARNESS_OPTIONS=j%~2"
+    shift
+    shift
+    goto parse_args
+)
+set "JCPAN_ARGS=%JCPAN_ARGS% %1"
+shift
+goto parse_args
+
+:run
+"%SCRIPT_DIR%jperl.bat" "%SCRIPT_DIR%src\main\perl\bin\cpan" %JCPAN_ARGS%

src/main/java/org/perlonjava/core/Configuration.java

@@ -33,7 +33,7 @@ public final class Configuration {
      * Automatically populated by Gradle/Maven during build.
      * DO NOT EDIT MANUALLY - this value is replaced at build time.
      */
-    public static final String gitCommitId = "69f34aac5";
+    public static final String gitCommitId = "1fbd174ea";
 
     /**
      * Git commit date of the build (ISO format: YYYY-MM-DD).

src/main/perl/lib/CPAN/Distribution.pm

@@ -4219,6 +4219,11 @@ sub install {
                 $CPAN::Config->{mbuild_install_build_command} ?
                     $CPAN::Config->{mbuild_install_build_command} :
                         $self->_build_command();
+        # On PerlOnJava, jperl is a bash wrapper script so chained shebangs
+        # don't work.  Replace bare "./Build" with "$^X Build".
+        if ($Config::Config{archname} =~ /\bjava\b/ && $mbuild_install_build_command eq './Build') {
+            $mbuild_install_build_command = "$^X Build";
+        }
         my $install_directive = $^O eq 'VMS' ? '"install"' : 'install';
         $system = sprintf("%s %s %s",
                           $mbuild_install_build_command,
@@ -4757,6 +4762,13 @@ sub _build_command {
     elsif ($^O eq 'VMS') {
         return "$^X Build.com";
     }
+    # When the perl interpreter is a wrapper script (e.g. jperl on
+    # PerlOnJava), chained shebangs don't work — the kernel runs
+    # /bin/bash on the Build script instead of interpreting it as Perl.
+    # Detect this via archname containing "java" and invoke explicitly.
+    elsif ($Config::Config{archname} =~ /\bjava\b/) {
+        return "$^X Build";
+    }
     return "./Build";
 }
 

src/main/perl/lib/Devel/Peek.pm

@@ -0,0 +1,81 @@
+package Devel::Peek;
+
+#      Devel/Peek.pm
+#
+#      Original Copyright (c) 1995-2000 Ilya Zakharevich
+#
+#      You may distribute under the terms of either the GNU General Public
+#      License or the Artistic License, as specified in the README file.
+#
+#      PerlOnJava stub implementation.
+#      PerlOnJava implementation by Flavio S. Glock.
+#
+# Minimal Devel::Peek for PerlOnJava.
+# The JVM uses tracing GC, not reference counting, so SV internals
+# are not directly accessible.  This stub provides enough for modules
+# like Params::Validate whose tests use SvREFCNT().
+
+use strict;
+use warnings;
+
+our $VERSION = '1.34';
+
+use Exporter 'import';
+our @EXPORT    = qw(Dump DumpArray DumpProg);
+our @EXPORT_OK = qw(Dump DumpArray DumpProg SvREFCNT DeadCode
+                     fill_mstats mstats_fillhash mstats2hash
+                     DumpWithOP);
+
+# SvREFCNT - return the reference count of a scalar.
+# JVM uses tracing GC, not reference counting.  Return 1 as a safe
+# default (the value is "alive" if we can see it).
+sub SvREFCNT { return 1 }
+
+# Dump - print internal representation of a Perl value.
+# Not meaningful on JVM; emit a short placeholder.
+sub Dump {
+    my ($sv, $lim) = @_;
+    $lim = 4 unless defined $lim;
+    my $ref  = ref($sv) || 'SCALAR';
+    my $val  = defined $sv ? (ref($sv) ? "$sv" : $sv) : 'undef';
+    print STDERR "SV = $ref\n  VALUE = $val\n  (Devel::Peek::Dump stub on PerlOnJava)\n";
+}
+
+sub DumpArray {
+    my ($lim, @vals) = @_;
+    for my $i (0 .. $#vals) {
+        print STDERR "Elt No. $i\n";
+        Dump($vals[$i], $lim);
+    }
+}
+
+sub DumpProg   { print STDERR "DumpProg not available on PerlOnJava\n" }
+sub DumpWithOP { print STDERR "DumpWithOP not available on PerlOnJava\n" }
+sub DeadCode   { return 0 }
+
+sub fill_mstats      { return }
+sub mstats_fillhash  { return }
+sub mstats2hash      { return }
+
+1;
+
+__END__
+
+=head1 NAME
+
+Devel::Peek - PerlOnJava stub for SV introspection
+
+=head1 DESCRIPTION
+
+This is a stub implementation of Devel::Peek for PerlOnJava.
+The JVM uses tracing garbage collection rather than reference counting,
+so SV internals are not directly accessible.
+
+C<SvREFCNT()> always returns 1 (the value is alive if reachable).
+C<Dump()> prints a short placeholder to STDERR.
+
+=head1 SEE ALSO
+
+L<B>, L<Devel::Peek> (full version on CPAN)
+
+=cut

src/main/perl/lib/Module/Build/Base.pm

@@ -48,6 +48,32 @@ if (!$loaded) {
     # This makes _backticks() use backticks instead of fork+pipe
     no warnings 'redefine';
     *have_forkpipe = sub { 0 };
+
+    # Auto-enable pureperl_only for modules that support it.
+    # PerlOnJava runs on JVM and cannot compile XS/C code.
+    # Module::Build's process_xs_files() only skips XS when BOTH
+    # pureperl_only AND allow_pureperl are true.  Since pureperl_only
+    # defaults to 0, modules like Params::Validate (allow_pureperl=1)
+    # still attempt XS compilation and die.  Fix: auto-set pureperl_only
+    # when the module declares allow_pureperl and no C compiler exists.
+    my $orig_process_xs_files = \&Module::Build::Base::process_xs_files;
+    *Module::Build::Base::process_xs_files = sub {
+        my $self = shift;
+        if ($self->allow_pureperl && !$self->have_c_compiler) {
+            $self->pureperl_only(1);
+            return;
+        }
+        return $self->$orig_process_xs_files(@_);
+    };
+
+    my $orig_process_support_files = \&Module::Build::Base::process_support_files;
+    *Module::Build::Base::process_support_files = sub {
+        my $self = shift;
+        if ($self->allow_pureperl && !$self->have_c_compiler) {
+            return;
+        }
+        return $self->$orig_process_support_files(@_);
+    };
 }
 
 1;

src/main/perl/lib/TAP/Parser/Iterator/Process.pm

@@ -174,6 +174,16 @@ sub _initialize {
           = join( ' ', $exec, map { $_ =~ /\s/ ? qq{"$_"} : $_ } @command );
         open( $out, "$command|" )
           or die "Could not execute ($command): $!";
+
+        # Try to use IO::Select on the pipe handle for parallel execution.
+        # On platforms without fork (e.g. PerlOnJava), open3 is unavailable
+        # but pipe opens may still produce selectable file handles.
+        eval {
+            require IO::Select;
+            if ( defined fileno($out) ) {
+                $sel = IO::Select->new($out);
+            }
+        };
     }
 
     $self->{out}        = $out;