Test faster (#7574)

* Reduce dependencies: rm App::prove

* Reduce dependencies: rm File::Util

* Depedency cleanup: rm Test::Harness (with explicit version)

* Install modules with cpm instead of cpanm

* Simplified Webpack GitHub action setup

* Use 'yarn' to speed up dependency checks

* Reduce number of setup steps for clarity

* Adjust JS build caching criteria

* Move checkout *before* run init

* Start splitting run over multiple jobs

* Split build up in more (smaller) parts

* Split database testing into its own job

* Split off webservices testing

* Split out perl tests not dependent on database and/or browser

* Running outside of UI/, the dependency is missing -- npx restores it

* Upload test coverage after pure perl tests

* Name each Coveralls upload uniquely

* Stop pure perl coverage from failing

* Invoke 'jest' through npx instead of directly

* Make yarn install dev dependencies for redocly, openapi-merge-cli and jest-environment-jsdom

* Remove package-lock references; collect test results in a single step

* Use yarn to run tests instead of npx

* Don't finish parallel tests if some tests fail

* Fix node_modules path in CircleCI config

* Try coaxing coverage data out of starman

* Kill starman on the side of UI tests too; Try killing with -term to see if that saves cover data...

* Move Perl Setup to composite workflow

* Wait for cover to produce db

* Based on Yves's work, wait longer after stopping Starman

* Rename coverage set to where it really comes from

* Properly set JS Tests coverage

---------

Co-authored-by: Yves Lavoie <ylavoie@yveslavoie.com>

.circleci/config.yml

@@ -135,20 +135,21 @@ commands:
           keys:
             # Find a cache corresponding to this specific package-lock.json checksum
             # when this file is changed, this key will fail
-            - v1-npm-deps-{{ checksum "UI/package-lock.json" }}
+            - v1-npm-deps-{{ checksum "UI/yarn.lock" }}
             # Find the most recently generated cache used from any branch
             - v1-npm-deps-
 
       - run:
           name: Make JS + README
           command: |
+            sudo npm install -g yarn
             make js
             make readme
 
       - save_cache:
-          key: v1-npm-deps-{{ checksum "UI/package-lock.json" }}
+          key: v1-npm-deps-{{ checksum "UI/yarn.lock" }}
           paths:
-            - $HOME/.npm
+            - UI/node_modules
 
       - run:
           name: Set up dirs and files

.github/workflows/Install TEX & Perl/action.yml

@@ -0,0 +1,63 @@
+name: Install TEX & Perl
+description: "Install TEX & packages, Perl"
+
+inputs:
+  perl-version:
+    description: "Required perl version"
+    required: true
+  DEVEL_COVER_OPTIONS:
+    description: "Devel::Cover options base"
+    required: true
+  coverage:
+    description: "Coverage indicator"
+    default: ''
+
+runs:
+  using: "composite"
+  steps:
+    - name: Install TinyTex
+      uses: r-lib/actions/setup-tinytex@v2
+
+    - name: Setup Perl environment
+      uses: shogo82148/actions-setup-perl@v1
+      with:
+        perl-version: ${{ inputs.perl-version }}
+        install-modules-with: cpm
+        install-modules: |
+          Devel::Cover
+          Devel::Cover::Report::Coveralls
+        # Features and Devel modules
+        install-modules-args: >
+          --notest
+          --resolver=metacpan
+          --with-develop
+          --feature=starman
+          --feature=latex-pdf-ps
+          --feature=openoffice
+          --feature=xls
+          --feature=edi
+
+    - name: Pre-run installation steps
+      shell: bash
+      run: |
+        sudo apt -q -y install gettext
+
+        # Install missing TeX packages
+        tlmgr update --self
+        tlmgr install koma-script
+
+    - name: Setup coverage
+      shell: bash
+      run: |
+        echo "PERL5OPT=-MDevel::Cover=${{ inputs.DEVEL_COVER_OPTIONS }} -MSyntax::Keyword::Try::Deparse" >> $GITHUB_ENV
+        echo "YATH_DEVEL_COVER_OPTIONS=--cover=${{ inputs.DEVEL_COVER_OPTIONS }}" >> $GITHUB_ENV
+        echo "JSCOVERAGE=--coverage" >> $GITHUB_ENV
+      if: ${{ inputs.coverage }}
+
+    - name: Starting 'starman'
+      shell: bash
+      run: |
+        mkdir -p logs
+        starman --preload-app --pid starman.pid --workers $JOB_COUNT \
+                --max-requests 50000 --error-log logs/starman-error.log \
+                -Ilib -Iold/lib --port 5762 bin/ledgersmb-server.psgi &