Display actual Python exceptions in parsl_test_runner CI logs

[Copilot]

The booktests_parallel_no_docker CI job shows generic "unix exit code 1" errors without Python tracebacks, making debugging impossible.

Changes

Combine stderr/stdout in bash_app (line 27)

• Added 2>&1 redirection to capture all error output in stdout log file

Display error logs on first failure (lines 98-109)

• Read and print last 2000 chars of stdout log before retry
• Error handling for file I/O exceptions

Display error logs on retry failure (lines 127-138)

• Read and print last 2000 chars of retry log before marking FAILED
• Error handling for file I/O exceptions

Example Output

Before:

Test tb_elliptic_pde failed with error: bash_app run_single_test failed with unix exit code 1

After:

--- Error details for tb_elliptic_pde (first attempt) ---
Traceback (most recent call last):
  File "tb_elliptic_pde.py", line 2, in <module>
    from testbook import testbook
ModuleNotFoundError: No module named 'testbook'
--- End of error details for tb_elliptic_pde ---

Original prompt

Problem

The booktests_parallel_no_docker CI job is failing with 8 test failures, but the logs only show generic "bash_app run_single_test failed with unix exit code 1" errors without revealing the actual Python exceptions or stack traces. This makes debugging impossible.

Failing tests:

• tb_elliptic_pde
• tb_vectorized_qmc
• tb_plot_proj_function
• tb_joss2025
• tb_vectorized_qmc_bayes
• tb_gbm_examples
• tb_linear_scrambled_halton
• tb_why_add_q_to_mc_blog

CI Run: https://github.com/QMCSoftware/QMCSoftware/actions/runs/20197244148/job/57983013219#step:15:163

Root Cause

The parsl_test_runner.py script redirects test output to log files but doesn't read and display those logs when tests fail. The error handling only shows the exception message from Parsl, not the underlying Python test errors.

Required Changes

Modify test/booktests/parsl_test_runner.py to:

1. Capture stderr and stdout together in the bash_app to ensure all error output is captured
2. Read and print error log contents when tests fail (both on first attempt and retry)
3. Display detailed failure information including the actual Python traceback/error messages to aid debugging

Specific Implementation

1. In the run_single_test bash_app (around line 24-28):

   • Combine stderr and stdout using 2>&1 redirection to ensure all output goes to the stdout file

2. In the exception handler for first attempt failures (around line 88-96):

   • After catching the exception, read the error/output log files
   • Print the contents (last 2000 characters to avoid log overflow) to show what actually failed

3. In the exception handler for retry failures (around line 109-119):

   • Similarly read and print the retry error log contents
   • Display a clear failure details section with the actual Python error

4. Ensure error details are printed before marking tests as FAILED so they appear in CI logs

This will make the actual test errors visible in GitHub Actions logs, allowing developers to see ImportErrors, ModuleNotFoundErrors, assertion failures, or any other Python exceptions that are causing the tests to fail.

Expected Outcome

After this fix, when tests fail in CI, the logs will show the actual Python exceptions and stack traces, making it immediately clear what needs to be fixed (e.g., missing dependencies, import errors, assertion failures, etc.).

This pull request was created as a result of the following prompt from Copilot chat.

Problem

The booktests_parallel_no_docker CI job is failing with 8 test failures, but the logs only show generic "bash_app run_single_test failed with unix exit code 1" errors without revealing the actual Python exceptions or stack traces. This makes debugging impossible.

Failing tests:

• tb_elliptic_pde
• tb_vectorized_qmc
• tb_plot_proj_function
• tb_joss2025
• tb_vectorized_qmc_bayes
• tb_gbm_examples
• tb_linear_scrambled_halton
• tb_why_add_q_to_mc_blog

CI Run: https://github.com/QMCSoftware/QMCSoftware/actions/runs/20197244148/job/57983013219#step:15:163

Root Cause

The parsl_test_runner.py script redirects test output to log files but doesn't read and display those logs when tests fail. The error handling only shows the exception message from Parsl, not the underlying Python test errors.

Required Changes

Modify test/booktests/parsl_test_runner.py to:

1. Capture stderr and stdout together in the bash_app to ensure all error output is captured
2. Read and print error log contents when tests fail (both on first attempt and retry)
3. Display detailed failure information including the actual Python traceback/error messages to aid debugging

Specific Implementation

1. In the run_single_test bash_app (around line 24-28):

   • Combine stderr and stdout using 2>&1 redirection to ensure all output goes to the stdout file

2. In the exception handler for first attempt failures (around line 88-96):

   • After catching the exception, read the error/output log files
   • Print the contents (last 2000 characters to avoid log overflow) to show what actually failed

3. In the exception handler for retry failures (around line 109-119):

   • Similarly read and print the retry error log contents
   • Display a clear failure details section with the actual Python error

4. Ensure error details are printed before marking tests as FAILED so they appear in CI logs

This will make the actual test errors visible in GitHub Actions logs, allowing developers to see ImportErrors, ModuleNotFoundErrors, assertion failures, or any other Python exceptions that are causing the tests to fail.

Expected Outcome

After this fix, when tests fail in CI, the logs will show the actual Python exceptions and stack traces, making it immediately clear what needs to be fixed (e.g., missing dependencies, import errors, assertion failures, etc.).

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.