Classify WP Codebox failures and preserve test result artifact

[chubes4]

Summary

While validating Extra-Chill/data-machine#2173, the full Composer test path reached WP Codebox test execution and exited with an unclassified WP Codebox failure. The output referenced a .pg-test-result.txt structured log path, but that artifact was not present afterward, leaving no clear underlying failure details to inspect.

Motivating evidence: Extra-Chill/data-machine#2173

Command / Context

Repository/worktree:

/Users/chubes/Developer/data-machine@fix-issue-2172-outcome-classes

Command run from that worktree:

composer test

That maps through composer.json to:

homeboy test data-machine

The runner resolved the WP Codebox backend and began running PHPUnit tests for the data-machine plugin.

Observed Behavior

The command reached WP Codebox test execution and failed with an unclassified exit:

Running PHPUnit tests via WP Codebox...
  Plugin: data-machine (/Users/chubes/Developer/data-machine@fix-issue-2172-outcome-classes)
  Backend: wp-codebox (WordPress Playground runtime)

============================================
UNCLASSIFIED WP CODEBOX FAILURE (exit=1)
============================================

--- Structured log (/Users/chubes/Developer/data-machine@fix-issue-2172-outcome-classes/.pg-test-result.txt) ---
STAGE_BEGIN:boot
STAGE_OK:boot
STAGE_BEGIN:install
STAGE_BEGIN:load_deps
STAGE_OK:load_deps
STAGE_BEGIN:load_component
PLUGIN_DETECTED data-machine.php
PLUGIN_LOAD_CONTEXT data-machine.php activate=false stage=load_component hook=muplugins_loaded wp_installing=true
STAGE_OK:load_component
STAGE_OK:install
STAGE_BEGIN:activation
PLUGIN_ACTIVATE agents-api/agents-api.php
PLUGIN_ACTIVATE_BEGIN agents-api/agents-api.php stage=activation hook=none wp_installing=true
PLUGIN_ACTIVATE_OK agents-api/agents-api.php stage=activation hook=none wp_installing=true
PLUGIN_ACTIVATE data-machine/data-machine.php
PLUGIN_ACTIVATE_BEGIN data-machine/data-machine.php stage=activation hook=none wp_installing=true
PLUGIN_ACTIVATE_OK data-machine/data-machine.php stage=activation hook=none wp_installing=true
STAGE_OK:activation
NOTICE:reopened WordPress action after install: wp_abilities_api_categories_init
NOTICE:reopened WordPress action after install: wp_abilities_api_init
STAGE_BEGIN:load_fixtures
STAGE_OK:load_fixtures
STAGE_BEGIN:discover_tests
NOTICE:phpunit.xml.dist not readable at /wordpress/wp-content/plugins/data-machine/phpunit.xml.dist; using defaults
DISCOVERY: dirs=/wordpress/wp-content/plugins/data-machine/tests suffixes=Test.php prefixes=test- excludes=0 found=100
STAGE_OK:discover_tests
STAGE_BEGIN:load_tests
STAGE_OK:load_tests
STAGE_BEGIN:run_tests
RUNNING 100 TEST FILES

============================================
BUILD FAILED: WP Codebox exited with code 1 (unclassified)
============================================

The structured result JSON from Homeboy reported:

failure.category=findings
failure.phase=test
failure.summary=test phase reported failures (exit 1)
phase.summary=test phase failed without structured counts (exit 1)
hints included: No tests ran — the runner failed before producing results. See raw_output.stderr_tail / raw_output.stdout_tail for the underlying error...

After the command completed, .pg-test-result.txt was not present in the Data Machine worktree. A direct read failed with file not found, and a glob for **/.pg-test-result.txt under the worktree found no files.

Expected Behavior

When WP Codebox reaches test execution and exits non-zero, the caller should get at least one of:

• A classified failure category/reason that points to the underlying failure mode.
• A preserved .pg-test-result.txt artifact at the path shown in output.
• A clear explanation that the artifact could not be produced or was intentionally cleaned up, including where equivalent raw logs can be found.

The output should not simultaneously point operators to .pg-test-result.txt and leave no artifact behind.

Why This Matters

The focused smoke tests for Extra-Chill/data-machine#2173 passed, but the full composer test path could not provide actionable diagnostics. That makes it hard to tell whether the failure is a plugin regression, a runner issue, or a harness/artifact-retention problem.

Notes

This issue is filed in the canonical WP Codebox repo (chubes4/wp-codebox), not the older sandbox-runtime handle/name.