Skip to content

feat(testing): add ExpectTestStatus, ExpectAssertionsCount, ExpectTestResultAttribute (#36) - #264

Open
rossaddison wants to merge 7 commits into
php-testo:1.xfrom
rossaddison:feat/36-expect-test-attributes
Open

feat(testing): add ExpectTestStatus, ExpectAssertionsCount, ExpectTestResultAttribute (#36)#264
rossaddison wants to merge 7 commits into
php-testo:1.xfrom
rossaddison:feat/36-expect-test-attributes

Conversation

@rossaddison

@rossaddison rossaddison commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes #36.

Adds three PHP attributes that let a test method assert properties of the inner stub TestResult returned by TestRunner::runTest(), without writing explicit assertion code in the test body:

  • #[ExpectTestStatus(Status::X)] — asserts stubResult->status
  • #[ExpectAssertionsCount(N)] — asserts stubResult->summary->metric('assertions')
  • #[ExpectTestResultAttribute('key')] — asserts stubResult->getAttribute('key') !== null (repeatable)

How it works

ExpectInterceptor (registered automatically by InjectPlugin) runs at ORDER_CLOSE_TO_TEST - 1. It reads the expect attributes from $info->testDefinition->reflection, calls $next() to execute the outer test, extracts $outerResult->result as the stub TestResult, then validates each declared expectation. Any mismatches are combined into a single Status::Failed with one RuntimeException. Pre-existing outer failures (or non-terminal statuses) are passed through untouched.

Files added

File Purpose
core/Testing/Attribute/ExpectTestStatus.php Attribute class
core/Testing/Attribute/ExpectAssertionsCount.php Attribute class
core/Testing/Attribute/ExpectTestResultAttribute.php Repeatable attribute class
core/Testing/Internal/ExpectInterceptor.php Interceptor implementation

Files modified

File Change
core/Testing/InjectPlugin.php Registers ExpectInterceptor

Test plan

  • 11 unit tests in tests/Core/Testing/Unit/ExpectInterceptorTest.php — all pass
  • Psalm: new files 100% clean (2 pre-existing errors in ChannelRenderer.php unrelated)
  • Full Core/Testing/Unit suite: 15/15 passed

🤖 Generated with Claude Code

@rossaddison
rossaddison requested a review from a team as a code owner July 6, 2026 14:45
@codecov

codecov Bot commented Jul 6, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

rossaddison and others added 6 commits August 13, 2026 17:11
@rossaddison @claude
feat(testing): add ExpectTestStatus, ExpectAssertionsCount, ExpectTes…
defe6ae 
…tResultAttribute (php-testo#36)

Three PHP attributes allow a test that uses TestRunner::runTest() to
assert properties of the inner stub TestResult without writing explicit
assertion code:

- #[ExpectTestStatus(Status::X)]   — validates stub.status
- #[ExpectAssertionsCount(N)]      — validates stub.summary.metric('assertions')
- #[ExpectTestResultAttribute(K)]  — validates stub.getAttribute(K) is not null (repeatable)

ExpectInterceptor reads these from the test method's reflection, runs the
test, extracts outerResult->result as the stub TestResult, and converts any
mismatches into a single Status::Failed with a combined message. Pre-existing
outer failures are preserved untouched. Registered automatically via InjectPlugin.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@rossaddison @claude
fix(output): widen Style::dim to string; fix ChannelRenderer::formatT…
49386b3 
…ime return type

Style::dim() worked correctly with any string (empty or not) but declared
@PARAM non-empty-string, which Psalm flagged at every call site where a plain
string was passed. Removed the over-restrictive annotation.

ChannelRenderer::formatTime() claimed @return non-empty-string with a
/** @var non-empty-string */ inline cast, which Psalm 7 does not accept.
Replaced date() with integer arithmetic + sprintf so the implementation is
cleaner, and removed the annotation since Psalm 7 does not narrow sprintf
to non-empty-string for this version.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@github-actions @rossaddison
style(cs): apply php-cs-fixer
37373cf
@rossaddison @claude
fix(phpunit-mirror): add .placeholder.php so EmptyRun stub directory …
22ea9a2 
…is mirrored

tests/Application/Stub/EmptyRun/ is intentionally empty — it is the test
fixture for EmptyRunTest, which asserts that a Testo run over an empty
directory yields Status::Risky with zero tests collected.

Git does not track empty directories, and bin/build-phpunit.php only copies
*.php files when populating the tests/PhpUnit/ mirror, so the mirror never
contained tests/PhpUnit/Application/Stub/EmptyRun/. The mirrored EmptyRunTest
resolved __DIR__ . '/../../Stub/EmptyRun' to that missing path and threw
InvalidArgumentException: File or directory not found — aborting Infection's
initial PHPUnit test run on every CI push to 1.x.

Add .placeholder.php (no namespace, no classes, no tests) to the source
directory. The build script copies it verbatim into the mirror, which creates
the required directory. Testo's FinderConfig still discovers zero tests there,
so Status::Risky is reported and the assertion holds.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@rossaddison
test(codecov): attribute Expect* constructors to ExpectInterceptorTes…
aa3ffbf 
…t coverage

ExpectAssertionsCount, ExpectTestStatus, and ExpectTestResultAttribute each had
0% coverage per Codecov, despite ExpectInterceptorTest exercising every
constructor via newInstance(). Testo's codecov plugin scopes coverage per test
to the classes named in #[Covers(...)] on that test, so lines executed by a
test are only credited to files the test explicitly declares — and this test
only declared #[Covers(ExpectInterceptor::class)].

Add #[Covers(...)] for the three attribute classes so their already-exercised
constructors are credited.
@rossaddison @claude
test: fix createTestInfoFor() for the rebased CaseDefinition/CaseInfo…
1dc4f3d 
… constructors

1.x moved 25 commits since this PR was opened, including required-argument
additions to CaseDefinition::$file and CaseInfo::$suiteIdentity. Updates
the test helper to pass both, matching the pattern used elsewhere in the
suite (Path::create(__FILE__), a real SuiteIdentity).

Verified after rebase:
- composer rector:ci: clean, 0 files
- Full-project Psalm (--no-cache): clean, exit 0
- Full Testo suite: 1682 passed, 6 failed/7 error (same pre-existing
  Bench/Self baseline as the current rector/* PR series, unrelated)
- ExpectInterceptorTest: 11/11 passed

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@rossaddison
rossaddison force-pushed the feat/36-expect-test-attributes branch from 1a5556e to 1dc4f3d Compare August 13, 2026 16:20
rossaddison added a commit to rossaddison/testo that referenced this pull request Aug 13, 2026
@rossaddison @claude
fix(error-handler): make ErrorHandlerInterceptor fiber-safe; promote …
e8a7008 
…CapturedErrors to public

Addresses roxblnfk's review on php-testo#262:

- set_error_handler()/restore_error_handler() operate on one process-global
  stack. The old code installed its handler once before $next() and
  restored once after — but $next() can suspend a fiber mid-test while a
  sibling test interleaves, so the handler stayed installed (and, on an
  interleaved resume, restore_error_handler() could pop a sibling's frame
  instead of its own). Same defect as php-testo#254.

  Fixed by wrapping $next() in its own fiber and swapping the handler on
  every suspend/resume — restore (native stack pop) on suspend, reinstall
  on resume — mirroring the already-reviewed pattern in
  MockeryInterceptor::run() and MessengerHub::scope().

  Regression test added (restoresTheOuterHandlerWhileSuspendedAndReinstallsItsOwnOnResume):
  confirmed it fails against the old code (an error fired while suspended
  was wrongly captured by this test's own handler instead of reaching the
  outer one) and passes against the fix.

- Promoted Internal\CapturedErrors to a public Testo\ErrorHandler\CapturedErrors
  class (Copilot review comment): the plugin's own docs already tell
  consumers to read this attribute off TestResult, so it was never really
  internal — it just wasn't marked as such. Fixes the @api-marked
  ErrorHandlerPlugin's docblock referencing an internal type.

- The other Copilot comment (restrict the failOnError status upgrade to
  Status::Passed) was already fixed in a prior commit on this branch — no
  change needed.

Also rebased onto current 1.x (26 commits behind), resolving one real
conflict in composer.json (version bumps landed upstream since this PR
opened) and the same CaseDefinition/CaseInfo required-argument fix already
applied on php-testo#264.

The second question from review — what should happen if a test changes
the error handler itself mid-run — is intentionally left open; per
roxblnfk's own comment it needs research/discussion before implementing,
not a quick fix.

Verified:
- composer rector:ci: clean, 0 files
- Full Testo suite: 1682 passed, 6 failed/7 error (same pre-existing
  Bench/Self baseline as the current rector/* PR series, unrelated)
- ErrorHandlerInterceptorTest: 11/11 passed, including the new
  fiber-safety regression test (confirmed it fails against the old code)
- Psalm: this repo's Psalm CI only covers core/ (confirmed via psalm.xml
  and psalm.yml's trigger paths) — plugin/error-handler/ was never in
  scope, unchanged by this fix

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Testing package

1 participant

@rossaddison