Skip to content

Fix/pr 5554 rebased#5557

Merged
DavertMik merged 2 commits into
4.xfrom
fix/pr-5554-rebased
May 11, 2026
Merged

Fix/pr 5554 rebased#5557
DavertMik merged 2 commits into
4.xfrom
fix/pr-5554-rebased

Conversation

@DavertMik
Copy link
Copy Markdown
Contributor

@DavertMik DavertMik commented May 11, 2026

Reproduction project: https://github.com/gololdf1sh/codeceptjs-plugins-test


Issue 1 — aiTrace: step artifacts race condition

Steps to reproduce:

  1. Enable aiTrace plugin
  2. Run a test that performs multiple steps
  3. Check trace.md — artifacts (screenshots, HTML, ARIA) are missing or incomplete for some steps

Cause: persistStep() was called outside the recorder chain — the async capture ran in parallel with test execution, leading to race conditions.

Fix: Moved persistStep() inside recorder.add() so artifact capture is sequenced with the test flow.


Issue 2 — analyze: AI report silently skipped

Steps to reproduce:

  1. Enable analyze plugin, set ANTHROPIC_API_KEY
  2. Run tests with --ai flag where some tests fail
  3. Observe: no AI report is printed

Cause: Two issues: (a) printReport() is async but was called without await — the event handler returned immediately; (b) process.exit() in run.js fires on a 1-second timeout, killing the process before the AI API response arrives.

Fix: Added await to printReport(), set CODECEPT_DISABLE_AUTO_EXIT=1 before the report, and call process.exit() explicitly after the report finishes.


Issue 3 — screencast: recording starts too late (misses first steps)

Steps to reproduce:

  1. Enable screencast plugin with on: 'test'
  2. Run a test with navigation + interactions
  3. Check the recorded video — the first steps are missing

Cause: Recording was triggered on event.step.started — the first step's actions could execute before the screencast started.

Fix: Moved the recording start to event.test.started, which fires after helper._before() (page is available) but before any steps execute.


Issue 4 — pageInfo: plugin silently disabled (helper resolved too early)

Steps to reproduce:

  1. Enable pageInfo plugin
  2. Run a failing test
  3. Observe: no pageInfo artifact is generated, no error message

Cause: Two issues: (a) pickActingHelper() was called at plugin init time when Container.helpers() may not be populated yet — if it returned null, the plugin silently exited; (b) recorder.add() calls lacked force=true, so they were skipped when the recorder was in a rejected state (which it is after a test failure).

Fix: Moved helper resolution inside the event.test.failed handler (lazy), and added force=true to both recorder.add() calls.


Issue 5 + Issue 9 — aiTrace: failed step artifacts lost / event.step.failed async handler race

Steps to reproduce:

  1. Enable aiTrace plugin
  2. Run a test that fails on a specific step
  3. Check trace.md — the failed step has no screenshot, HTML, or ARIA snapshot

Cause: event.step.failed handler was async but Node.js EventEmitter doesn't await async handlers. The artifact capture promise was fire-and-forget — event.test.failed / persist() ran before capture finished.

Fix: Changed event.step.failed to a sync handler that stores the capture promise in pendingArtifactCapture. The event.test.failed handler awaits this promise (inside recorder.add with force=true) before persisting the trace.


Issue 6 — heal: steps filter in recipes ignored during code suggestions

Steps to reproduce:

  1. Define a heal recipe with steps: ['click'] filter
  2. Trigger healing on a fillField failure
  3. Observe: the recipe is used despite the step name not matching

Cause: getCodeSuggestions() in heal.js didn't filter recipes by their steps array — it matched only by context name.

Fix: Added .filter(r => !r.steps || !stepName || r.steps.includes(stepName)) after matchRecipes() in getCodeSuggestions().


Issue 7 — heal: healLimit not enforced correctly

Steps to reproduce:

  1. Set healLimit: 2 in heal plugin config
  2. Run a test where 4+ steps fail
  3. Observe: all steps are healed, limit is ignored

Cause: Two issues: (a) healTries was never reset between tests; (b) the limit check was outside recorder.catchWithoutStop — since catchWithoutStop registers a persistent handler, the outer check ran only once at registration time.

Fix: Reset healTries = 0 in event.test.before, and moved the limit check inside the catchWithoutStop callback so it's evaluated on every caught error.


Issue 10 — analyze: container typo (lowercase)

Steps to reproduce:

  1. Enable analyze plugin with --ai flag
  2. Run tests where some fail
  3. Observe: ReferenceError: container is not defined in console

Cause: Line 297 used container.plugins() (lowercase) instead of Container.plugins().

Fix: Changed container to Container.

gololdf1sh and others added 2 commits May 11, 2026 01:54
…encast, pageInfo, heal

Fix 8 bugs across 6 plugin files where async operations outside the
recorder chain, missing force flags, and incorrect filtering caused
silent data loss, premature process exit, and broken healing limits.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
analyze.js: CODECEPT_DISABLE_AUTO_EXIT was removed in #5556 (autoExit
refactor). Push printReport onto the recorder so it's awaited inside
codecept.run()'s done() before autoExit fires.

screencast tests: emit event.test.started in unit tests to match the
production event sequence (asyncWrapper.js fires it between
event.test.before and the first event.step.started).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@DavertMik DavertMik merged commit 8d4478c into 4.x May 11, 2026
11 of 12 checks passed
@DavertMik DavertMik deleted the fix/pr-5554-rebased branch May 11, 2026 20:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants