Skip to content

[APPS-2792] Add: local-execution resilience tests - #496

Draft
tyffical wants to merge 4 commits into
tiffany.trinh/apps-2792-dev-verify-clifrom
tiffany.trinh/apps-2792-local-execution-resilience-tests
Draft

[APPS-2792] Add: local-execution resilience tests#496
tyffical wants to merge 4 commits into
tiffany.trinh/apps-2792-dev-verify-clifrom
tiffany.trinh/apps-2792-local-execution-resilience-tests

Conversation

@tyffical

@tyffical tyffical commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Motivation

  • Local-execution resilience testing milestone from the Local Node Execution Kickoff doc.
    • Renamed/descoped from "Chaos-engineering resilience testing" since domains/chaos-engineering targets Datadog-owned Kubernetes clusters — a mismatch for this in-process, customer-dev-server model.
  • Two targeted checks, not fixes:
    • Empirically confirm the RFC's "no process isolation" decision has the failure modes it assumes, rather than leave them unverified.

Architecture

  • The process.exit() test can't call runScriptLocally directly in the main Jest process — exiting would kill the whole test run.
  • It spawns the fixture as a child process instead, and asserts on the child's exit behavior:
Jest test process
  └─ spawns ──▶ local-execution.process-exit.fixture.ts (child process)
                  runScriptLocally() → customer fn calls process.exit()
                  → child terminates immediately
  └─ asserts on the child's exit code

Changes

4 changes across 5 files
What changed File
New test confirming a synchronous CPU-bound loop starves the event loop, so the current Promise.race timeout never fires — it can only settle once the loop finishes on its own. local-execution.resilience.test.ts
New test confirming process.exit() inside the customer function terminates the whole process immediately, bypassing runScriptLocally's try/finally cleanup — spawns a dedicated fixture as its own Jest process (since process.exit() can't safely run inside this Jest process) so the observed behavior is the real executeScriptLocally/runScriptLocally code path, not a hand-rolled emulation of it. local-execution.resilience.test.ts, local-execution.process-exit.fixture.ts
The spawned fixture's --globalSetup is overridden to a new no-op, since it doesn't touch the fixtures directory and doesn't need the real globalSetup.ts's yarn install + git init/config cost. noopGlobalSetup.ts, local-execution.resilience.test.ts
Extracted the identical func/stubExecuteAction fixture data — previously declared separately in this file, the resilience test, and the spawned fixture — into a new shared module, since the fixture already proves shared imports work fine in its spawned process (it already imports mockLogger/moduleResolverFor from a shared helpers module). local-execution.fixtures.ts, local-execution.test.ts, local-execution.resilience.test.ts, local-execution.process-exit.fixture.ts
  • The third checklist item ("the queue survives a rejected execution and keeps running") is already covered by an existing test in local-execution.test.ts.

QA Instructions

  • Test-only change with no HTTP surface — verified via the unit test commands below, not a live endpoint.
yarn workspace @dd/tests test:unit packages/plugins/apps/src/vite/local-execution.resilience.test.ts
# Expected: 2 passed ✅ VERIFIED
yarn build:all && yarn test:unit
# Full, unscoped suite — a scoped run can't catch a process-wide guard leaking into
# an unrelated package's tests via a shared Jest worker (see the Confluence QA guide).
# Expected: Test Suites: 89 passed / Tests: 2152 passed, 1 skipped ✅ VERIFIED
yarn workspace @dd/apps-plugin run typecheck
# Expected: clean exit ✅ VERIFIED

Blast Radius

  • Test-only change; no production code touched.
  • Risk: low.

Out of Scope / Follow-ups

1 item deferred
Item Status Next step
Whether to pursue real process/thread isolation (e.g. pooled worker_threads) to close the confirmed sync-hang and process.exit() gaps deferred These tests exist to inform that decision with real data — a follow-up design discussion, not blocking this PR.

Documentation

@tyffical
tyffical force-pushed the tiffany.trinh/apps-2792-local-execution-resilience-tests branch from 441fc4e to bfed8d6 Compare September 2, 2026 20:03
@tyffical
tyffical force-pushed the tiffany.trinh/apps-2792-local-execution-resilience-tests branch from bfed8d6 to 586cdec Compare September 2, 2026 22:29
@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Sep 2, 2026

Copy link
Copy Markdown

Tests

All CI checks and tests passed.

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 9d778b3 | Docs | View more details | Give us feedback!

@tyffical
tyffical force-pushed the tiffany.trinh/apps-2792-local-execution-resilience-tests branch from 586cdec to 3b7dbea Compare September 2, 2026 22:50
@tyffical
tyffical requested a balanced review from Copilot September 2, 2026 22:55
@tyffical

tyffical commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 2, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-02T22:58:35.639644Z 3b7dbea Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The process-exit test relies on Jest-buffered output that may be lost during immediate termination.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Friend, this PR adds resilience tests documenting known limitations of in-process local execution.

Changes:

  • Tests CPU-bound timeout starvation and process.exit() behavior.
  • Adds a child-process Jest fixture.
  • Extracts shared local-execution fixtures.
File summaries
File Description
local-execution.test.ts Reuses shared fixtures and strengthens cleanup.
local-execution.resilience.test.ts Adds resilience tests.
local-execution.process-exit.fixture.ts Exercises process termination safely.
local-execution.fixtures.ts Centralizes test data.
Review details

Suppressed comments (1)

packages/plugins/apps/src/vite/local-execution.process-exit.fixture.ts:24

  • The inline resolver construction hides fixture setup inside the execution under test. Create a named resolver first and pass it to executeScriptLocally so setup and execution remain separate.
        moduleResolverFor(func, {
  • Files reviewed: 4/4 changed files
  • Comments generated: 3
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread packages/plugins/apps/src/vite/local-execution.process-exit.fixture.ts Outdated
Comment thread packages/plugins/apps/src/vite/local-execution.resilience.test.ts
Comment thread packages/plugins/apps/src/vite/local-execution.resilience.test.ts Outdated
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: 3b7dbea94d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@tyffical
tyffical force-pushed the tiffany.trinh/apps-2792-local-execution-resilience-tests branch from 3b7dbea to bc57444 Compare September 2, 2026 23:39
@tyffical
tyffical force-pushed the tiffany.trinh/apps-2792-local-execution-resilience-tests branch from 570d957 to e762309 Compare September 3, 2026 00:43
@tyffical
tyffical force-pushed the tiffany.trinh/apps-2792-local-execution-resilience-tests branch from e762309 to 1b96d9c Compare September 3, 2026 01:13
@tyffical
tyffical force-pushed the tiffany.trinh/apps-2792-local-execution-resilience-tests branch from 1b96d9c to e785272 Compare September 3, 2026 06:08
@tyffical
tyffical force-pushed the tiffany.trinh/apps-2792-local-execution-resilience-tests branch from e785272 to d226647 Compare September 3, 2026 06:19
@tyffical
tyffical force-pushed the tiffany.trinh/apps-2792-local-execution-resilience-tests branch from d226647 to 949fcfb Compare September 3, 2026 06:35
@tyffical
tyffical force-pushed the tiffany.trinh/apps-2792-local-execution-resilience-tests branch from 949fcfb to 54caac0 Compare September 3, 2026 08:08
…des (Milestone 7)

process.exit() inside a customer function can't be tested in-process without
killing the Jest worker running the assertion. Spawns
local-execution.process-exit.fixture.ts as its own Jest process so the
process.exit() call runs through the real executeScriptLocally/
runScriptLocally code path instead of a hand-rolled emulation of it.
func/stubExecuteAction were declared identically in three files, including
the spawned-process fixture. The fixture already imports mockLogger/
moduleResolverFor from a shared helpers module today, proving the spawned
process only isolates process.exit() from the parent Jest worker -- module
resolution works the same as any other file. Centralizing removes the
duplication without touching that isolation.
…iling

Doc comments over ~5 lines and block comments over ~3 lines were
restating context better captured concisely; trimmed each to its
single most important WHY.
@tyffical
tyffical force-pushed the tiffany.trinh/apps-2792-local-execution-resilience-tests branch from 54caac0 to 255e6d1 Compare September 3, 2026 08:21
The spawned process.exit() test re-ran globalSetup.ts's real yarn install +
git setup on every invocation, eating into its own timeout budget for no
benefit since it never touches the fixtures directory that setup is for —
overridden via --globalSetup to a no-op. Also: resolve the jest config path
through the existing ROOT constant instead of a fragile relative traversal,
tighten the busy-loop test's margin, and dedupe a comment repeated in two
eslint-disable justifications.
@tyffical
tyffical force-pushed the tiffany.trinh/apps-2792-local-execution-resilience-tests branch from 255e6d1 to 9d778b3 Compare September 3, 2026 08:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants