Skip to content

👷 add dev-server CLI + manual-testing skill#4337

Merged
BenoitZugmeyer merged 8 commits intomainfrom
benoit/dev-server-cli
Mar 17, 2026
Merged

👷 add dev-server CLI + manual-testing skill#4337
BenoitZugmeyer merged 8 commits intomainfrom
benoit/dev-server-cli

Conversation

@BenoitZugmeyer
Copy link
Copy Markdown
Member

@BenoitZugmeyer BenoitZugmeyer commented Mar 13, 2026

Motivation

Manual testing of SDK features is currently hard to reproduce and share with reviewers.
When writing PR test instructions, there's no structured way to verify what events were
actually sent to intake. This PR introduces a daemonized dev-server CLI that makes
manual testing steps reproducible and self-documenting. It also ships a /manual-testing
Claude Code skill that automates the full workflow: starts the server, creates a sandbox
page, drives it with playwright-cli, inspects intake, and outputs ready-to-paste PR
test instructions.

Changes

  • Replace the single scripts/dev-server.ts with a daemonized CLI (yarn dev-server <command>)
  • start: spawns the server as a detached background process, writes PID/URL to .dev-server/state
  • stop: kills the daemon and clears state
  • status: shows whether the server is running, with URL, PID, and file paths
  • logs: streams the server log file, with --follow/-f support and ANSI stripping when piped
  • intake: reads captured intake requests from .dev-server/intake-requests.json, with selectors
    (rum-views, rum-errors, rum-actions, logs-events, replay-records, etc.), human-readable
    output by default (TTY) and NDJSON when piped (--json to override)
  • Intake proxy logs requests to a write stream and prints jq query examples in --help
  • Add .claude/skills/manual-testing.md — a Claude Code skill (/manual-testing) that
    orchestrates the full end-to-end test flow using the dev-server CLI and playwright-cli,
    then emits self-contained bash test instructions ready to paste into the PR

Test instructions

You can generate these instructions automatically by running /manual-testing in Claude Code.

yarn dev-server start         # starts the server, prints URL and log path
yarn dev-server status        # confirms it's running
yarn dev-server logs -f       # follow logs in another terminal

# open the sandbox in a browser, interact with it, then:
yarn dev-server intake rum-views | jq .view.loading_time
yarn dev-server intake rum-errors | jq .error.message
yarn dev-server intake        # human-readable output in terminal

yarn dev-server stop
yarn dev-server status        # confirms it stopped

Checklist

  • Tested locally
  • Tested on staging
  • Added unit tests for this change.
  • Added e2e/integration tests for this change.
  • Updated documentation and/or relevant AGENTS.md file

@BenoitZugmeyer BenoitZugmeyer force-pushed the benoit/dev-server-cli branch from 55d290c to d3fbb39 Compare March 13, 2026 17:26
@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da bot commented Mar 13, 2026

Bundles Sizes Evolution

📦 Bundle Name Base Size Local Size 𝚫 𝚫% Status
Rum N/A 175.02 KiB N/A N/A N/A
Rum Profiler 6.16 KiB 6.16 KiB 0 B 0.00%
Rum Recorder 27.46 KiB 27.46 KiB 0 B 0.00%
Logs 56.80 KiB 56.80 KiB 0 B 0.00%
Rum Slim 130.66 KiB 130.66 KiB 0 B 0.00%
Worker 23.63 KiB 23.63 KiB 0 B 0.00%
🚀 CPU Performance
Action Name Base CPU Time (ms) Local CPU Time (ms) 𝚫%
RUM - add global context 0.004 0.008 +100.00%
RUM - add action 0.0131 0.0204 +55.73%
RUM - add error 0.0127 0.0234 +84.25%
RUM - add timing 0.003 0.0052 +73.33%
RUM - start view 0.0121 0.0208 +71.90%
RUM - start/stop session replay recording 0.0006 0.0008 +33.33%
Logs - log message 0.0145 0.0178 +22.76%
🧠 Memory Performance
Action Name Base Memory Consumption Local Memory Consumption 𝚫
RUM - add global context 27.21 KiB 26.87 KiB -347 B
RUM - add action 51.76 KiB 52.10 KiB +350 B
RUM - add timing 26.28 KiB 26.13 KiB -152 B
RUM - add error 56.24 KiB 55.26 KiB -1006 B
RUM - start/stop session replay recording 25.86 KiB 25.61 KiB -256 B
RUM - start view 461.46 KiB 461.16 KiB -310 B
Logs - log message 43.98 KiB 44.76 KiB +803 B

🔗 RealWorld

@datadog-prod-us1-6
Copy link
Copy Markdown

datadog-prod-us1-6 bot commented Mar 13, 2026

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 77.23%

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: f166f10 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!

@BenoitZugmeyer BenoitZugmeyer force-pushed the benoit/dev-server-cli branch 6 times, most recently from 118d843 to 035b301 Compare March 13, 2026 23:29
Base automatically changed from benoit/e2e-offline-support to main March 16, 2026 09:53
@BenoitZugmeyer BenoitZugmeyer force-pushed the benoit/dev-server-cli branch from 035b301 to 29fddaa Compare March 16, 2026 10:36
@BenoitZugmeyer BenoitZugmeyer changed the title 👷 add dev-server CLI with start, stop, status, logs, and intake subcommands 👷 add dev-server CLI + manual-testing skill Mar 16, 2026
@BenoitZugmeyer BenoitZugmeyer force-pushed the benoit/dev-server-cli branch 2 times, most recently from a526088 to d8c752a Compare March 16, 2026 15:25
This makes the intake proxy middleware self-contained: callers no longer
need to set up connect-busboy separately.
Replaces the single-file dev-server script with a daemonized CLI.
The server is spawned as a detached background process, with its PID
and URL persisted to `.dev-server/state` and logs written to
`.dev-server/logs`. This enables reproducible manual testing workflows
where the dev server URL can be referenced in PR test instructions.
@BenoitZugmeyer BenoitZugmeyer force-pushed the benoit/dev-server-cli branch from d8c752a to 2157b7d Compare March 16, 2026 15:29
@BenoitZugmeyer BenoitZugmeyer marked this pull request as ready for review March 16, 2026 15:30
@BenoitZugmeyer BenoitZugmeyer requested a review from a team as a code owner March 16, 2026 15:30
@BenoitZugmeyer BenoitZugmeyer force-pushed the benoit/dev-server-cli branch from 2157b7d to dea124b Compare March 16, 2026 15:34
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2157b7d2ed

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

return
}

process.kill(state.pid)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Verify daemon identity before sending kill signal

stop sends process.kill(state.pid) after only checking that the PID exists; isRunning() in state.ts is just process.kill(pid, 0). If the dev-server process exited and that PID was reused, yarn dev-server stop can kill an unrelated process on the host instead of the daemon.

Useful? React with 👍 / 👎.

@amortemousque amortemousque self-assigned this Mar 16, 2026
Comment on lines +10 to +11
// eslint-disable-next-line local-rules/disallow-test-import-export-from-src, local-rules/disallow-protected-directory-import
import { createIntakeProxyMiddleware } from '../../../test/e2e/lib/framework/intakeProxyMiddleware.ts'
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: I'm not too happy with this, but I'm not sure where this intakeProxyMiddleware should stay, whether in the test/e2e or the scripts folder. I chose to keep it there for now, we might have more shared code in the future, we can always revisit.

Also, move the skill to the right directory
@BenoitZugmeyer BenoitZugmeyer merged commit 083ef39 into main Mar 17, 2026
21 checks passed
@BenoitZugmeyer BenoitZugmeyer deleted the benoit/dev-server-cli branch March 17, 2026 12:49
@github-actions github-actions bot locked and limited conversation to collaborators Mar 17, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants