optimize: move docs-noob-tester setup to pre-agent-steps, slim prompt, reduce timeout#28343
Merged
optimize: move docs-noob-tester setup to pre-agent-steps, slim prompt, reduce timeout#28343
Conversation
- Move deterministic setup (npm install, server start, readiness poll, bridge IP detection) to pre-agent-steps, saving ~600K–900K tokens/run by eliminating ~10–15 agent setup turns - Slim duplicate Playwright connectivity fallback from prompt body (already covered by shared/docs-server-lifecycle.md import) - Reduce timeout-minutes: 45 → 30 (observed run was 25.3min, post- optimization should fit well within 30min) - Agent now reads server URL from /tmp/gh-aw/agent/server-url.txt - Renumber prompt steps 2–6 → 1–5 reflecting removed setup step - Recompile lock file" Agent-Logs-Url: https://github.com/github/gh-aw/sessions/c7cceec6-aeff-4e6a-9099-d878ea038089 Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
- Add comment explaining the 45-iteration wait loop (135s max wait) - Improve Playwright example to show reading URL from server-url.txt - Fix 'Vite dev server' → 'Astro development server' for accuracy Agent-Logs-Url: https://github.com/github/gh-aw/sessions/c7cceec6-aeff-4e6a-9099-d878ea038089 Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Optimize documentation noob tester for better token efficiency
optimize: move docs-noob-tester setup to pre-agent-steps, slim prompt, reduce timeout
Apr 24, 2026
This comment has been minimized.
This comment has been minimized.
Contributor
|
Hey One thing to address before moving out of draft:
If you'd like a hand finishing up, you can assign this prompt to your coding agent:
|
pelikhan
approved these changes
Apr 24, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
Optimizes the docs-noob-tester agentic workflow to reduce token usage and runtime by moving deterministic docs-server setup out of the agent prompt and tightening runtime limits.
Changes:
- Added
pre-agent-stepsto install docs deps, start the Astro dev server, wait for readiness, and write a Playwright server URL to/tmp/gh-aw/agent/server-url.txt. - Slimmed the agent prompt to remove duplicated lifecycle/connectivity instructions and updated step numbering.
- Reduced workflow/agent execution timeout from 45 to 30 minutes and regenerated the lock file accordingly.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/docs-noob-tester.md |
Adds pre-agent server setup + updates prompt to consume precomputed server URL and reduces timeout. |
.github/workflows/docs-noob-tester.lock.yml |
Regenerated lockfile reflecting the new timeout and inserted pre-agent steps in the compiled workflow. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 2/2 changed files
- Comments generated: 4
Comment on lines
+53
to
+57
| for i in {1..45}; do # 45 attempts × 3s = 135s max wait | ||
| STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:4321/gh-aw/) | ||
| [ "$STATUS" = "200" ] && echo "Server ready at http://localhost:4321/gh-aw/!" && break | ||
| echo "Waiting for server... ($i/45) (status: $STATUS)" && sleep 3 | ||
| done |
Comment on lines
+58
to
+64
| - name: Detect bridge IP and write server URL | ||
| run: | | ||
| SERVER_IP=$(ip -4 route get 1.1.1.1 2>/dev/null | awk '{print $7; exit}') | ||
| if [ -z "$SERVER_IP" ]; then SERVER_IP=$(hostname -I | awk '{print $1}'); fi | ||
| mkdir -p /tmp/gh-aw/agent | ||
| echo "http://${SERVER_IP}:4321/gh-aw/" > /tmp/gh-aw/agent/server-url.txt | ||
| echo "Playwright server URL: http://${SERVER_IP}:4321/gh-aw/" |
| ## Step 6: Cleanup | ||
| ## Step 5: Cleanup | ||
|
|
||
| Follow the shared **Documentation Server Lifecycle Management** instructions for cleanup (section "Stopping the Documentation Server"). |
| - name: Install docs dependencies | ||
| run: | | ||
| cd "${{ github.workspace }}/docs" | ||
| npm install |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A single run of
docs-noob-testerconsumed 3M tokens across 60 turns for a task scoped to 3 pages — roughly 20× expected. ~10–15 of those turns were spent on deterministic server setup with no AI decision-making required.Changes
Move setup to
pre-agent-steps(~600K–900K tokens/run saved)Four steps now run before the agent starts, eliminating the setup turns entirely:
npm installin/docsnohup npm run dev -- --host 0.0.0.0 --port 4321)curlloop, 45 × 3s = 135s max)/tmp/gh-aw/agent/server-url.txtThe agent reads the pre-computed Playwright URL from that file instead of computing it:
cat /tmp/gh-aw/agent/server-url.txt # → http://172.30.x.x:4321/gh-aw/Slim the prompt (~50K–100K tokens/run saved)
Removed the ~30-line duplicate Playwright connectivity fallback block — identical guidance already exists in the
shared/docs-server-lifecycle.mdimport. Condensed to a single pointer.Reduce
timeout-minutes: 45 → 30Observed run completed in 25.3 min. Post-optimization (setup no longer in agent time) fits well within 30 min with headroom.