docs(skill): correct verified command drift in the agent skill

[thdurante]

Summary

Four places where the agent skill recommended commands or semantics that don't match the CLI. Each verified against source, not assumed.

Stacked on #39 (base is that branch, not main) — both touch SKILL.md and evals.json. Retarget to main once #39 merges.

Closes DHQ-695.

What was wrong

Drift	Verified how
deploy --wait documented as blocking	WantsJSON() is JSONMode || !IsTTY (envelope.go:51-56) and the JSON branch returns at deploy.go:588, so if wait at :598 is unreachable. Confirmed by control test on WantsJSON.
"non-zero = failure"	watch.go:108-111 — case "cancelled": … return nil. Cancelled exits 0. dhq rollback shares the watcher.
ssh-commands create shown as a usable release hook	ssh_commands.go:229-231 — only --command, --description, --timing. No callback phase, targeting, timeout, or halt-on-error.
CLAUDE.md said 8 reference docs	Directory has 9.

The one worth reading

dhq deploy --wait is inert whenever stdout isn't a terminal — the trigger is the missing TTY, not --json. So in every CI job, pipe, redirect and agent invocation, --wait does nothing and the command exits 0 while the deployment is still running. Four skill examples taught exactly that pattern.

This PR documents the behaviour and replaces those examples with the safe composition. It deliberately does not change the CLI — that was a conscious call, so the flag remains silently inert for existing users. Worth a separate decision.

id=$(dhq deploy -p my-app -s Production --json | jq -r '.data.identifier')
dhq deployments watch "$id" -p my-app
status=$(dhq deployments show "$id" -p my-app --json=status | jq -r '.status')
[ "$status" = "completed" ] || exit 1

The status == "completed" assertion is required rather than belt-and-braces: cancelled exits 0 too. Note .status, not .data.status — --json=<fields> unwraps the envelope and emits the selected fields at the top level.

Review round

Four findings, all fixed, all threads resolved.

Reviewer	Finding	Outcome
Codex (P1)	--json=status unwraps the envelope, so .data.status is null — the verification recipe would exit 1 after every successful deploy	Fixed in 785ed0c, both call sites; added a note on the shape difference
Codex (P2)	The deploy-and-verify eval passed for a response containing only deployments show	Fixed in 785ed0c — now requires watch, show and completed
CodeRabbit	--json=…,server — Deployment has no server field (that's DeploymentStep), so it is silently dropped	Fixed in 592956f → servers
CodeRabbit	Eval-suite count in CLAUDE.md stale	Fixed in 592956f → 69 (not the suggested 52; the stacked branches add 5 and 4 that aren't visible from this diff)

The P1 is worth flagging to a reviewer: the recipe added here to stop automation trusting a bad deploy would, as first written, have failed the job after every good one.

Blast radius

Docs and evals only — no Go source touched. deployments.md, SKILL.md, configuration.md, servers.md, CLAUDE.md, evals.json.

Test plan

• go build ./cmd/dhq/, go vet ./..., go test ./... — 879 passing, unchanged (docs-only)
• All 7 acceptance criteria checked mechanically: no --wait --json left anywhere in skills/, reference count matches the directory, cancelled documented as unsuccessful, safe composition present, ssh-commands limitation documented, post-deploy check documented with ordering/variables/failure/timeout/beta-gate, evals added
• deployment-checks flags in the recommended snippet verified to exist (deployment_checks.go:106-118)
• Reviewer: ./skill-evals/deployhq/run-evals.sh for the three new cases (needs an API key)

3 new evals: automation must not reach for --wait; a cancelled deploy must be detected by status not exit code; post-release reconcile must use a deployment check rather than ssh-commands.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Hibw5xsRNGDkYzr1hXsDQz

Summary by CodeRabbit

• Documentation

  • Clarified that automated and non-interactive deployments require explicit monitoring and verification of final completion status.
  • Added warnings that cancelled deployments may return success without deploying code.
  • Updated deployment examples to capture deployment IDs, monitor progress, and confirm completion.
  • Documented post-deployment checks for SSH hook behavior, including configuration, ordering, targeting, timeouts, and failure handling.

• Tests

  • Added evaluation scenarios for deployment verification, cancelled deployments, and post-deployment SSH reconciliation.

[linear]

DHQ-695

[coderabbitai]

Walkthrough

DeployHQ documentation now distinguishes interactive and automated deployment behavior. It requires explicit completion checks, documents cancelled deployment handling, and adds guidance for post-deployment SSH deployment checks. Evaluation cases cover these requirements.

Changes

DeployHQ verification guidance

Layer / File(s)	Summary
Deployment behavior and status contracts skills/deployhq/SKILL.md, skills/deployhq/references/deployments.md	The documentation defines TTY-specific --wait behavior, queued deployment responses, successful cancellation exit codes, and required completed status validation.
Automated deployment verification CLAUDE.md, skills/deployhq/references/deployments.md, skills/deployhq/references/servers.md, skill-evals/deployhq/evals.json	Automation examples and evaluations require deployment creation, watching, and explicit completion checks. The architecture documentation updates the related file and evaluation counts.
Post-deployment SSH checks skills/deployhq/references/configuration.md, skill-evals/deployhq/evals.json	The documentation and evaluations describe deployment-checks create, including defaults, ordering, failure handling, timeout, server targeting, and availability.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• deployhq/deployhq-cli#10: Adds the deployment-checks workflow documented and evaluated here.
• deployhq/deployhq-cli#14: Updates related DeployHQ SSH-command behavior.
• deployhq/deployhq-cli#20: Updates related SSH command documentation.

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the documentation change that corrects verified CLI behavior drift in the agent skill.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch thiago/dhq-695-correct-verified-command-drift-in-the-deployhq-agent-skill

---

_{Comment @coderabbitai help to get the list of available commands.}

[thdurante]

@codex review this

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3a9aca8f33

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[coderabbitai]

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@CLAUDE.md`:
- Line 45: Update the evaluation-suite total documented in CLAUDE.md from 49 to
52, after verifying that skill-evals/deployhq/evals.json contains the three
newly added cases.

In `@skill-evals/deployhq/evals.json`:
- Around line 653-664: Update the workflow-deploy-wait-timeout eval case in
evals.json to remove the stale dhq deploy --wait --timeout expectation. Require
the agent workflow to create the deployment, watch it, and use deployments show
to explicitly verify status == completed, consistent with the
deploy-and-verify-in-automation case and SKILL.md behavior.

In `@skills/deployhq/references/deployments.md`:
- Around line 88-89: Update the `dhq deployments show` example to request the
`servers` JSON field instead of `server`, matching the serialized
`Deployment.Servers` property while preserving the other selected fields.

In `@skills/deployhq/references/servers.md`:
- Around line 264-266: Update the manual production deploy example to capture
the deployment identifier from dhq deploy, wait for completion with deployments
watch, and perform a final status == "completed" check. Keep the existing
project and Production server parameters while replacing the standalone deploy
--json command with the full verification sequence.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: d8783982-2ef9-4488-a677-01843c466ba1

📥 Commits

Reviewing files that changed from the base of the PR and between c7b4792 and 3a9aca8.

📒 Files selected for processing (6)

• CLAUDE.md
• skill-evals/deployhq/evals.json
• skills/deployhq/SKILL.md
• skills/deployhq/references/configuration.md
• skills/deployhq/references/deployments.md
• skills/deployhq/references/servers.md

[facundofarias]

👍🏼