Skip to content

advisor: soft checkpoints + onboarding hint (#236)#242

Merged
uta2000 merged 18 commits intomainfrom
feat/advisor-integration
Apr 17, 2026
Merged

advisor: soft checkpoints + onboarding hint (#236)#242
uta2000 merged 18 commits intomainfrom
feat/advisor-integration

Conversation

@uta2000
Copy link
Copy Markdown
Owner

@uta2000 uta2000 commented Apr 17, 2026

Summary

Integrates the Claude advisor tool (beta advisor-tool-2026-03-01) into feature-flow as a complement to codex-consultation (#235). Closes #236.

Two additive changes:

  1. Four soft advisor-call hints in judgment-heavy skills (design-verification, verify-acceptance-criteria, and two in consult-codex). Non-blocking — Claude decides per-invocation whether to call advisor().
  2. Onboarding/settings hint — new hooks/scripts/advisor-hint.js SessionStart script nudges Sonnet users (rate-limited to once/day, permanently dismissible) to enable the advisor beta header. Full docs at docs/advisor.md; one-paragraph pointer in README. New feature-flow:settings advisor [dismiss] sub-action.

Deferred: tiered stuck-mode machinery and per-call observability — advisor runs as a server-side sub-inference with no hook-visible surface today.

Design Doc

Issue #236 is the authoritative design. docs/plans/2026-04-17-advisor-integration.md is a thin pointer to it; docs/plans/2026-04-17-advisor-integration-plan.md is the 15-task implementation plan.

Scope

  • 4 SKILL.md one-liner inserts (design-verification, verify-acceptance-criteria, consult-codex ×2)
  • 2 new JS scripts (skills/settings/scripts/advisor-headers.js, skills/settings/scripts/check-advisor.js) + 1 new hook script (hooks/scripts/advisor-hint.js) + 2 colocated test suites
  • 1 new third entry in the existing hooks/hooks.json SessionStart array (no new hook event type)
  • 1 new `advisor` sub-action handler in `skills/settings/SKILL.md`
  • 1 new docs/advisor.md (4 required sections: What it is / How to enable / When it's worth it / Relationship to codex-consultation)
  • README "Optional Enhancements / Advisor Tool" section
  • .feature-flow.yml commented defaults for `advisor.enabled` + `hints.advisor.dismissed`
  • ## Relationship to advisor tool section appended to docs/plans/2026-04-14-codex-consultation.md

Review Iteration

An initial code-review pass (pr-review-toolkit) flagged four findings — all addressed in commit refactor(advisor): address code-review findings:

  1. Critical — test hermeticity: test run() helpers now delete XDG_CONFIG_HOME and APPDATA to prevent the developer's real settings.json from bleeding through on Linux/Windows.
  2. Important — Sonnet gate: advisor-hint.js now calls detectSonnet() as Condition 5. Fails open (eligible) when CLAUDE_MODEL unset; suppresses hint for non-Sonnet models.
  3. Important — YAML regex over-capture: replaced unbounded regex with an indentation-aware line-by-line parser (parseHintsAdvisorDismissed) to prevent false matches from sibling hints.* blocks with a dismissed field.
  4. Important — duplication: advisor-hint.js now imports detection helpers from check-advisor.js (~30 fewer lines duplicated).

Test Plan

  • node hooks/scripts/advisor-hint.test.js — 13 passing (was 11; +2 Sonnet-gate tests)
  • node skills/settings/scripts/check-advisor.test.js — 14 passing (was 12; +2 sibling-regex tests)
  • node hooks/scripts/version-check.test.js — 29 passing (baseline unchanged)
  • node hooks/scripts/verdict-gate.test.js — 10 passing (baseline unchanged)
  • All 11 ACs from issue advisor: tiered stuck mode + soft checkpoints + settings hint #236 verified via grep/file checks (see plan Task 15)
  • Manual smoke: enable advisor beta header in ~/.claude/settings.json, confirm hint disappears
  • Manual smoke: run feature-flow:settings advisor dismiss, confirm hints.advisor.dismissed: true is written and next SessionStart produces no hint output
  • Manual smoke: verify advisor-hint.js silently exits 0 when `~/.feature-flow/` can't be written (e.g. readonly HOME)

Acceptance Criteria Status

All 11 ACs PASS. See docs/plans/2026-04-17-advisor-integration-plan.md Task 15 for the verification script.

Closes #236

🤖 Generated with Claude Code

@uta2000
Copy link
Copy Markdown
Owner Author

uta2000 commented Apr 17, 2026

Manual smoke tests executed (2026-04-17) — all three PASS:

  1. advisor beta header suppresses hint

    • Without header → hint shown
    • With ANTHROPIC_BETA containing advisor-tool-2026-03-01 → no output (stdout + stderr empty)
  2. dismiss flow suppresses hint

    • Before dismiss (advisor.enabled: true, no hints block) → hint shown
    • After setting hints.advisor.dismissed: true → no output
    • Bonus: rate-limiter state file NOT written when hint is suppressed by dismiss (the hook short-circuits before markShownToday)
  3. readonly HOME → silent exit 0

    • Set HOME to a chmod 555 tmpdir where ~/.feature-flow/hint-state.json cannot be written
    • Hook exits 0
    • stderr is empty (no error traces, no 'hook error' messages)
    • Hint still fires on stdout (correct — rate-limiter write failure is designed to be silent)

All test-plan checkboxes can now be checked. No blockers for merge.

uta2000 and others added 18 commits April 17, 2026 15:29
Adds the implementation plan (15 tasks, 11 ACs mapped) and a thin
design-doc pointer for the advisor integration feature. Issue #236 is
the authoritative design; the pointer exists to satisfy feature-flow's
design-doc artifact contract without duplicating content.

Verification pass on 2026-04-17 resolved the [B#235] dependency
(consult-codex merged in 2c289be) and corrected the hook extension
target from a non-existent `hooks/scripts/session-start/index.js` to a
new `hooks/scripts/advisor-hint.js` following the `version-check.js`
pattern.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move check-advisor test file from hooks/scripts/ to skills/settings/scripts/
to match the repo convention (version-check.js + version-check.test.js are
colocated). Update the path.resolve() inside the test and all AC verification
commands in Task 15 accordingly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ceptance-criteria

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…consult-codex

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… detection

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…r and gate logic

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…mmand

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…miss)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… relationship

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ADME

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ted defaults to .feature-flow.yml

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…onsultation design doc

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…hermeticity, Sonnet gate)

Addresses four code-review findings from the pr-review-toolkit pass:

1. Test hermeticity (Critical): both test run() helpers now delete
   XDG_CONFIG_HOME and APPDATA from the env passed to the script, so the
   platform-specific getSettingsPath() falls back to HOME-relative paths
   and cannot read the developer's real settings.json on Linux/Windows.
   advisor-hint.test.js also deletes CLAUDE_MODEL by default.

2. Sonnet gate added (Important): advisor-hint.js now calls detectSonnet()
   (imported from check-advisor.js) as Condition 5 of the gate. The hint
   is suppressed for non-Sonnet models; when CLAUDE_MODEL is unset,
   detectSonnet returns null and the gate fails open (hint eligible),
   matching the spec in issue #236.

3. YAML regex over-capture (Important): the previous unbounded
   `((?:[ \t]+.*\n)*)` capture under `hints.advisor:` would absorb
   arbitrary sibling blocks like `hints.other.dismissed: true` and
   falsely return dismissed: true. Replaced with an indentation-aware
   line-by-line parser (parseHintsAdvisorDismissed) in check-advisor.js.
   Added two regression tests for sibling-block isolation.

4. Duplication removed (Important): advisor-hint.js no longer reimplements
   getSettingsPath, header detection, or dismissed-parsing. It imports
   readSettings, detectHeaderPresent, parseHintsAdvisorDismissed, and
   detectSonnet from check-advisor.js via the CLAUDE_PLUGIN_ROOT-relative
   path pattern (same mechanism already used for advisor-headers.js).
   check-advisor.js now wraps main() in `require.main === module` so it
   runs as a CLI but also exports helpers.

Test results after refactor:
  check-advisor.test.js: 14 passed (was 12)
  advisor-hint.test.js:  13 passed (was 11)
  version-check.test.js: 29 passed (unchanged)
  verdict-gate.test.js:  10 passed (unchanged)
All 4 Change-1 SKILL.md greps still match.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@uta2000 uta2000 force-pushed the feat/advisor-integration branch from 8053b0b to 4e03f23 Compare April 17, 2026 19:30
@uta2000 uta2000 merged commit 5a33f11 into main Apr 17, 2026
@uta2000 uta2000 deleted the feat/advisor-integration branch April 17, 2026 19:31
uta2000 added a commit that referenced this pull request Apr 17, 2026
…236 #238 #239

Aligns version fields across .claude-plugin/plugin.json (already at 1.36.0
from #240), .claude-plugin/marketplace.json, and .feature-flow.yml. Cuts the
[1.36.0] - 2026-04-17 release heading in CHANGELOG.md bundling the five
features shipped today:

- #238 Scope-critique pass for design-verification (PR #243, this PR)
- #236 Advisor tool integration (PR #242)
- #239 Senior developer panel, Phase 1c code review (PR #241)
- #235 Codex consultation, Phase 1+2 opt-in (PR #240)
- #234 Quick-path triage with code-aware scope confirmation (PR #237)

Consolidates .changelogs/238.md fragment into CHANGELOG.md (per the Ship
phase convention) and deletes the fragment. Updates README with a
"What's New in 1.36.0" section listing all five PRs, adds a
Codex Consultation subsection under Optional Enhancements, adds a
Quick-path triage subsection, and updates the Skills table to include
consult-codex and the scope-critique pass note for design-verification.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
uta2000 added a commit that referenced this pull request Apr 17, 2026
* docs: add implementation plan for scope-critique-pass (#238)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(design-verification): add scope-critique pass (#238)

Adds a ## Scope critique section to skills/design-verification/SKILL.md and a
supporting reference file with five strategic-shape questions (scope,
dependencies, simpler alternatives, observability, config surface) that run
after the existing compatibility check to catch oversized scope, phantom
dependencies, and unobservable capability bets before a design reaches
create-issue.

Motivated by #236 post-creation review: 6 strategic issues bypassed existing
brainstorming, design-verification, and self-review gates. The new pass fills
that gap with a bias-resistance checklist, red-flag lists per question, and a
findings table that promotes BLOCKER rows to the top-level Blockers list.

Closes #238

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: add changelog fragment for #238

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: bump version to 1.36.0 and consolidate changelog for #234 #235 #236 #238 #239

Aligns version fields across .claude-plugin/plugin.json (already at 1.36.0
from #240), .claude-plugin/marketplace.json, and .feature-flow.yml. Cuts the
[1.36.0] - 2026-04-17 release heading in CHANGELOG.md bundling the five
features shipped today:

- #238 Scope-critique pass for design-verification (PR #243, this PR)
- #236 Advisor tool integration (PR #242)
- #239 Senior developer panel, Phase 1c code review (PR #241)
- #235 Codex consultation, Phase 1+2 opt-in (PR #240)
- #234 Quick-path triage with code-aware scope confirmation (PR #237)

Consolidates .changelogs/238.md fragment into CHANGELOG.md (per the Ship
phase convention) and deletes the fragment. Updates README with a
"What's New in 1.36.0" section listing all five PRs, adds a
Codex Consultation subsection under Optional Enhancements, adds a
Quick-path triage subsection, and updates the Skills table to include
consult-codex and the scope-critique pass note for design-verification.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

advisor: tiered stuck mode + soft checkpoints + settings hint

1 participant