Skip to content

Replace per-skill AI disclosure with a PreToolUse hook#125889

Draft
lewing wants to merge 12 commits intodotnet:mainfrom
lewing:hooks/ai-disclosure
Draft

Replace per-skill AI disclosure with a PreToolUse hook#125889
lewing wants to merge 12 commits intodotnet:mainfrom
lewing:hooks/ai-disclosure

Conversation

@lewing
Copy link
Member

@lewing lewing commented Mar 21, 2026 •
edited
Loading

Replace per-skill AI disclosure with a single PreToolUse hook

Replaces the per-skill duplication from #125842 with a single PreToolUse hook that reminds about AI-generated content disclosure at the point of action.

What changed

  • Added .github/hooks/hooks.json — a single PreToolUse hook (Copilot CLI format) with command (Node.js), bash, and powershell variants that fires on write-verb tool calls (Bash|PowerShell|create|comment|update|edit|review|push|merge) and checks for gh pr/issue commands or GitHub MCP tool usage
  • Added .claude/settings.json — equivalent Claude Code hook configuration
  • Kept the full "AI-Generated Content Disclosure" section in copilot-instructions.md (unchanged from Require AI-generated content disclosure in copilot instructions and skills #125842) — this is the source of truth
  • Removed the duplicated disclosure text from 6 skill files:
    • api-proposal/SKILL.md
    • ci-analysis/SKILL.md
    • code-review/SKILL.md
    • issue-triage/SKILL.md
    • performance-benchmark/SKILL.md
    • vmr-codeflow-status/SKILL.md

Design

  • The hook is a nudge, not a gate — it writes to stderr and always exits 0, so it never blocks tool execution
  • A single consolidated hook covers both gh CLI commands and GitHub MCP tools, avoiding doubled context overhead from running two hooks on every tool call
  • The short message (~15 tokens) avoids accumulating tokens in long sessions, pointing to the full rule in copilot-instructions.md
  • Cross-tool compatible: Copilot CLI (v1.0.6+), VS Code, and Claude Code

Note

This PR was drafted with the help of an AI agent.

cc @stephentoub

@lewing
Replace per-skill AI disclosure with a PreToolUse hook
148082d 
Move the AI-generated content disclosure from duplicated text in
copilot-instructions.md and 6 SKILL.md files into a single
.github/hooks/hooks.json PreToolUse hook that fires when gh CLI
commands post to GitHub (pr/issue create/comment/edit/review).

This eliminates token waste from repeated instructions and ensures
the disclosure applies uniformly to all GitHub-posting actions,
including any future skills that use gh.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 21, 2026 15:20
@github-actions github-actions bot added the area-skills Agent Skills label Mar 21, 2026
@lewing lewing marked this pull request as draft March 21, 2026 15:22
Copilot AI reviewed Mar 21, 2026
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR centralizes the repository’s “AI-generated content disclosure” reminder by removing duplicated disclosure paragraphs from Copilot instructions and multiple skill prompts, and introducing a single lifecycle hook intended to trigger only when gh is used to post to GitHub.

Changes:

  • Added a .github/hooks/hooks.json PreToolUse hook to detect gh pr/issue {create,comment,edit,review,merge} and emit a disclosure reminder.
  • Removed the duplicated disclosure text from .github/copilot-instructions.md and several .github/skills/*/SKILL.md files.
  • Kept skill workflows otherwise unchanged, relying on the hook as the new single source of truth.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
.github/hooks/hooks.json Adds a PreToolUse hook for Bash that detects certain gh commands and prints a disclosure reminder.
.github/copilot-instructions.md Removes the “AI-Generated Content Disclosure” section from the main Copilot instructions.
.github/skills/api-proposal/SKILL.md Removes the per-skill disclosure paragraph from the publish phase guidance.
.github/skills/ci-analysis/SKILL.md Removes the per-skill disclosure paragraph near the top-level workflow guidance.
.github/skills/code-review/SKILL.md Removes the per-skill disclosure paragraph from the review output format guidance.
.github/skills/issue-triage/SKILL.md Removes the per-skill disclosure paragraph from the anti-patterns section.
.github/skills/performance-benchmark/SKILL.md Removes the per-skill disclosure paragraph near EgorBot posting instructions.
.github/skills/vmr-codeflow-status/SKILL.md Removes the per-skill disclosure paragraph near the top-level workflow guidance.

@lewing
Address review: pure shell hook, broader matcher, keep fallback pointer
691492f 
- Replace node -e with grep+shell (no external runtime dependency)
- Broaden matcher to Bash|PowerShell for cross-platform coverage
- Add one-line disclosure pointer back to copilot-instructions.md
  for discoverability in environments without hook support

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
lewing
lewing commented Mar 21, 2026
View reviewed changes
Copy link
Member Author

@lewing lewing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed all three review comments in 691492f:

  1. Node dependency → Replaced node -e with pure shell (grep -qE ... && echo >&2; :). No external runtime needed, always exits 0.
  2. Matcher scope → Broadened to "Bash|PowerShell" for cross-platform coverage.
  3. Discoverability → Added a one-line pointer back in copilot-instructions.md linking to .github/hooks/hooks.json so the rule is visible even in environments without hook support.

Note

This comment was generated with the help of Copilot.

@lewing lewing requested a review from Copilot March 21, 2026 15:29
@lewing
Use official hook format with bash/powershell keys
d740af9 
Use the version 1 hooks schema with separate bash and powershell
keys for cross-platform support. Pure shell builtins on both
platforms — no dependency on grep, node, or other external tools.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI reviewed Mar 21, 2026
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

lewing and others added 2 commits March 21, 2026 10:34
@lewing
Soften disclosure wording: reminder, not enforcement
081b325 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@lewing
Broaden hook to cover all GitHub interactions, not just gh CLI
f750f38 
- Shell hook: match any command mentioning 'github' or 'gh' (covers
  gh CLI, curl to api.github.com, etc.)
- GitHub MCP tools hook: unconditional reminder for any tool with
  'github' in the name
- copilot-instructions.md: remove gh-specific wording; the rule
  applies to all GitHub posting methods

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 21, 2026 15:39
Copilot AI reviewed Mar 21, 2026
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

lewing and others added 2 commits March 21, 2026 10:49
@lewing
Drop MCP tool hook — keep it simple
8605e50 
The copilot-instructions.md rule is the source of truth. The shell
hook is a best-effort nudge when gh/github commands run, not a gate.
Remove the overly broad GitHub MCP tool matcher that would spam on
every read-only query.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@lewing
Add write-only MCP tool hook alongside shell hook
44b9d43 
Two hooks:
1. Shell (Bash|PowerShell): fires when command mentions github/gh
2. MCP tools: matcher catches write verbs (create|comment|update|
   edit|review|push|merge), then body checks for 'github' in the
   tool args to avoid false positives from non-GitHub tools

No metadata on MCP tools to distinguish read vs write, so we match
on tool name patterns containing write-like verbs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 21, 2026 15:52
Copilot AI reviewed Mar 21, 2026
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

lewing and others added 2 commits March 21, 2026 11:00
@lewing
Tighten hook: write-only gh subcommands + short message
1c10b95 
Shell hook now only fires on gh pr/issue create/comment/review/edit/
merge/close — not on reads like 'gh pr view' or 'gh issue list'.
Message shortened to ~15 tokens referencing copilot-instructions.md
for the full rule, avoiding context bloat in long sessions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@lewing
Restore full disclosure section in copilot-instructions.md
5a6fd89 
The hook message is a short pointer ('per copilot-instructions.md'),
so the full rule needs to live here. Restores the original section
from dotnet#125842. The per-skill duplicates remain removed — the hook
provides the point-of-action reminder, this section is the source
of truth.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 21, 2026 16:03
Copilot AI reviewed Mar 21, 2026
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

danmoseley
danmoseley approved these changes Mar 21, 2026
View reviewed changes
Copy link
Member

@danmoseley danmoseley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm if this works with all agents?

@lewing
Copy link
Member Author

lewing commented Mar 21, 2026

Note

This comment was generated by Copilot CLI during local hook testing.

Testing hooks locally — please ignore this comment.

@danmoseley
Copy link
Member

danmoseley commented Mar 21, 2026

random commits in this PR?

@lewing
Consolidate AI disclosure hooks into a single preToolUse hook
1e2ccaf 
Merge the two separate preToolUse hooks (one for Bash/PowerShell gh CLI
commands, one for GitHub MCP tools) into a single hook that covers both
surfaces. This halves the number of hook executions per tool call,
reducing context overhead.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@lewing lewing force-pushed the hooks/ai-disclosure branch from 24bc9a0 to 1e2ccaf Compare March 21, 2026 18:05
Copilot AI review requested due to automatic review settings March 21, 2026 18:05
@lewing
Copy link
Member Author

lewing commented Mar 21, 2026

random commits in this PR?

fixed

Copilot AI reviewed Mar 21, 2026
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

lewing and others added 2 commits March 21, 2026 14:15
@lewing
Universal hook format: works on all four agent hosts
4d27624 
Switch to Claude nested format (PascalCase events, matcher→hooks
structure) — the only format that works across all tools:
- bash key: pure shell builtins (Copilot CLI, VS Code on Linux/macOS)
- powershell key: native PS (Copilot CLI, VS Code on Windows)
- command key: node -e fallback (Claude Code, Gemini CLI)

Add .claude/settings.json with command-only hooks since Claude Code
doesn't read .github/hooks/.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@lewing
Consolidate back to single hook entry
b6477d3 
Two entries double the context cost since both matchers fire on
Bash tool calls. One combined entry with merged matcher and stdin
checks covers both gh CLI and GitHub MCP tools.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 21, 2026 19:18
Copilot AI reviewed Mar 21, 2026
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

@build-analysis build-analysis bot mentioned this pull request Mar 21, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@danmoseley danmoseley danmoseley approved these changes

@stephentoub stephentoub Awaiting requested review from stephentoub

Assignees

@lewing lewing

Labels

area-skills Agent Skills

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@lewing @danmoseley