Make default VS Code Copilot graphify instructions stricter so agents stop bypassing GRAPH_REPORT.md#688
Closed
mvanderbend-msoft wants to merge 1 commit into
Conversation
…g GRAPH_REPORT.md The previous default copilot-instructions.md section was too soft. In practice Copilot agents (both VS Code Copilot Chat and the GitHub Copilot CLI) frequently bypassed it - they answered architecture/how-to/where-is questions straight from memory or by grepping source, instead of reading graphify-out/GRAPH_REPORT.md first. That defeated the point of the graph. This change replaces _VSCODE_INSTRUCTIONS_SECTION with a more prescriptive version: - Mandatory first tool call: for any architecture/codebase question, the agent's first tool call must be reading GRAPH_REPORT.md. - Explicit trigger list (how do I, where is, add a handler, explain the architecture, etc.) so the agent recognises when the rule fires. - Trust the graph: explicit instruction NOT to re-verify by opening source files after the graph for conceptual questions. - Narrow allowlist: source reads still allowed for actual code modification / debugging / when the graph genuinely lacks detail or is stale. - Self-check: the agent is told to verify before each reply that its first tool call read the report. Tested locally with the Copilot CLI on a Java REST API repo: with the soft wording the agent answered 'what's needed to add a new endpoint?' without ever reading GRAPH_REPORT.md; with the stricter wording it consistently reads the report first and grounds its answer in god nodes / communities. Scope: only touches _VSCODE_INSTRUCTIONS_SECTION (the VS Code / Copilot install path). Marker (## graphify) is preserved so re-runs and vscode uninstall keep working. Existing repos that already have the soft section won't be auto-upgraded (install is a no-op when the marker is present), same behaviour as today. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
mvanderbend-msoft
force-pushed
the
strict-vscode-instructions-v2
branch
from
May 3, 2026 15:51
7d03335 to
a4da70b
Compare
Author
|
@safishamsi could you check this one out? |
Collaborator
|
Implemented directly in v6 with a slightly tighter version — same behaviour, trimmed the per-reply self-check. Thanks for the detailed writeup and repro! |
hypnwtykvmpr
pushed a commit
to hypnwtykvmpr/vampyre
that referenced
this pull request
May 7, 2026
…ce GRAPH_REPORT.md first Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
matzls
pushed a commit
to matzls/graphify
that referenced
this pull request
May 10, 2026
…ce GRAPH_REPORT.md first Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The default
## graphifysection thatgraphify vscode installwrites to.github/copilot-instructions.mdis too soft. In practice, Copilot agents (both VS Code Copilot Chat and the GitHub Copilot CLI) frequently bypass it — they answer architecture / how-to / where-is questions straight from memory or by grepping source, instead of readinggraphify-out/GRAPH_REPORT.mdfirst. That defeats the whole point of having the graph.What this PR changes
Replaces
_VSCODE_INSTRUCTIONS_SECTIONingraphify/__main__.pywith a more prescriptive version:graphify-out/GRAPH_REPORT.md.how do I…,where is…,add a handler,explain the architecture, etc.Why
Tested locally with the Copilot CLI on a Java REST API repo:
what's needed to add a new endpoint?without ever readingGRAPH_REPORT.md.GRAPH_REPORT.mdfirst and grounds its answer in god nodes / communities.Scope
_VSCODE_INSTRUCTIONS_SECTION(the VS Code / Copilot install path).## graphify) is still detected so re-runs andvscode uninstallkeep working. Note: existing repos that already have the soft section won't be auto-upgraded (the install is a no-op when the marker is present), same behaviour as today.Test
vscode install/vscode uninstallstill work end-to-end on a fresh repo and on a repo that already has the new section.