Skip to content

Make default VS Code Copilot graphify instructions stricter so agents stop bypassing GRAPH_REPORT.md#688

Closed
mvanderbend-msoft wants to merge 1 commit into
Graphify-Labs:v6from
mvanderbend-msoft:strict-vscode-instructions-v2
Closed

Make default VS Code Copilot graphify instructions stricter so agents stop bypassing GRAPH_REPORT.md#688
mvanderbend-msoft wants to merge 1 commit into
Graphify-Labs:v6from
mvanderbend-msoft:strict-vscode-instructions-v2

Conversation

@mvanderbend-msoft

Copy link
Copy Markdown

The default ## graphify section that graphify vscode install writes to .github/copilot-instructions.md is too soft. In practice, Copilot agents (both VS Code Copilot Chat and the GitHub Copilot CLI) frequently bypass it — they answer architecture / how-to / where-is questions straight from memory or by grepping source, instead of reading graphify-out/GRAPH_REPORT.md first. That defeats the whole point of having the graph.

What this PR changes

Replaces _VSCODE_INSTRUCTIONS_SECTION in graphify/__main__.py with a more prescriptive version:

  • Mandatory first tool call: for any architecture/codebase question, the agent's first tool call must be reading graphify-out/GRAPH_REPORT.md.
  • Explicit trigger list: how do I…, where is…, add a handler, explain the architecture, etc.
  • Trust the graph: explicit instruction not to re-verify by opening source files after the graph for conceptual questions.
  • Narrow allowlist: source reads are still allowed for actual code modification / debugging / when the graph is genuinely missing detail or is stale.
  • Self-check: the agent is told to verify before each reply that its first tool call read the report.

Why

Tested locally with the Copilot CLI on a Java REST API repo:

  • With the soft wording, the agent answered what's needed to add a new endpoint? without ever reading GRAPH_REPORT.md.
  • With the stricter wording, the agent consistently reads GRAPH_REPORT.md first and grounds its answer in god nodes / communities.

Scope

  • Only touches _VSCODE_INSTRUCTIONS_SECTION (the VS Code / Copilot install path).
  • Does not touch the Antigravity, Kiro, Aider, Claude, Codex, etc. templates — those can be aligned in a follow-up if this approach is accepted.
  • Does not change install/uninstall logic; the marker (## graphify) is still detected so re-runs and vscode uninstall keep working. Note: existing repos that already have the soft section won't be auto-upgraded (the install is a no-op when the marker is present), same behaviour as today.

Test

vscode install / vscode uninstall still work end-to-end on a fresh repo and on a repo that already has the new section.

…g GRAPH_REPORT.md

The previous default copilot-instructions.md section was too soft. In practice Copilot agents (both VS Code Copilot Chat and the GitHub Copilot CLI) frequently bypassed it - they answered architecture/how-to/where-is questions straight from memory or by grepping source, instead of reading graphify-out/GRAPH_REPORT.md first. That defeated the point of the graph.

This change replaces _VSCODE_INSTRUCTIONS_SECTION with a more prescriptive version:

- Mandatory first tool call: for any architecture/codebase question, the agent's first tool call must be reading GRAPH_REPORT.md.

- Explicit trigger list (how do I, where is, add a handler, explain the architecture, etc.) so the agent recognises when the rule fires.

- Trust the graph: explicit instruction NOT to re-verify by opening source files after the graph for conceptual questions.

- Narrow allowlist: source reads still allowed for actual code modification / debugging / when the graph genuinely lacks detail or is stale.

- Self-check: the agent is told to verify before each reply that its first tool call read the report.

Tested locally with the Copilot CLI on a Java REST API repo: with the soft wording the agent answered 'what's needed to add a new endpoint?' without ever reading GRAPH_REPORT.md; with the stricter wording it consistently reads the report first and grounds its answer in god nodes / communities.

Scope: only touches _VSCODE_INSTRUCTIONS_SECTION (the VS Code / Copilot install path). Marker (## graphify) is preserved so re-runs and vscode uninstall keep working. Existing repos that already have the soft section won't be auto-upgraded (install is a no-op when the marker is present), same behaviour as today.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mvanderbend-msoft
mvanderbend-msoft force-pushed the strict-vscode-instructions-v2 branch from 7d03335 to a4da70b Compare May 3, 2026 15:51
@mvanderbend-msoft

Copy link
Copy Markdown
Author

@safishamsi could you check this one out?

@safishamsi

Copy link
Copy Markdown
Collaborator

Implemented directly in v6 with a slightly tighter version — same behaviour, trimmed the per-reply self-check. Thanks for the detailed writeup and repro!

@safishamsi safishamsi closed this in f61bb27 May 3, 2026
hypnwtykvmpr pushed a commit to hypnwtykvmpr/vampyre that referenced this pull request May 7, 2026
…ce GRAPH_REPORT.md first

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
matzls pushed a commit to matzls/graphify that referenced this pull request May 10, 2026
…ce GRAPH_REPORT.md first

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants