Skip to content

feat: add verified-code-answers plugin to fix unverified assertions (fixes #29753)#29755

Open
HarshalJain-cs wants to merge 4 commits intoanthropics:mainfrom
HarshalJain-cs:fix/verified-code-answers-plugin
Open

feat: add verified-code-answers plugin to fix unverified assertions (fixes #29753)#29755
HarshalJain-cs wants to merge 4 commits intoanthropics:mainfrom
HarshalJain-cs:fix/verified-code-answers-plugin

Conversation

@HarshalJain-cs
Copy link
Copy Markdown

Summary

Fixes #29753 — Claude Code presents unverified assertions with the same confidence as verified ones.

Problem

When Claude Code answers a question about code behavior, there is no discernible difference in tone, confidence, or presentation between:

  1. An answer that was verified by reading the actual source code with tools
  2. An answer that was pattern-matched from training data without checking

The existing system prompt instruction "Do not say things you haven't verified" is insufficient on its own, as Claude still generates confident unverified answers.

Solution

This PR adds a new plugin verified-code-answers that injects a SessionStart hook with three explicit behavioral rules:

Rule 1: Read Before You Assert

Claude must use file reading tools (Read, Glob, Grep) to inspect actual source code before answering any factual question about code behavior.

Rule 2: Caveat Unverified Answers

If Claude cannot read the relevant source file, it must explicitly caveat its answer (e.g., "I haven't read the source file, but based on common patterns...").

Rule 3: Never Present Unverified Information as Verified

Code-behavior questions must never be answered with the same confidence as verified answers unless the code was actually read.

Files Added

plugins/verified-code-answers/
  .claude-plugin/plugin.json       # Plugin metadata
  hooks/hooks.json                  # SessionStart hook configuration
  hooks-handlers/session-start.sh  # Core instruction injection script
  README.md                         # Documentation

Testing

This plugin works identically to the existing explanatory-output-style plugin in structure and hook mechanism. Install by copying to your Claude Code plugins directory.

Adds a new plugin that injects a SessionStart hook instructing Claude to
always read source files before answering factual questions about code
behavior. If Claude cannot read a file, it must explicitly caveat the
answer as unverified.

Fixes: anthropics#29753
This script implements a SessionStart hook for the Verified Code Answers plugin, providing guidelines for verifying code behavior before answering user questions.
This README introduces the Verified Code Answers Plugin, detailing its purpose, rules for code behavior assertions, installation instructions, and file structure.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Claude Code presents unverified assertions with same confidence as verified ones

1 participant