PromptGuard

PromptGuard is a local-first prompt governance extension for VS Code. It brings linting, quality scoring, cost awareness, safety checks, and optimization previews to the prompt-writing workflow.

The current optimization workflow targets a local PromptGuard Proxy. The extension sends prompts to the proxy, receives a compressed prompt plus token-savings metadata, and shows that compressed prompt for manual copy/paste into VS Code Chat.

What it does

• Runs deterministic prompt analysis locally.
• Scores context, specificity, constraints, examples, formatting, safety, efficiency, and maintainability.
• Detects ambiguity, missing role/format/constraints, repetition, injection patterns, potential secrets, and PII.
• Estimates tokens, cost, latency, and savings; stores searchable local history.
• Warns while typing when workspace budget limits are exceeded and suggests fixes automatically.
• Detects duplicated ideas across semantic prompt blocks and suggests merges.
• Supports workspace, team, and global prompt templates with variables and snippet expansion.
• Validates workspace prompt policy rules from promptguard.json.
• Browses workspace policy packs from promptguard.policy-packs.json.
• Supports simplified workspace promptguard.json policies with max tokens, output requirements, secret blocking, and constraint checks.
• Validates workspace token, cost, and latency budgets from promptguard.budget.json.
• Budget mode warns live while typing when max tokens, cost, or latency are exceeded.
• Provides an in-extension PromptGuard Settings panel for common preferences without manual JSON edits.
• Browses workspace prompt templates from promptguard.templates.json.
• Stores opt-in local learning signals without raw prompt text.
• Runs workspace benchmark suites from promptguard.benchmarks.json.
• Exports workspace audit reports with team analytics into .promptguard/audit-reports.
• Browses opt-in provider guidance for Groq, OpenAI, Claude, and Gemini from promptguard.providers.json.
• Manages provider opt-in state in promptguard.providers.json.
• Exports browser-friendly and JetBrains-friendly prompt handoff artifacts with target-specific next steps.
• Provides editor decorations, hover explanations, quick fixes, a Git-style optimization diff view, sidebar navigation, and a Chart.js dashboard.
• Adds prompt analytics with average tokens, ambiguity, redundancy, quality, savings, cost, and trend charts.
• Adds @promptguard /analyze to VS Code Chat. It analyzes chat prompts locally and uses the selected model's tokenizer when the provider exposes one.
• Uses the local PromptGuard Proxy to return compressed_prompt, token savings, AST stats, and local history metadata instead of an LLM completion.
• Can fetch recent proxy history context with GET /history/context?limit=N and pin long-term memory entries with POST /history/memory.

Chat integration and model costs

In VS Code Chat, type @promptguard /analyze followed by the prompt you want to assess. Pressing Enter submits the request to PromptGuard and returns the score, safety findings, and token information inside Chat.

To prepare a prompt for manual use in VS Code Chat, run PromptGuard: Compress Prompt via Local Proxy. PromptGuard sends the prompt to the local proxy and shows a diff between the original and compressed prompt.

VS Code does not expose a global before-send/Enter interception API for other extensions' chat participants. That means PromptGuard cannot inspect, block, or alter a prompt sent directly to Copilot, Claude, Codex, or another provider. A regular extension can only handle chat turns it owns. This design intentionally protects chat-provider boundaries and user privacy.

When the selected chat model exposes countTokens, PromptGuard displays an exact input-token count. Pricing is provider- and contract-specific, so it is intentionally configured locally rather than guessed. Add a profile to promptguard.modelPricing, for example:

"promptguard.modelPricing": [
  { "match": "gpt-4.1", "inputPerMillionUsd": 2, "outputPerMillionUsd": 8 },
  { "match": "claude", "inputPerMillionUsd": 3, "outputPerMillionUsd": 15 }
]

PromptGuard matches match against VS Code's selected-model vendor, ID, and family. It calculates inputTokens × inputPrice + estimatedOutputTokens × outputPrice, divided by one million. If the model has no token counter it labels the input count as estimated; if no profile matches it shows cost as unavailable rather than inventing a price.

Local proxy optimization

PromptGuard now assumes a local-only PromptGuard Proxy architecture for optimization. The proxy does not forward prompts to remote LLM providers and does not return a provider completion. Instead, it returns:

• mode = "local_only"
• provider and model metadata
• compressed_prompt
• optimized_payload
• optimization stats such as original tokens, optimized tokens, saved tokens, compression ratio, per-pass savings, and latency
• ast counts for message and line nodes
• history metadata including the local JSONL path and counts
• history can be null when no history metadata is available
• instructions as a single newline-joined string for manual use

The extension uses that payload to power the local optimizer UI, the optimization preview, and dashboard/runtime status messaging. If the proxy is unreachable or returns malformed JSON, PromptGuard keeps local analysis available and reports the proxy failure without fabricating a completion.

The dashboard now uses push-based live mode. On startup it performs one lightweight GET /health check, then opens WS /dashboard/ws and waits for proxy-pushed connected and optimization_result events. If the socket drops, PromptGuard preserves the last rendered dashboard data, shows reconnecting status, and retries with exponential backoff. Continuous /stats or /metrics polling is no longer used in live mode; those endpoints are only used for manual refresh/debug actions.

Recent context is fetched newest-first (bottom-up) from the proxy history file. Long-term reusable notes can be saved back to the proxy memory endpoint from the local optimizer panel.

Cloud prompt logging onboarding

Prompt logging is opt-in. When promptguard.apiBaseUrl is configured, the first extension activation asks for permission to collect the user's email address, selected project context, original prompts, and any generated improved prompts. After consent, the extension verifies the email with the PromptGuard API's OTP flow, asks the user to create or choose a project, and stores the 24-hour bearer token in VS Code Secret Storage.

Configure the deployed API URL in VS Code settings:

"promptguard.apiBaseUrl": "https://your-promptguard-api.example"

The extension never contains the MongoDB URI or API email-provider secrets.

Install and run locally

npm install
npm run compile

Open this folder in VS Code and press F5 to start an Extension Development Host. Select prompt text (or open a prompt document) and run PromptGuard: Analyze Current Prompt from the Command Palette.

Commands

Command	Purpose
PromptGuard: Analyze Current Prompt	Runs local rules and opens analytics.
PromptGuard: Compress Prompt via Local Proxy	Sends the prompt to the local proxy and opens the compressed-prompt preview with savings metrics.
PromptGuard: Open Dashboard	Shows quality, costs, rule findings, and live proxy updates from the dashboard websocket, with manual refresh available for debug fallback.
PromptGuard: Settings	Opens the PromptGuard settings panel for preferences and runtime toggles.
PromptGuard: Show History	Searches local prompt snapshots.
PromptGuard: Browse Policy Packs	Opens the workspace policy pack catalog.
PromptGuard: Run Benchmark Suites	Evaluates workspace benchmark cases and opens a markdown report.
PromptGuard: Export Audit Report	Writes a markdown audit export and opens it in the editor.
PromptGuard: Browse Opt-In Providers	Opens guidance for Groq, OpenAI, Claude, and Gemini provider setup.
PromptGuard: Manage Provider Opt-In	Enables or disables OpenAI, Claude, or Gemini in the workspace catalog.
PromptGuard: Browse Prompt Templates	Reviews reusable templates, scopes, and repeated prompt prefixes.
PromptGuard: Insert Template Snippet	Inserts a template with snippet placeholders into the active editor.
PromptGuard: Convert Current Prompt to Template	Generates a reusable template from a repeated prompt prefix.
PromptGuard: Export Prompt Handoff	Exports browser or JetBrains handoff artifacts for the current prompt.
PromptGuard: Open Duplicate Detection	Shows similar semantic blocks, estimated savings, and a merge suggestion.

Workspace Policy

Create a promptguard.json file in the workspace root to enforce organization policy rules. The simplified format below is supported and is checked automatically by the editor lint pipeline and by PromptGuard: Validate Workspace Policy.

{
  "maxTokens": 1000,
  "requireOutput": true,
  "forbidSecrets": true,
  "requireConstraints": true
}

maxTokens flags prompts that are too long, requireOutput requires an explicit output shape, forbidSecrets blocks secret-like strings, and requireConstraints requires an explicit constraint or requirement section.

Settings Panel

Use PromptGuard: Settings to manage common preferences like analysis mode, minimum prompt length, learning store, and budget mode.

Architecture

flowchart LR
  E[VS Code editor] --> A[PromptAnalyzer]
  A --> R[Independent rules]
  A --> S[Scorer]
  A --> C[Cost estimator]
  A --> O[Optimizer]
  E --> LP[Local PromptGuard Proxy]
  LP --> O
  LP --> H2[History context and memory]
  A --> U[Decorations / Dashboard / Diff]
  A --> H[Memento history]
  RP[Future rule plugins] --> A
  M[Future AI providers with consent] --> O

Loading

PromptGuard now uses a deterministic prompt AST internally before analysis. The AST keeps the raw prompt compatible with existing rules while preparing the codebase for section-aware linting, duplicate detection, and structured optimization in later modules.

Rules live in src/heuristics and implement PromptRule; integrations implement ModelProvider. The extension's optimization path now targets the local PromptGuard Proxy contract, while policy controls, CI linting, and marketplace plugins stay behind stable seams.

Automation

• Run npm run check to execute typecheck, compile, and tests in the same order used by local hooks and CI.
• Run npm run install-hooks once to configure Git to use the repository hooks in .githooks.
• GitHub Actions runs the same checks from .github/workflows/ci.yml on push and pull request.

Screenshots

Dashboard and optimization-diff screenshots will be added before marketplace release.

Roadmap

• Prompt benchmark suites and evaluation criteria
• Enterprise policy packs, team analytics, and audit exports
• Opt-in OpenAI, Claude, and Gemini providers
• Git hooks, CI/CD prompt linting, and GitHub Actions
• Browser and JetBrains extensions

Contributing

Run npm run typecheck and npm run compile before submitting a pull request. Add rules as isolated classes, avoid network access in the analysis path, and maintain strict TypeScript types.

License

MIT. See LICENSE.