Skip to content

Input hardening against agent hallucinations #350

@BYK

Description

@BYK

Summary

Harden input validation across the CLI to defend against agent hallucinations — malformed inputs that are plausible mistakes for an LLM but unlikely from a human.

Motivation

From You Need to Rewrite Your CLI for AI Agents (Justin Poehnelt, Google DevRel):

"Humans typo. Agents hallucinate. The failure modes are completely different."

"An agent might embed ?fields=name inside a resource ID — has happened. An agent might pass a pre-URL-encoded string that gets double-encoded — common."

The agent is not a trusted operator. The CLI should be the last line of defense against hallucinated inputs.

Current state

We already have good validation in places:

  • ✅ Prototype pollution protection in sentry api field keys (__proto__, constructor, prototype)
  • ✅ Trace ID format validation (strict 32-char hex regex)
  • ✅ Issue ID parsing with format-specific error messages
  • ✅ Limit bounds checking
  • ✅ HTTP method whitelist in sentry api

Gaps

1. Resource ID injection

Org slugs, project slugs, and issue IDs are interpolated into URLs but not checked for ?, #, %, or whitespace. An agent could hallucinate:

sentry issue list "my-org?query=foo"     # query injection
sentry project view "my-project#anchor"  # fragment injection
sentry issue view "CLI-G%20extra"        # pre-encoded space

Fix: Add a validateSlug() helper that rejects ?, #, %, whitespace, and control characters in org/project slugs. Apply it at the CLI argument parsing layer.

2. Control characters

No validation against ASCII control characters (below 0x20) in any string input. An agent could embed invisible characters in arguments.

Fix: Add a rejectControlChars() check to string inputs.

3. sentry api endpoint path traversal

The endpoint positional in sentry api is not checked for .. segments:

sentry api "../../admin/settings/"  # path traversal

While the Sentry API server handles this safely (it would just 404), rejecting .. on the client side provides defense-in-depth and better error messages.

Fix: Reject endpoints containing .. path segments in normalizeEndpoint().

4. Double URL encoding

Pre-encoded strings (%2F, %3A) passed as arguments could get double-encoded when interpolated into URLs.

Fix: Detect and reject %XX patterns in resource identifiers (slugs, IDs) — these should always be plain strings.

Implementation approach

Create a shared src/lib/input-validation.ts module with reusable validators:

/** Reject control characters (below ASCII 0x20) */
export function rejectControlChars(input: string, label: string): void;

/** Reject query injection characters in slugs and IDs */
export function validateResourceId(input: string, label: string): void;

/** Reject path traversal in API endpoints */  
export function validateEndpoint(endpoint: string): void;

/** Reject pre-URL-encoded strings */
export function rejectPreEncoded(input: string, label: string): void;

Apply these at the argument parsing layer (src/lib/arg-parsing.ts) so all commands benefit automatically.

Priority

Low-medium. The Sentry API handles most of these server-side (404, 400 responses). This is defense-in-depth — improving error messages and catching problems earlier. The most impactful item is #1 (resource ID injection) since it could cause confusing API errors.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions