Skip to content

Security/stability: cap LLM response bodies before reading into memory #48

Description

@CryptoJones

Problem

OpenAiCompatibleLlmClient reads successful LLM responses with ReadAsStringAsync, and the error path reads the entire body before truncating it for logs. A malicious, misconfigured, or broken endpoint can return an arbitrarily large body.

Impact

A single draft request can force large memory allocations and destabilize the API process.

References

  • api/ApplyTrack.Api/Llm/OpenAiCompatibleLlmClient.cs

Acceptance criteria

  • Use HttpCompletionOption.ResponseHeadersRead.
  • Reject responses with Content-Length over a configured/default cap.
  • Stream response bodies with a hard byte cap before JSON parsing/logging.
  • Tests cover oversized success and error responses.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingsecuritySecurity hardeningstabilityReliability and failure-mode hardening

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions