Skip to content

bug: Anthropic silently converts URL images to placeholder text — lossy and undocumented #64

@stackbilt-admin

Description

@stackbilt-admin

Bug

When a message contains an image specified by URL (rather than base64), the Anthropic provider silently replaces it with a text placeholder instead of throwing an error.

Location: src/providers/anthropic.ts:498–501

if (image.url) {
  return {
    type: 'text' as const,
    text: `[Image URL: ${image.url}]`
  };
}

The Anthropic native API does not support image content blocks with a URL source — only base64. This conversion is technically correct in that it avoids an API error, but it is lossy: the image is not processed at all. The caller receives a successful response from what is effectively a text-only request.

Why this is wrong

  • The LLMImageInput interface (src/types.ts) accepts both data (base64) and url — consumers reasonably expect URL images to work or fail explicitly
  • There is no warning or error surfaced to the caller
  • The behavior differs silently from OpenAI (which accepts image URLs natively)

Fix

Replace the silent conversion with a ConfigurationError:

if (image.url) {
  throw new ConfigurationError(
    'Anthropic does not support image URLs — convert to base64 before sending, ' +
    'or use an OpenAI-compatible provider for URL-based vision.'
  );
}

Alternatively, auto-fetch the URL and convert to base64 in the provider — but explicit failure is preferable so callers know what's happening.

Acceptance criteria

  • Passing a URL image to AnthropicProvider throws ConfigurationError (or auto-fetches and converts — decision left to implementor)
  • Behavior is documented in a JSDoc comment on the image-handling path
  • Test added for the URL image path on Anthropic

Found by

Codebase audit (automated) — src/providers/anthropic.ts:498–501

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions