Skip to content

OpenAI Chat transforms silently drops :document shorthand #331

@TheRealNeil

Description

@TheRealNeil

Summary

ActiveAgent::Providers::OpenAI::Chat::Transforms.normalize_message handles the :image shorthand but not :document. When a message hash like { document: "data:application/pdf;base64,..." } is passed, the document data is silently dropped — it ends up as a top-level key on the message hash rather than becoming a content block.

This affects both the OpenAI Chat and OpenRouter providers (which inherits from OpenAI Chat).

Steps to reproduce

# Image shorthand works correctly
img = { image: "data:image/png;base64,abc" }
result = ActiveAgent::Providers::OpenAI::Chat::Transforms.normalize_message(img)
# => { role: "user", content: [{ type: "image_url", image_url: { url: "data:image/png;base64,abc" } }] }

# Document shorthand is silently dropped
doc = { document: "data:application/pdf;base64,abc" }
result = ActiveAgent::Providers::OpenAI::Chat::Transforms.normalize_message(doc)
# => { role: "user", document: "data:application/pdf;base64,abc" }
#    ^ document data lost — not in content array

Root cause

In lib/active_agent/providers/open_ai/chat/transforms.rb, normalize_message (around line 118) has shorthand handling for :text and :image but no branch for :document:

content = if msg_hash.key?(:content)
  msg_hash[:content]
elsif msg_hash.key?(:text) && msg_hash.key?(:image)
  # ...
elsif msg_hash.key?(:image)
  [ { type: "image_url", image_url: { url: msg_hash[:image] } } ]
elsif msg_hash.key?(:text)
  msg_hash[:text]
else
  nil  # <-- :document falls through here
end

The unrecognised :document key then gets passed as extra_params on line 140:

extra_params = msg_hash.except(:role, :content, :text, :image)

Affected providers

Provider :image :document
Anthropic
OpenAI Responses
OpenAI Chat ❌ silently dropped
OpenRouter (extends OpenAI Chat) ❌ inherited bug

Note: ActiveAgent::Base#prompt explicitly extracts both :image and :document from options (line 217 of base.rb), and the OpenRouter ContentType class correctly handles { document: ... } at the content-part level. It's only the message-level normalisation bridge that's missing.

Expected behaviour

normalize_message({ document: "data:application/pdf;base64,..." }) should produce a properly structured message with the document as a content block, consistent with how :image is handled.

Versions

  • activeagent 1.0.1
  • openai 0.59.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions