Skip to content

[Go] Base64 media content duplicated in ChatCompletion messages #3767

@shyer

Description

@shyer

Describe the bug
When sending a message containing a base64 image (e.g., data:image/jpeg;base64,...) using NewMediaPart, Genkit duplicates the content in the OpenAI ChatCompletion request:

  1. Once as a normal user message (UserMessage)
  2. Again as an ImageContentPart

This happens because the same media part is added both to the text content and the image content part.

To Reproduce
Steps to reproduce the behavior. Code samples.

https://genkit.dev/docs/models/?lang=go#multimodal-input-1

image, err := os.ReadFile("photo.jpg")
if err != nil {
    log.Fatal(err)
}

resp, err := genkit.Generate(ctx, g,
    ai.WithModelName("openai/gpt-4.1-nano"),
    ai.WithMessages(
        ai.NewUserMessage(
            ai.NewMediaPart("image/jpeg", "data:image/jpeg;base64,"+base64.StdEncoding.EncodeToString(image)),
            ai.NewTextPart("Compose a poem about this image."),
        ),
    ),
)

Expected behavior
Base64 image content should appear only once in the ChatCompletion request, as an ImageContentPart. It should not be duplicated as normal user text.

Screenshots
If applicable, add screenshots to help explain your problem.
base64 content in role: user and image_url is same content

Image

Runtime (please complete the following information):

  • OS: [e.g. Linux, MacOS]
  • Version [e.g. 22]

** Go version

  • run go version at paste here

Additional context
Add any other context about the problem here.

https://github.com/firebase/genkit/blob/main/go/plugins/compat_oai/generate.go#L116-L135

			oaiMessages = append(oaiMessages, openai.UserMessage(content))

			parts := []openai.ChatCompletionContentPartUnionParam{}
			for _, p := range msg.Content {
				if p.IsMedia() {
					part := openai.ImageContentPart(
						openai.ChatCompletionContentPartImageImageURLParam{
							URL: p.Text,
						})
					parts = append(parts, part)
					continue
				}
			}
			if len(parts) > 0 {
				oaiMessages = append(oaiMessages, openai.ChatCompletionMessageParamUnion{
					OfUser: &openai.ChatCompletionUserMessageParam{
						Content: openai.ChatCompletionUserMessageParamContentUnion{OfArrayOfContentParts: parts},
					},
				})
			}

Metadata

Metadata

Assignees

Labels

bugSomething isn't workinggo

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions