Skip to content

Python: Forward GitHub Copilot input attachments as inline blobs#7300

Merged
giles17 merged 3 commits into
microsoft:mainfrom
giles17:copilot-attachments
Jul 23, 2026
Merged

Python: Forward GitHub Copilot input attachments as inline blobs#7300
giles17 merged 3 commits into
microsoft:mainfrom
giles17:copilot-attachments

Conversation

@giles17

@giles17 giles17 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Motivation & Context

The Python GitHubCopilotAgent built the outgoing prompt purely from message
text, so any DataContent (images, documents, and other inline binary content)
passed on input was silently dropped — a user attaching an image received no
error and nothing reached Copilot. This closes that functional gap and removes
the silent data loss ahead of the 1.0 release.

Description & Review Guide

  • What are the major changes?

    • Added RawGitHubCopilotAgent._prepare_attachments_for_copilot(...), which
      scans outgoing messages for data content and maps each to the Copilot SDK's
      inline BlobAttachment (base64 payload + MIME type).
    • Wired attachments into both send paths: send_and_wait(...) (non-streaming)
      and send(...) (streaming).
    • Only base64 data: content is forwarded. Data content with no media type,
      and malformed / non-base64 data: URIs, are skipped with a warning rather
      than raising (a non-base64 data: URI is classified as type="data" but has
      no base64 payload, so extracting its bytes would otherwise error).
    • Remote URIs (e.g. https:// links) are not forwarded and are not added to
      the prompt, which is built from text content only.
    • Added unit tests: non-streaming, streaming, text-only (no attachments),
      missing media type, and the non-base64 data: URI case.
  • What is the impact of these changes?

    • Image/document attachments now reach GitHub Copilot. No public API change —
      the behavior activates automatically for messages that already carry
      DataContent. Non-breaking.
  • What do you want reviewers to focus on?

    • The mapping of data content to BlobAttachment, and the handling of
      unsupported/edge content (skip-with-warning rather than raising).

Related Issue

No existing tracking issue yet. Happy to open one and link it with a closing
keyword if preferred.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

The Python GitHubCopilotAgent built the prompt from message text only, so
DataContent (images/documents) passed on input was silently dropped. The .NET
provider already forwards these as attachments.

Map input data content to the Copilot SDK's inline BlobAttachment (base64,
no temp files) in both the streaming and non-streaming send paths. Data content
without a media type is dropped with a warning instead of silently.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ac85c429-4115-42ef-a18a-f576e3cf03f7
Copilot AI review requested due to automatic review settings July 23, 2026 18:15
@giles17
giles17 temporarily deployed to github-app-auth July 23, 2026 18:15 — with GitHub Actions Inactive
@giles17
giles17 temporarily deployed to github-app-auth July 23, 2026 18:15 — with GitHub Actions Inactive
@giles17
giles17 temporarily deployed to github-app-auth July 23, 2026 18:15 — with GitHub Actions Inactive
@agent-framework-automation agent-framework-automation Bot added the python Usage: [Issues, PRs], Target: Python label Jul 23, 2026
@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
packages/github_copilot/agent_framework_github_copilot
   _agent.py4272294%67–68, 110, 119–121, 125, 546, 561–562, 641, 654, 790, 794, 962, 996–997, 1036, 1039, 1176, 1250, 1268
TOTAL45564446990% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
9397 33 💤 0 ❌ 0 🔥 2m 27s ⏱️

@giles17
giles17 marked this pull request as ready for review July 23, 2026 18:19
@giles17
giles17 temporarily deployed to github-app-auth July 23, 2026 18:20 — with GitHub Actions Inactive

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds Python GitHub Copilot provider support for forwarding inline binary DataContent (e.g., images/documents) as Copilot SDK inline blob attachments, closing parity with the .NET Copilot provider and preventing silent attachment loss.

Changes:

  • Add _prepare_attachments_for_copilot(...) to scan outgoing Message.contents for data content and convert it to Copilot SDK BlobAttachment objects (base64 payload + MIME type).
  • Wire attachment forwarding into both non-streaming (send_and_wait) and streaming (send) request paths.
  • Add unit tests covering non-streaming, streaming, text-only (no attachments), and missing-media-type behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
python/packages/github_copilot/agent_framework_github_copilot/_agent.py Builds and passes Copilot SDK attachments from inline data content for both streaming and non-streaming runs.
python/packages/github_copilot/tests/test_github_copilot_agent.py Adds test coverage validating blob attachment forwarding and the missing-media-type behavior.

Comment thread python/packages/github_copilot/agent_framework_github_copilot/_agent.py Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Code Review

Reviewers: 5 | Confidence: 90% | Result: All clear

Reviewed: Correctness, Security Reliability, Test Coverage, Failure Modes, Design Approach


Automated review by giles17's agents

Address PR review feedback:
- Guard _get_data_bytes_as_str against ContentError so a non-base64 data:
  URI (which _validate_uri still classifies as type="data") is skipped with a
  warning instead of failing the entire Copilot request.
- Correct the docstring: remote URIs and non-base64 data URIs are neither
  attached nor added to the prompt (the prompt is built from text content only).
- Add tests for the non-base64 data URI path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ac85c429-4115-42ef-a18a-f576e3cf03f7
@giles17
giles17 temporarily deployed to github-app-auth July 23, 2026 18:30 — with GitHub Actions Inactive
@giles17
giles17 enabled auto-merge July 23, 2026 18:31
@giles17
giles17 temporarily deployed to github-app-auth July 23, 2026 18:32 — with GitHub Actions Inactive
@giles17
giles17 temporarily deployed to github-app-auth July 23, 2026 18:33 — with GitHub Actions Inactive
The end-to-end non-base64 data URI test failed in CI because GitHubCopilotAgent's
telemetry layer serializes message content (observability._to_otel_part ->
_get_data_bytes_as_str), which raises ContentError on a non-base64 data: URI
before the attachment code runs. That is an unrelated core-observability
limitation, not attachment behavior.

Use RawGitHubCopilotAgent (no telemetry layer) for that test so it isolates the
provider's send path. The direct helper test still covers the ContentError guard.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ac85c429-4115-42ef-a18a-f576e3cf03f7
@giles17
giles17 added this pull request to the merge queue Jul 23, 2026
Merged via the queue into microsoft:main with commit 8c05750 Jul 23, 2026
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants