spec: Extend Activity Schema to Support Multimodal Interactions with Streaming#468
Merged
spec: Extend Activity Schema to Support Multimodal Interactions with Streaming#468
Conversation
…416 Co-authored-by: gurubhg <89311725+gurubhg@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Update schema based on final proposal
Align Activity Schema spec with final multimodal proposal (issue #416)
Mar 2, 2026
Copilot
AI
changed the title
Align Activity Schema spec with final multimodal proposal (issue #416)
Extend Activity Protocol to v3.4: Multimodal Streaming and Session Lifecycle
Mar 2, 2026
Copilot
AI
changed the title
Extend Activity Protocol to v3.4: Multimodal Streaming and Session Lifecycle
Activity Protocol v3.4: Multimodal streaming and session lifecycle
Mar 2, 2026
Co-authored-by: gurubhg <89311725+gurubhg@users.noreply.github.com>
Copilot
AI
changed the title
Activity Protocol v3.4: Multimodal streaming and session lifecycle
Move Voice message spec section from Event activity to Message activity
Mar 2, 2026
…l interaction flow Co-authored-by: gurubhg <89311725+gurubhg@users.noreply.github.com>
Copilot
AI
changed the title
Move Voice message spec section from Event activity to Message activity
Fix multimodal voice spec: move Voice message to Message activity and cross-link sections
Mar 2, 2026
…n both directions Co-authored-by: gurubhg <89311725+gurubhg@users.noreply.github.com>
Copilot
AI
changed the title
Fix multimodal voice spec: move Voice message to Message activity and cross-link sections
spec: clarify bidirectionality of Voice message and Media streaming events in activity schema
Mar 2, 2026
…ot client-to-Agent only Co-authored-by: gurubhg <89311725+gurubhg@users.noreply.github.com>
Copilot
AI
changed the title
spec: clarify bidirectionality of Voice message and Media streaming events in activity schema
spec: fix bidirectionality of Voice message and Media streaming events
Mar 2, 2026
Co-authored-by: gurubhg <89311725+gurubhg@users.noreply.github.com>
Copilot
AI
changed the title
spec: extend Activity Protocol 3.4 with multimodal voice/streaming support
spec: Update PR description with structured multimodal streaming summary
Mar 2, 2026
gurubhg
changed the title
Copilot
AI
changed the title
Extend Activity Schema to Support Multimodal Interactions with Streaming
spec: Extend Activity Protocol to support multimodal interactions with streaming (Provisional 3.4)
Mar 2, 2026
…o + Changes + Key design decisions + Related) Co-authored-by: gurubhg <89311725+gurubhg@users.noreply.github.com>
Copilot
AI
changed the title
spec: Extend Activity Protocol to support multimodal interactions with streaming (Provisional 3.4)
spec: Rewrite Appendix I changelog entry for multimodal streaming (PR #468)
Mar 2, 2026
Copilot
AI
changed the title
spec: Rewrite Appendix I changelog entry for multimodal streaming (PR #468)
spec: Update PR description for PR #468 to match PR #423 structured format
Mar 2, 2026
github-actions
Bot
added
the
Specs
gurubhg
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the Activity Protocol Activity spec to document multimodal voice/audio interactions with real-time streaming semantics while aiming to remain backward compatible (no new activity types or schema fields).
Changes:
- Adds a
Voice messagepattern usingmessage.valueType+message.valuefor single-activity audio payloads. - Introduces reserved media streaming events (
Media.Start,Media.Chunk,Media.End) and ties them to the existingstreamInfo/streaminfoentity for sequencing. - Adds
session.init/session.update/session.endsession lifecycle commands and a multimodal interaction flow description.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
tracyboehrer
approved these changes
Mar 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR implements the approved proposal from issue #416 to document multimodal
voice/audio interactions with streaming support in the Activity Protocol spec.
Changes:
Key design decisions (per AP Core Committee):
Related: #416