Skip to content

[Feat] Support audio requests across chat providers - #1173

Merged
mrubens merged 3 commits into
developfrom
feature/chat-provider-audio-1axnlui81g7xq
Aug 8, 2026
Merged

[Feat] Support audio requests across chat providers#1173
mrubens merged 3 commits into
developfrom
feature/chat-provider-audio-1axnlui81g7xq

Conversation

@roomote-roomote

@roomote-roomote roomote-roomote Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Opened on behalf of Matt Rubens. Follow up by mentioning @roomote-roomote, in the web UI, or in Slack.

What changed

Added inbound audio and voice-note transcription for Telegram, Discord, and Teams. Each provider now recognizes audio-only task entries and follow-ups, downloads media through its authenticated bounded transport, and appends the shared transcript or actionable warning to the task prompt.

The shared transcription layer normalizes provider MIME metadata and owns consistent transcript, unsupported-model, oversized-media, and failure messages. Extension inference is limited to absent or generic MIME metadata so explicitly typed video and other non-audio attachments never enter audio transcription. Teams file-download wrappers use their nested content type, filename, or file type so uploaded audio is still recognized correctly. Telegram and Teams attachment downloads also enforce byte limits while streaming rather than buffering an unbounded response.

Why this change was made

Audio requests worked in Slack but were ignored or reduced to unusable attachment placeholders in the other chat providers. This makes voice-driven task entry consistent across supported communication surfaces without exposing provider credentials or signed attachment URLs to agents.

Impact

Users can start tasks and send active follow-ups with Telegram voice/audio messages, Discord audio attachments, and Teams audio attachments, including normal Teams file uploads. Unsupported models, media above 20 MiB, unsupported formats, and download failures remain actionable text instead of producing an empty request, while video uploads retain their explicit media classification.

@roomote-community

roomote-community Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

No new code issues found. See task

  • Video MIME types can be relabeled as audio by filename fallback (packages/cloud-agents/src/server/audio-transcription.ts:55).
  • Teams file-download audio attachments are ignored because their wrapper MIME type disables filename inference (packages/communication/src/teams-activity.ts:590).

Reviewed f2f73c2

Comment thread packages/cloud-agents/src/server/audio-transcription.ts
@roomote-roomote

roomote-roomote Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the Teams file-download wrapper feedback in f2f73c2. Wrapper attachments now use nested MIME, filename, or fileType metadata to recognize uploaded audio, while explicit nested video MIME types remain excluded. Targeted tests and the full pre-push checks pass.

Comment thread packages/communication/src/teams-activity.ts Outdated
@mrubens
mrubens marked this pull request as ready for review August 8, 2026 03:46
@mrubens
mrubens merged commit a54d880 into develop Aug 8, 2026
17 of 18 checks passed
@mrubens
mrubens deleted the feature/chat-provider-audio-1axnlui81g7xq branch August 8, 2026 03:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant