[Bug]: Telegram streaming + [[audio_as_voice]] causes TTS to render as plain audio attachment, not voice bubble

[muyidesignos-cpu]

[Bug]: Telegram streaming + [[audio_as_voice]] causes TTS to render as plain audio attachment, not voice bubble

Bug Description

When display.platforms.telegram.streaming: true (or the equivalent per-platform streaming override is enabled), the text_to_speech tool's output—[[audio_as_voice]]\nMEDIA:/path/to/audio.ogg—is delivered as a plain audio file attachment on Telegram, instead of as a voice bubble (the round, auto-playing message shape that sendVoice produces). The same TTS pipeline renders correctly as a voice bubble once Telegram streaming is disabled.

This contradicts the documented behaviour of tts.edge.voice_compatible: true and the [[audio_as_voice]] directive (see website/docs/user-guide/features/tts.md), which are explicitly meant to route audio as native Telegram voice messages.

Steps to Reproduce

1. Install Hermes v0.18.0 (commit 009b42d0) on macOS or Linux.
2. Configure the TTS provider with voice_compatible: true (e.g. tts.edge.voice_compatible: true) and ensure the binary ffmpeg is on PATH (Hermes converts MP3 → Opus/OGG for Telegram).
3. Enable Telegram streaming — either at the top level or via display.platforms.telegram.streaming: true.
4. Connect the Telegram gateway and chat with the agent.
5. Trigger a TTS response with a voice-bubble-eligible directive (e.g. issue a text_to_speech tool call whose result includes the [[audio_as_voice]]\nMEDIA:<path>.ogg payload).
6. Observe the message in Telegram — it arrives as a regular audio file attachment (tap-to-play) rather than a voice bubble.

Expected Behavior

The audio file should be delivered via Telegram Bot API sendVoice so the client renders it as a voice bubble (round shape, auto-playable). The tts_tool.text_to_speech response confirms the conversion succeeds:

{
  "success": true,
  "file_path": "/Users/bruce/.hermes/audio_cache/tts_*.ogg",
  "media_tag": "[[audio_as_voice]]\nMEDIA:...ogg",
  "provider": "edge",
  "voice_compatible": true
}

Actual Behavior

The audio arrives as a plain attachment (mp3 filename is stripped by Telegram's client-side MEDIA: tag rendering, and the OGG payload is delivered through sendMessage text parsing rather than sendVoice). gateway.log shows:

gateway.run: Suppressing normal final send for session ...: final delivery already confirmed (streamed=True previewed=False content_delivered=True).

adapter.send_voice (which would call bot.send_voice) is never invoked.

Affected Component

• Gateway (Telegram/Discord/Slack/WhatsApp)
• Tools (terminal, file ops, web, code execution, etc.)

Messaging Platform

• Telegram

Root Cause Analysis

TL;DR: the streaming send path emits the whole final_response (including the MEDIA:<path> tag and [[audio_as_voice]] directive) as one sendMessage text. Telegram's client then renders the MEDIA: reference as an audio attachment instead of using sendVoice. The post-stream media-dispatch loop in gateway/run.py (which already knows how to route .ogg → adapter.send_voice) is skipped because final delivery already confirmed is set on the streamed turn.

Evidence

1. tools/tts_tool.py:2393–2428 — When voice_compatible: true is set on a builtin TTS provider (e.g. edge) and HERMES_SESSION_PLATFORM=telegram, Hermes converts the MP3 → Opus/OGG via ffmpeg and emits [[audio_as_voice]]\nMEDIA:<path>.ogg as the media_tag. The text_to_speech return value confirms voice_compatible: true.

2. gateway/platforms/base.py:3610–3635 — extract_media reads the directive: has_voice_tag = "[[audio_as_voice]]" in content, and tags every (path, has_voice_tag) pair in media_files.

3. gateway/platforms/base.py:3477–3481 — filter_media_delivery_paths carries (path, is_voice=True) into safe_media, which is what non_image_media in run.py:12730 consumes.

4. gateway/run.py:12758–12778 — Post-stream dispatch should iterate non_image_media and call adapter.send_voice(audio_path=...) for .ogg/.opus when should_send_media_as_audio(platform=telegram, ext, is_voice=True) returns True. That call path is correct; the Telegram plugin (plugins/platforms/telegram/adapter.py:5704–5728) does call self._bot.send_voice for OGG.

5. gateway/run.py:19348–19356 — But this dispatch is skipped when _stream_confirmed_final_delivery is True (i.e. the streaming consumer already sent the response). The relevant log:

   Suppressing normal final send ... final delivery already confirmed (streamed=True previewed=False content_delivered=True).
   

6. gateway/run.py:16085–16089 — Streaming itself is enabled when display.platforms.<platform>.streaming is True (resolves via resolve_display_setting). With streaming on, GatewayStreamConsumer.on_delta (in gateway/stream_consumer.py) accumulates the assistant's streaming chunks, including [[audio_as_voice]]\nMEDIA:...ogg, and sends them as ordinary message text via adapter.send(...) (see stream_consumer.py:919, 1039, 1227, 1263, 1400, 1758). Telegram's client-side MEDIA: parser then renders the OGG path as a regular audio attachment.

Why disabling streaming fixes it

With display.platforms.telegram.streaming: false, the assistant's full response reaches the post-stream dispatch loop in gateway/run.py:12758, which routes the .ogg correctly through adapter.send_voice → bot.send_voice, and Telegram renders a voice bubble.

Proposed Fix

Two complementary changes are likely needed:

1. In gateway/stream_consumer.py, when accumulating streamed content, detect the [[audio_as_voice]] directive and suppress streaming the chunk that contains the MEDIA:<path> tag so that the post-stream dispatch loop (which routes through send_voice) gets a chance to fire. The simplest implementation: when [[audio_as_voice]] is observed, finish the current draft/edit, send [[audio_as_voice]]\nMEDIA:<path>.ogg through the non-streaming delivery path (extract_media → send_voice), and resume streaming the remainder of the response.

2. Alternatively, in gateway/run.py:19348 (the suppression guard), skip suppression specifically when media_files contains an entry with is_voice=True AND that voice-audio file hasn't already been delivered through a non-voice channel — falling through to the post-stream media dispatch instead.

Either approach keeps Telegram streaming benefits for normal text responses while restoring the documented [[audio_as_voice]] → voice-bubble behaviour. A regression test in tests/gateway/test_telegram_audio_vs_voice.py already covers the post-stream path; a new test should cover the streaming + voice path.

Environment

• Hermes version: v0.18.0 (upstream commit 009b42d0)
• OS: macOS 26.3.1 (reproduces on Linux too — ffmpeg behavior is platform-independent)
• Python: 3.11.15
• TTS provider: edge (edge-tts); ffmpeg 8.1.2
• Relevant config:

  tts:
    provider: edge
    edge:
      voice: zh-CN-XiaoxiaoNeural
      voice_compatible: true
  streaming:
    enabled: false
  display:
    platforms:
      telegram:
        streaming: true    # ← triggers the bug

Additional Notes

• A related but distinct issue exists with ffmpeg on macOS Homebrew after x265 upgrades break the dyld ABI (soname 215 → 216). brew reinstall ffmpeg (8.1.2+) resolves that and is a prerequisite for this bug to reproduce (otherwise voice_compatible falls back to MP3 regardless).
• This was discovered while debugging why Telegram voice-bubble delivery did not work even with tts.edge.voice_compatible: true. The post-stream send_voice path is healthy; the streaming consumer's MEDIA-tag-in-text approach is what suppresses it.

[teknium1]

Thanks for the detailed reproduction and root-cause analysis. This behavior is already fixed on current main.

• f58902818d9302fcdd4b30cc29cf2b9bc5f9f5f4 (fix(gateway): deliver MEDIA: files after streaming responses) shipped in v2026.3.23.
• gateway/run.py:12309-12315 invokes _deliver_media_from_response() even when streaming has already delivered the text.
• gateway/run.py:13388-13448 extracts [[audio_as_voice]] media and routes eligible audio through adapter.send_voice.
• plugins/platforms/telegram/adapter.py:6155-6180 sends .ogg/.opus through Telegram Bot API send_voice.
• The linked PR fix(gateway/streaming): deliver [[audio_as_voice]] via send_voice during streaming #60580 was reviewed as redundant against this same generalized post-stream delivery path.

Automated hermes-sweeper review.