Feature: Gemini TTS style prompts and controlled audio tags

[barronlroth]

Summary

Gemini TTS is prompt-controllable, but Hermes currently treats it mostly like a plain text-to-speech backend: it passes the requested text directly into generateContent and only configures the model/voice. It would be useful to add first-class Gemini-only voice direction support so users can define a stable spoken persona and optionally pass safe audio tags.

This is especially useful for gateway voice replies, where the assistant's written personality does not automatically transfer to generated speech.

Current behavior

In tools/tts_tool.py, Gemini TTS currently:

• defaults to gemini-2.5-flash-preview-tts
• defaults to voice Kore
• reads optional tts.gemini.model, tts.gemini.voice, and tts.gemini.base_url
• sends the raw requested text as contents[0].parts[0].text
• notes in comments that Gemini supports prompt-style directions and tags such as [whispers]

But there is no dedicated config surface for:

• a reusable style/persona prompt
• loading voice instructions from a file
• policy-controlled audio tag passthrough or injection

Proposed feature

Add Gemini-specific TTS controls under tts.gemini, for example:

tts:
  provider: gemini
  gemini:
    model: gemini-3.1-flash-tts-preview
    voice: Algieba
    style_prompt_file: /Users/example/.hermes/tts/voice-style.md
    audio_tags:
      enabled: true
      passthrough: true
      auto_inject: conservative

1. style_prompt_file

If configured, Hermes would read a local text/markdown file and prepend its contents to the Gemini TTS prompt before the spoken text.

Example prompt construction:

<contents of style_prompt_file>

Speak the following message:
<assistant text>

This lets users maintain a stable spoken persona separately from the chat system prompt. Example use cases:

• dry British butler voice
• concise radio-host voice
• calm coaching voice
• accessibility-oriented slow/clear narration

Recommended behavior:

• Gemini-only initially; do not alter Edge/OpenAI/ElevenLabs/Mistral semantics.
• Fail soft if file is missing/unreadable: log warning and continue without the style prompt, or surface a clear TTS error depending on existing provider convention.
• Apply provider character/token limits after prompt composition, with a warning if truncation happens.
• Keep the style prompt local only; never include it in the visible chat reply.

2. Conservative audio tag support

Gemini TTS supports direction/tags in text prompts. Hermes could expose a safe, explicit policy for this instead of relying on ad-hoc prompt text.

Suggested options:

• audio_tags.enabled: master switch
• audio_tags.passthrough: preserve intentional tags already present in assistant text, e.g. [whispers]
• audio_tags.auto_inject: off | conservative

conservative should avoid theatrical overuse. The goal is not to sprinkle [sighs] everywhere; it is to support subtle voice direction when appropriate.

Possible examples:

Speak in a composed, dry, understated tone with restrained warmth.

or, only when explicitly requested by the assistant/user:

[whispers] Keep your voice low and confidential.

Why this belongs in Hermes

Hermes already has a personality/system prompt for text. Voice replies currently lose much of that character because TTS providers receive only the final text. Gemini is unusually capable here because its TTS API is prompt-controllable, so Hermes can expose a small provider-specific config surface without making every TTS backend pretend to support the same features.

This would make gateway voice replies feel much less generic while keeping the implementation simple and opt-in.

Implementation notes

Likely area:

• tools/tts_tool.py, _generate_gemini_tts(...)
• config defaults/documentation under hermes_cli/config.py / setup docs if desired

Possible prompt assembly:

1. Resolve Gemini config: model, voice, base_url, style_prompt_file, audio_tags.
2. Read style prompt if configured.
3. Build a Gemini-only prompt_text from style instructions + final assistant text.
4. Pass prompt_text into contents[0].parts[0].text.
5. Preserve existing output conversion flow.

Open questions

• Should style_prompt_file be provider-specific only (tts.gemini.style_prompt_file) or should Hermes eventually offer a generic tts.style_prompt_file with provider support flags?
• Should missing style files fail closed or warn-and-continue?
• Should audio tag passthrough sanitize to a small allowlist, or simply preserve bracketed tags when passthrough is enabled?

Non-goals

• Do not make all TTS providers share fake prompt controls they do not support.
• Do not add automatic dramatic tags by default.
• Do not expose secrets or local style-prompt contents in gateway-visible messages.