Skip to content

Add an OpenAI TTS provider #26

Description

@jason-shen

Problem

[tts] provider currently accepts cartesia, deepgram, elevenlabs, minimax, speechify, vibevoice — but not OpenAI, even though github.com/sashabaranov/go-openai is already a direct dependency and most users running StreamCore already have an [openai] api_key set for the LLM. Someone who wants to try the project with exactly one API key currently can't: they need an OpenAI key for the LLM and a second vendor's key for speech.

OpenAI's /v1/audio/speech also supports streamed chunked audio, so it fits the existing SynthesizeStream contract rather than degrading to the one-shot path.

Proposed change

Add internal/tts/openai.go implementing tts.Client:

type Client interface {
	Synthesize(ctx context.Context, text string) ([]byte, error)
	SynthesizeStream(ctx context.Context, text string) (<-chan StreamChunk, error)
}

internal/tts/deepgram.go and internal/tts/speechify.go are the closest models to copy — both are HTTP providers returning PCM. Read internal/tts/http.go first; the shared request/stream plumbing lives there and a new provider should reuse it rather than hand-rolling a client.

Things the implementation has to get right, all of which the existing providers already demonstrate:

  • Output format. The pipeline expects linear16 PCM, 16 kHz mono. Request pcm from the API and confirm the sample rate — if OpenAI returns 24 kHz you must resample; internal/tts/resample_test.go shows the helper that already exists for exactly this.
  • Streaming. Emit StreamChunks as bytes arrive so playback starts before synthesis finishes. Close the channel exactly once, including on the error path.
  • Cancellation. Barge-in cancels the context mid-utterance. Aborting must not leak the response body or the goroutine.

Wiring:

  1. internal/config/config.go — reuse [openai] api_key; add tts_model and tts_voice to OpenAIConfig (models gpt-4o-mini-tts / tts-1 / tts-1-hd, voices alloy, nova, …). Default the model and voice in Load().
  2. internal/tts/client.go — add case "openai": to newProviderClient, with the same ErrMissingAPIKey guard the others use, and add openai to the "supported:" list in the default branch's error string.
  3. config.toml.example + README provider table.

Acceptance criteria

  • [tts] provider = "openai" synthesizes speech end to end on a call.
  • Streaming emits multiple chunks for a long utterance, not one buffered blob.
  • Cancelling the context mid-stream closes the channel and releases the connection.
  • Audio is 16 kHz mono linear16 by the time it reaches the pipeline.
  • Unit test in the style of internal/tts/http_test.go using an httptest server.

Note for other providers

The same shape applies to Rime, PlayHT, Hume, Azure, and Google TTS. If you'd rather add one of those, that's welcome — open a separate issue referencing this one so two people don't land the same provider twice.

Pointers

  • internal/tts/client.go — the Client interface and newProviderClient switch
  • internal/tts/http.go, internal/tts/deepgram.go, internal/tts/speechify.go — closest existing patterns
  • internal/tts/errors.goErrMissingAPIKey

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestgood first issueScoped small, with enough context in the issue to start

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions