Skip to content

Optimize: skip sending audio bytes to Omi backend in custom STT mode #7067

@beastoin

Description

@beastoin

Context

When custom STT is active, CompositeTranscriptionSocket.send() sends raw audio bytes to both the primary socket (custom STT provider) and the secondary socket (Omi /v4/listen backend).

The backend correctly discards these audio bytes — it has 6 guards preventing STT processing when custom_stt=enabled (see #6637 for full codepath trace). No listening minutes are consumed. However, the audio upload is wasted bandwidth.

Current behavior

app/lib/services/sockets/composite_transcription_socket.dart lines 140-147:

void send(dynamic message) {
    if (_status != PureSocketStatus.connected) {
      return;
    }
    primarySocket.send(message);      // audio → custom STT provider ✅
    secondarySocket.send(message);    // audio → /v4/listen ❌ wasted bandwidth
}

Proposed fix

Add a flag to skip sending raw audio bytes to the secondary socket. The secondary socket should only receive forwarded suggested_transcript JSON from the primary socket (which already happens via _forwardAsSuggestedTranscript).

void send(dynamic message) {
    if (_status != PureSocketStatus.connected) return;
    primarySocket.send(message);
    if (message is! List<int>) {
        secondarySocket.send(message);  // only forward non-audio messages
    }
}

Or use a skipAudioToSecondary flag as proposed in the closed PR #6634.

Impact

  • Bandwidth savings for custom STT users (no unnecessary audio upload)
  • No functional change — backend already ignores the audio
  • Speaker ID ring buffer on the backend (transcribe.py:2468) will stop receiving audio in custom STT mode — verify this is acceptable or handle separately

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    p3Priority: Backlog (score <14)

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions