You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Feature Request: External Self Hosted Speech to Text Integration for T3 Code
#28636
Feature Request: External Self Hosted Speech to Text Integration for T3 Code
Summary
Add an optional external speech to text (STT) endpoint to T3 Code.
Users could connect an existing STT server through their local network, Tailscale, or a private HTTPS address.
The feature would add a microphone button to the message composer, similar to the voice input experience in the Codex desktop app.
The goal is to make voice input a native part of T3 Code without requiring users to install or manage another speech recognition model.
The workflow would be simple:
Press the microphone button in T3 Code.
Speak naturally.
T3 Code sends the audio to the configured STT endpoint.
The server transcribes the audio.
The transcript appears directly in the composer.
Edit or submit the prompt normally.
No OCR, voice commands, text to speech, or voice conversations are required.
The Problem: Speech Transcription Already Works, but the Workflow Is Incomplete
I already have a separate application for speech transcription. It connects to my existing self hosted STT API.
The transcription itself works well. The problem is the workflow when I want to use it with T3 Code.
My current workflow
Open or activate my separate transcription application.
Start recording.
Speak my prompt.
Stop recording and get the transcript.
Copy the transcribed text.
Switch back to T3 Code.
Paste the text into the message composer.
Review and submit the prompt.
This works, but it involves unnecessary context switching and manual copy and paste operations.
I would prefer to press a microphone button directly inside T3 Code and start speaking.
I do not need T3 Code to provide a new speech recognition model. I already have the infrastructure. I only want T3 Code to connect to it.
Why native integration would be better
Current external app workflow
Proposed native T3 Code integration
Switch to another application
Stay inside T3 Code
Start recording separately
Press the composer microphone
Copy the transcription manually
Insert text automatically
Paste into the composer
Text appears in the composer
Manage a separate interaction
Use one integrated workflow
Existing STT server
Reuse the same STT server
This is a relatively small integration compared with implementing an entire local STT engine.
It would make the application feel more polished and complete, particularly for users who already operate private AI services.
Proposed User Experience
Add a microphone button beside the existing composer controls.
Expected behavior
The user presses the microphone button.
T3 Code requests microphone permission if necessary.
Recording starts and a visible recording indicator appears.
Audio is sent to the selected transcription endpoint.
Partial results can appear while speaking when streaming is supported.
The user stops recording.
The final transcript is inserted into the existing composer draft.
The user reviews or edits the text before submitting it.
The transcription must not automatically submit a message.
Existing draft content and attachments must remain intact.
Recording cancellation must not remove existing text. Late transcription results must not overwrite a draft that the user has edited or replaced.
A keyboard shortcut would be useful but is not essential for the first implementation.
Where Transcription Requests Run
This is an important part of the proposal.
The microphone records audio on the user's device. The selected T3 Code environment server forwards the transcription request. The actual speech recognition runs on the external STT server configured by the user.
The recommended architecture is:
Client microphone → T3 Code environment server → External STT server → Environment server → Composer
Responsibilities
Component
Responsibility
T3 Code web or desktop client
Capture microphone audio and show the transcript
Selected T3 Code environment server
Forward audio to the configured STT endpoint and return results
External STT server
Run Whisper or another speech recognition model
T3 Code composer
Insert the recognized text into the editable draft
Coding agent provider
Receive the text only when the user submits the prompt
Why route requests through the T3 Code environment server?
T3 Code already has an architecture in which clients connect to selected environments through authenticated HTTP and WebSocket connections.
Using this existing connection offers several advantages.
1. The browser does not need direct access to the STT server.
For example, I could open T3 Code on my laptop while my T3 Code environment and STT server run on machines at home.
The laptop only needs access to the T3 Code environment.
The environment server can then connect to the STT endpoint through the local network or Tailscale.
2. API credentials can remain on the environment server.
The browser does not need to receive the STT API key.
3. Browser CORS restrictions are easier to handle.
The environment server makes the external API request instead of the browser.
Microphone permissions and secure context requirements still apply to the client.
4. It works with the existing remote architecture.
The same design can support local desktop environments, remote environments, and web clients.
Example deployment
Machine
Service
Laptop
T3 Code client and microphone
Desktop PC
T3 Code environment server
Dedicated GPU server
Existing STT API and speech model
Tailscale
Private network connectivity
The T3 Code environment server sends the STT request to the GPU server.
The GPU server performs the actual speech recognition and returns the result.
The client does not need to download, load, or execute the speech model.
If the STT server runs on the same machine as the T3 Code environment, the request can use a loopback address.
Important network requirement
The configured STT endpoint must be reachable from the T3 Code environment server, not necessarily from the device where the user presses the microphone button.
This distinction should be clear in the settings.
The endpoint is a setting of the selected environment. Different environments may use different STT servers.
Audio transport to a remote environment still uses the normal T3 Code connection. Depending on the selected route, traffic may pass through existing network tunnel infrastructure. The STT inference itself remains on the configured external server.
Configuration
Add an optional external transcription provider under Settings > Voice Input.
Suggested settings
Setting
Description
Default
External STT
Enable external transcription
Disabled
Endpoint URL
Address of the existing STT API
Empty
Protocol
HTTP transcription or WebSocket streaming
HTTP
Model
Optional model identifier
Server default
Language
Optional transcription language
Automatic
API key
Optional authentication token
Empty
Test connection
Check endpoint compatibility
Manual
The configuration should be associated with the selected T3 Code environment.
These are examples. The user supplies the actual address.
The interface should not require special Tailscale integration. A Tailscale address is simply another network endpoint.
The application should clearly report connection errors, authentication failures, unsupported protocols, and unreachable servers.
Transcription Protocols
Two methods would be useful.
Capability
HTTP transcription
WebSocket streaming
Audio transfer
Upload when recording ends
Send audio during recording
Live results
No
Yes, if supported
Implementation complexity
Lower
Higher
Existing API compatibility
Broad
Protocol dependent
Initial implementation
Recommended
Optional enhancement
Option A: OpenAI Compatible HTTP Transcription
This is the recommended minimum implementation.
After the user stops recording, T3 Code sends the audio to:
POST /v1/audio/transcriptions
The request uses multipart form data and includes the recorded audio and any required model identifier.
The server returns the transcription text.
T3 Code inserts that text into the composer.
Many existing STT services support this API format.
This would be enough to replace my separate transcription application for normal prompt dictation.
Option B: WebSocket Streaming
WebSocket streaming would make the experience even better.
While the user speaks:
The client captures audio in small chunks.
The chunks travel through the selected T3 Code environment to the STT service.
The STT service processes the audio.
Partial transcription results return to the client.
The composer shows provisional text.
Final results replace the provisional text when available.
This could make speaking into T3 Code feel similar to typing directly into the composer.
WebSocket transcription does not have one universal API contract. The implementation needs a documented protocol for audio formats, message types, and partial and final results.
For this reason, WebSocket support can be a separate improvement.
Tailscale and Private Infrastructure
Tailscale is particularly useful for this feature because users can access their own transcription hardware from different devices and networks.
The external STT service does not need a public IP address or a commercial cloud provider.
It can run on a private machine with a GPU.
The T3 Code environment only needs network access to that service.
Example
I could operate a dedicated GPU server at home with a Whisper based STT API.
When I use T3 Code from another computer, I could press the microphone button and speak.
T3 Code would forward the audio to my configured STT server through the environment connection.
The transcript would appear in the composer without opening my separate dictation application.
This would let me reuse the infrastructure I already maintain.
Privacy and Security
The feature should be disabled by default.
Recording should only start after explicit user action.
The application should not retain recorded audio after transcription unless the user explicitly enables that behavior.
API credentials should remain on the environment server and must not appear in logs or client responses.
The user should know which endpoint receives microphone audio.
The implementation should also support request timeouts, upload limits, recording duration limits, cancellation, and proper cleanup.
For remote connections, HTTPS and WSS should be supported.
Private HTTP endpoints can be supported when explicitly configured and permitted by the deployment.
Suggested Implementation Scope
The first version should remain small and focused.
Component
Required change
Settings
Add an external STT endpoint configuration
Composer
Add or reuse a microphone button
Client runtime
Capture audio and manage recording state
Environment server
Forward transcription requests
STT integration
Support an OpenAI compatible HTTP endpoint
Response handling
Return transcription text to the client
Composer state
Insert editable text safely
Error handling
Handle connection, authentication, and transcription errors
Tests
Verify recording, cancellation, draft safety, and remote requests
Where possible, reuse the existing voice input controller and transcription contracts.
No new local model management, speech model downloads, or GPU inference runtime should be required.
Optional future improvements
WebSocket streaming with live transcript previews.
Microphone selection.
Keyboard shortcuts.
Language selection and automatic detection.
Custom technical vocabulary.
Additional STT protocols.
Streaming recovery after temporary connection failures.
These are not necessary for the first implementation.
Relationship to Existing Work
There is already relevant voice transcription work in T3 Code.
This request should complement that work rather than replace it.
The important difference is external inference through a user supplied STT endpoint.
I do not want T3 Code to download or manage another speech model when I already operate a suitable transcription service.
I would prefer a small additional transcription provider that connects to my existing API.
I also understand that Discussion #15422 was closed after maintainer feedback against a configurable transcription service because of product complexity.
This proposal attempts to keep that complexity low through a disabled by default setting, one standard HTTP protocol, and reuse of the existing environment server architecture.
However, it still introduces a new capability and would need explicit maintainer approval.
Would the maintainers consider adding an optional external STT endpoint to the existing voice input architecture?
My main use case is simple: I already have a working speech transcription application and a self hosted STT API. I want to eliminate the need to copy and paste transcription results into T3 Code.
I believe native integration would make T3 Code feel more polished and complete.
I suggest starting with an OpenAI compatible HTTP endpoint, with requests forwarded by the selected T3 Code environment server.
The feature would let users bring their own transcription infrastructure without introducing another bundled model or requiring a commercial transcription provider.
I would appreciate feedback on whether this approach fits the project's design goals before implementation begins.
AI assistance disclosure: Development of this feature request was assisted by GPT 6.1 Sol.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Feature Request: External Self Hosted Speech to Text Integration for T3 Code
Summary
Add an optional external speech to text (STT) endpoint to T3 Code.
Users could connect an existing STT server through their local network, Tailscale, or a private HTTPS address.
The feature would add a microphone button to the message composer, similar to the voice input experience in the Codex desktop app.
The goal is to make voice input a native part of T3 Code without requiring users to install or manage another speech recognition model.
The workflow would be simple:
No OCR, voice commands, text to speech, or voice conversations are required.
The Problem: Speech Transcription Already Works, but the Workflow Is Incomplete
I already have a separate application for speech transcription. It connects to my existing self hosted STT API.
The transcription itself works well. The problem is the workflow when I want to use it with T3 Code.
My current workflow
This works, but it involves unnecessary context switching and manual copy and paste operations.
I would prefer to press a microphone button directly inside T3 Code and start speaking.
I do not need T3 Code to provide a new speech recognition model. I already have the infrastructure. I only want T3 Code to connect to it.
Why native integration would be better
This is a relatively small integration compared with implementing an entire local STT engine.
It would make the application feel more polished and complete, particularly for users who already operate private AI services.
Proposed User Experience
Add a microphone button beside the existing composer controls.
Expected behavior
The transcription must not automatically submit a message.
Existing draft content and attachments must remain intact.
Recording cancellation must not remove existing text. Late transcription results must not overwrite a draft that the user has edited or replaced.
A keyboard shortcut would be useful but is not essential for the first implementation.
Where Transcription Requests Run
This is an important part of the proposal.
The microphone records audio on the user's device. The selected T3 Code environment server forwards the transcription request. The actual speech recognition runs on the external STT server configured by the user.
The recommended architecture is:
Client microphone → T3 Code environment server → External STT server → Environment server → Composer
Responsibilities
Why route requests through the T3 Code environment server?
T3 Code already has an architecture in which clients connect to selected environments through authenticated HTTP and WebSocket connections.
Using this existing connection offers several advantages.
1. The browser does not need direct access to the STT server.
For example, I could open T3 Code on my laptop while my T3 Code environment and STT server run on machines at home.
The laptop only needs access to the T3 Code environment.
The environment server can then connect to the STT endpoint through the local network or Tailscale.
2. API credentials can remain on the environment server.
The browser does not need to receive the STT API key.
3. Browser CORS restrictions are easier to handle.
The environment server makes the external API request instead of the browser.
Microphone permissions and secure context requirements still apply to the client.
4. It works with the existing remote architecture.
The same design can support local desktop environments, remote environments, and web clients.
Example deployment
The T3 Code environment server sends the STT request to the GPU server.
The GPU server performs the actual speech recognition and returns the result.
The client does not need to download, load, or execute the speech model.
If the STT server runs on the same machine as the T3 Code environment, the request can use a loopback address.
Important network requirement
The configured STT endpoint must be reachable from the T3 Code environment server, not necessarily from the device where the user presses the microphone button.
This distinction should be clear in the settings.
The endpoint is a setting of the selected environment. Different environments may use different STT servers.
Audio transport to a remote environment still uses the normal T3 Code connection. Depending on the selected route, traffic may pass through existing network tunnel infrastructure. The STT inference itself remains on the configured external server.
Configuration
Add an optional external transcription provider under Settings > Voice Input.
Suggested settings
The configuration should be associated with the selected T3 Code environment.
Example endpoints
Local network:
http://192.168.1.50:8000/v1/audio/transcriptionsTailscale HTTPS:
https://stt.example.ts.net/v1/audio/transcriptionsWebSocket streaming:
wss://stt.example.ts.net/ws/transcribeThese are examples. The user supplies the actual address.
The interface should not require special Tailscale integration. A Tailscale address is simply another network endpoint.
The application should clearly report connection errors, authentication failures, unsupported protocols, and unreachable servers.
Transcription Protocols
Two methods would be useful.
Option A: OpenAI Compatible HTTP Transcription
This is the recommended minimum implementation.
After the user stops recording, T3 Code sends the audio to:
POST /v1/audio/transcriptionsThe request uses multipart form data and includes the recorded audio and any required model identifier.
The server returns the transcription text.
T3 Code inserts that text into the composer.
Many existing STT services support this API format.
This would be enough to replace my separate transcription application for normal prompt dictation.
Option B: WebSocket Streaming
WebSocket streaming would make the experience even better.
While the user speaks:
This could make speaking into T3 Code feel similar to typing directly into the composer.
WebSocket transcription does not have one universal API contract. The implementation needs a documented protocol for audio formats, message types, and partial and final results.
For this reason, WebSocket support can be a separate improvement.
Tailscale and Private Infrastructure
Tailscale is particularly useful for this feature because users can access their own transcription hardware from different devices and networks.
The external STT service does not need a public IP address or a commercial cloud provider.
It can run on a private machine with a GPU.
The T3 Code environment only needs network access to that service.
Example
I could operate a dedicated GPU server at home with a Whisper based STT API.
When I use T3 Code from another computer, I could press the microphone button and speak.
T3 Code would forward the audio to my configured STT server through the environment connection.
The transcript would appear in the composer without opening my separate dictation application.
This would let me reuse the infrastructure I already maintain.
Privacy and Security
The feature should be disabled by default.
Recording should only start after explicit user action.
The application should not retain recorded audio after transcription unless the user explicitly enables that behavior.
API credentials should remain on the environment server and must not appear in logs or client responses.
The user should know which endpoint receives microphone audio.
The implementation should also support request timeouts, upload limits, recording duration limits, cancellation, and proper cleanup.
For remote connections, HTTPS and WSS should be supported.
Private HTTP endpoints can be supported when explicitly configured and permitted by the deployment.
Suggested Implementation Scope
The first version should remain small and focused.
Where possible, reuse the existing voice input controller and transcription contracts.
No new local model management, speech model downloads, or GPU inference runtime should be required.
Optional future improvements
These are not necessary for the first implementation.
Relationship to Existing Work
There is already relevant voice transcription work in T3 Code.
Why this is different
PR #14882 already proposes substantial voice transcription functionality.
This request should complement that work rather than replace it.
The important difference is external inference through a user supplied STT endpoint.
I do not want T3 Code to download or manage another speech model when I already operate a suitable transcription service.
I would prefer a small additional transcription provider that connects to my existing API.
I also understand that Discussion #15422 was closed after maintainer feedback against a configurable transcription service because of product complexity.
This proposal attempts to keep that complexity low through a disabled by default setting, one standard HTTP protocol, and reuse of the existing environment server architecture.
However, it still introduces a new capability and would need explicit maintainer approval.
Relevant Technical Resources
Request for Maintainer Feedback
Would the maintainers consider adding an optional external STT endpoint to the existing voice input architecture?
My main use case is simple: I already have a working speech transcription application and a self hosted STT API. I want to eliminate the need to copy and paste transcription results into T3 Code.
I believe native integration would make T3 Code feel more polished and complete.
I suggest starting with an OpenAI compatible HTTP endpoint, with requests forwarded by the selected T3 Code environment server.
The feature would let users bring their own transcription infrastructure without introducing another bundled model or requiring a commercial transcription provider.
I would appreciate feedback on whether this approach fits the project's design goals before implementation begins.
AI assistance disclosure: Development of this feature request was assisted by GPT 6.1 Sol.
All reactions