Voice Input with Voice Activation to Chat in Zed

[D3m0n5c1l]

Body

Problem Statement

With the widespread adoption of AI, people have begun using prompts for a wide range of purposes, including application development, research, and support in everyday areas such as healthcare, finance, and politics. To make prompts effective and valuable, they often need to be both detailed and well-structured while remaining concise. However, many users underestimate the advantages of creating prompts through speech rather than typing. Verbal input allows people to provide richer context and more detailed instructions with less effort, potentially improving the quality of AI-generated responses. In addition, AI should be accessible to everyone. Many people with disabilities live without one or both hands, which can significantly limit their ability to type complex prompts on a keyboard. Voice-based interaction can help remove these barriers and make AI technologies more inclusive and accessible.

Proposal

For a quite popular open-source code editor Zed, we propose a voice chat extension ZedVC. ZedVC will be capable of "listening" to user's prompts, converting the prompts to the user's native language words and sentences, and sending the prompt for Zed processing. Users will be able to see the prompt processing in real-time and edit out specific words to match their prompt vision. As of now, the proposal is impossible to achieve, as Zed does not provide functionality for chat interaction via extensions. We want to fix the gap by contributing to the extension API of the Zed editor source code.

Key Features

• The user activates prompt recording by uttering a predefined start command, describes the request verbally, terminates the recording with a predefined end command, and aborts any work at any stage with a specific stop word.
• ZedVC performs real-time Speech-to-Text (STT) conversion, allowing the user to validate and refine the generated prompt prior to submission.
• The platform supports multiple languages to promote accessibility and accommodate users across diverse geographic regions and language communities.

Notes

1. Within the next month, we will be actively contributing to the following feature, as we have selected it for Capstone Project discipline in our bachelor degree (2nd Year, Innopolis University).
2. We have created the new discussion in order to modify the issue description and bring more attention to the lack of voice input to chat in Zed.

[dantetemplar]

Related:

• Add voice dictation / microphone input for external ACP agents #54489
• Dictation for Assistant (inline and chat-based) #16410
• Agent speech to text dictation feature #52393

[macraig]

Opened an issue to track this here: #59726