Voice input / transcription

[AltanS]

Two PRs have now added speech input to the composer, from two different people:
#91 and
#115. Both were careful work. No issue has ever asked
for it, so people want this enough to just build it, and I am clearly missing something.

Collie's composer is a plain textarea, so your phone keyboard's own microphone already types into it
and the per-pane draft keeps the result. No setup, and on a modern phone it stays on the device. What
does that not give you? Reply with a number, or tell me what I left out.

1. The keyboard covers the pane output you are replying to.
2. You still have to tap Send. Not hands free.
3. Accuracy on identifiers, filenames, flags, command names.
4. It gives up on a pause, so long prompts get cut.
5. You have to speak the punctuation.
6. Your language, or mixing languages.
7. Not available to you: managed device, Siri off, a keyboard without it.
8. You are not on a phone. Desktop browsers have no keyboard microphone.
9. You would rather use a model you run yourself.

The two PRs point different ways. #91 keeps Send explicit and lets you point at your own endpoint.
#115 sends hands free by default. Those are not the same feature, and knowing which one people want
matters more than the code does.

What makes me hesitate is the machine running Collie picking up a credential, an outbound network
path, or a long-running child process in order to transcribe. That bridge binds to loopback and
manages one front door, and I would rather it stayed that shape. Doing it in the browser is one way
around that, but I am not claiming it is the only one. If there is an arrangement I have not thought
of, I would much rather hear it than have someone spend another weekend on a PR I decline.

Thanks to @en-ver and @ardaaltinors.

[en-ver]

@AltanS

1. The keyboard covers the pane output you are replying to.
2. You still have to tap Send. Not hands free.
3. Accuracy on identifiers, filenames, flags, command names.
4. It gives up on a pause, so long prompts get cut.
5. You have to speak the punctuation.
6. Your language, or mixing languages.
7. Not available to you: managed device, Siri off, a keyboard without it.
8. You are not on a phone. Desktop browsers have no keyboard microphone.
9. You would rather use a model you run yourself.

None of those. Android keyboard's speech-to-text is stupid. It misrecognizes words and doesn't care about punctuation and context, which causes it to transcribe incorrectly. I have to spend a lot of time correcting mistranscribed words. On the other hand, the prompts are often very long, and typing them manually is very challenging. And you have to do it on the go, which is very inconvenient. With the speech-to-text feature, I can type a huge prompt in seconds.

[AltanS]

I see, thanks for your input. Will have to think about this and where it fits into the current app, but if collie is helping with ergonomics it does make sense to look at voice inputs more seriously.

[en-ver]

@AltanS My pull request assumes the optional usage of the speech-to-text feature. Those who don't want it just don't configure it, and it will be disabled for them. But I think most users will benefit from it.

[ardaaltinors]

Totally agree! Also, I love prompting in a mix of languages (Turkish + English). iOS dictation cant handle that tbh

[en-ver]

Lucky you, iOS )) but somebody uses Android )) and here OpenAI transcription shines.