Streaming support for voice chat

[PavelNikolaichev]

I think it might be a good idea to have support for streaming LLM response during the voice chat mode. I think it would improve overall feedback loop for end user.

One of the possible solutions might be smth like voicechat2, which utilizes sockets to constantly check for a stream response from LLM. It sends the response sentence by sentence or so.

If you have anything to ask or add, feel free to do so!

[PavelNikolaichev]

Gonna implement this one myself

[nathanhere]

interested in this too!

[TecReaGroup]

Gonna implement this one myself

Hope soon! Thanks! It's really useful!

[danielcederberg]

Want this too!

[PavelNikolaichev]

Hi, sorry guys, I forgot completely to leave a comment on it, this stuff was not relevant, that's why I haven't implement it. You can check the debug's on the frontend and see that's voice input does already streams. If you want voice only mode or smth, you might want to try adding VAD before or after audio preprocessing. Shouldn't be too hard to configure.