Why not add GPU code for Kokoro TTS?

[alex-ie]

I have tried again to use TTS (previously was several months ago) and promptly discovered kokoro uses CPU only. Why not add code for GPU?

I have found some repos that claim to have it: https://github.com/simonfxr/kokoro.cpp, that one mentions: Inference code derived from CrispASR.

[LostRuins]

Kokoro is small enough (82m params only) that it runs faster than realtime on CPU.

[alex-ie]

faster than realtime on CPU.

Very slow. Say I want to TTS a book for later listening, I do not want to put PC on full CPU load for many hours, I rather load GPU for minutes.

Also maybe I do not know how to use kcpp properly, but in my experience kcpp UI generates a recording, then I can listen, it does not stream in real time.

[LostRuins]

fair, its just that current implementation does not have a gpu accelerated version. maybe in future i can improve it

[alex-ie]

That is what this discussion is all about. I understand the load for adding new models rather often - those are very important tasks. I wanted to discuss how difficult it is to add GPU code from e.g. link I have found. I have not tried that code btw. Cheers.