LiveSynth

LiveSynth is a program that allows you to generate a voice with AI using OpenAI's Whisper and the ElevenLabs API. By holding a key, you can record your voice, have it transcribed, and used to generate an AI voice from what you said. I use this to give myself an interesting voice in games. Windows is not and will not be supported lol.

Requirements

• Linux
• PipeWire or PulseAudio
• CUDA

Usage

python whisper.py --key alt_r --voice 21m00Tcm4TlvDq8ikWAM --api-key <your xi-api-key>

The --key option expects the name of an x11 keysym, those are listed here.

The default whisper model requires 5GB of vram. This can be changed with --model or CPU transcription can be used with --cpu. The available whisper models are documented here.

To output to a custom sink (useful for using as an input device) you can use the --output-sink option which expects the name or serial of the sink.

--input-source is also available to choose a specific input.

Creating a virtual sink (named LiveSynth) and source (named LiveSynthSource) can be done with a couple commands:

pactl load-module module-null-sink sink_name="LiveSynth" sink_properties=device.description="LiveSynth Sink"
pactl load-module module-remap-source master="LiveSynth.monitor" source_name="LiveSynthSource" source_properties=device.description="LiveSynth Source"

Caveats

• Whisper uses a lot of VRAM (5GB for medium)
• The ElevenLabs API often takes a while to return a result. Hopefully open source AI voice will catch up soon because this sucks.