AI Girlfriend with VTube Studio

Anime AI Waifu is an AI powered voice assistant with VTuber's model, that combines the charm of anime characters with cutting-edge technologies. This project is meant to create an engaging experience where you can interact with desired character in real-time without powerful hardware.

Features

• 🎤 Voice Interaction: Speak to your AI waifu and get instant (almost) responses.

  • Whisper - openai's paid speech recognition.
  • Google sr - free speech recognition alternative.
  • Console - if you don't want use microphone just type prompts with your keyboard.

• 🤖 AI Chatbot Integration: Conversations are powered by an AI chatbot, ensuring engaging and dynamic interactions.

  • Openai's 'gpt-3.5-turbo' or any other available model.
  • File with personality and behaviour description.
  • Remembers previous messages.

• 📢 Text-to-Speech: Hear your AI waifu's responses as she speaks back to you, creating an immersive experience.

  • Google tts - free and simple solution.
  • ElevenLabs - amazing results, tons of voices.
  • Console - get text responses in your console (but VTube model will be just idle).

• 🌐 Integration with VTube Studio: Seamlessly connect your AI waifu to VTube Studio for an even more lifelike and visually engaging interaction.

  • Lipsync while talking.

Showcase

live demo here : https://www.youtube.com/shorts/SL3E6bAc6Qc

*Demonstration in real time without cutouts or speed up. This is real delay in answers.

Installation

1. Install Python 3.10.5 if you don't already have it installed.

2. Clone the repository by running git clone https://github.com/JarikDem-Bot/ai-waifu.git

3. Install the required Python packages by running pip install -r requirements.txt in the project directory.

4. Create .env file and enter your API keys

   .env template

   VTS_TOKEN=''
   OPENAI_API_KEY='YOUR_OPEN_AI_KEY'
   ELEVENLABS_API_KEY='YOUR_ELEVENLABS_KEY'

5. Install VB-Cable

6. Install and set VTube Studio

   Settings:

   • Select CABLE Output as microphone. Select Preview microphone audio to hear waifu's answers

     

   • Select input and output for Mouth Open. Optionally you can set "breathing" to get idle movents.

     

7. Select your required settings in main.py in waifu.initialise

8. Run the project by executing python main.py in the project directory.

Depending on the selected input mode, program may send all recorded sounds to the 3-rd parties.

License

MIT