Qwen3-TTS Voice Cloning Web Application

A FastAPI + pure JavaScript web application for voice cloning using Qwen3-TTS (1.7B Base model).

Features

• Live Voice Recording: Record directly from your browser microphone
• Voice Cloning: Create voice profiles from 3+ second audio clips
• Text-to-Speech: Generate speech in your cloned voice
• Waveform Visualization: Interactive audio players with waveform display (WaveSurfer.js)
• Multi-Language Support: Auto-detect or select Chinese, English, or Japanese
• Voice Profile Management: Download, import, and manage multiple voice profiles with server-side name storage
• Text Chunking: Automatically split long text for synthesis
• GPU Accelerated: Runs on NVIDIA RTX 3090 with 24GB VRAM
• Model Auto-Offload: Frees GPU memory after 10 minutes of inactivity
• Modern UI: Horizontal 3-panel layout (Capture, Profile, Generate)

Requirements

• Python 3.12+
• NVIDIA GPU with 12GB+ VRAM (RTX 3090 recommended)
• CUDA 12.8+

Project Structure

qwen-tts/
├── venv/                      # Python virtual environment
├── app/
│   ├── main.py                # FastAPI application (serves static files)
│   ├── model_manager.py        # Qwen3-TTS model manager
│   ├── schemas.py             # Pydantic data models
│   ├── utils.py               # Audio processing utilities
│   ├── profiles/              # Persisted voice profiles (JSON files)
│   └── static/               # Frontend static files
│       ├── index.html         # Main HTML
│       ├── app.js            # Main application logic (JavaScript)
│       └── style.css         # CSS styles
└── requirements.txt          # Python dependencies

Installation

1. Python Virtual Environment

python3 -m venv venv
source venv/bin/activate

2. Install Python Dependencies

pip install -r requirements.txt

Optional: Install flash-attn for reduced GPU memory usage

pip install flash-attn --no-build-isolation

Running the Application

cd app
source ../venv/bin/activate
python main.py

Then open http://localhost:8000 in your browser.

Usage

1. Record Your Voice

• Click "Start Recording"
• Speak clearly for 3+ seconds (any duration works)
• Click "Stop Recording"
• Preview your recording

2. Create Voice Profile

• Enter the exact transcript of what you said
• Select language (or use Auto-detect)
• Click "Clone Voice"
• Enter a name for your voice profile

3. Generate Speech

• Select a voice profile from the dropdown
• Enter text you want to synthesize
• Select language
• Click "Generate Speech"
• Play or download the generated audio

4. Voice Profile Management

• Create: Multiple voice profiles with custom names
• Download: Save voice profile as JSON file for backup
• Import: Load previously saved voice profile
• Delete: Remove voice profiles (delete all, including the last one)
• Persistence: Profiles persist across server restarts (saved to app/profiles/)

API Endpoints

POST /api/record-voice

Create voice profile from recorded audio.

Request:

{
  "audio_base64": "base64_encoded_audio_string",
  "transcript": "What user said in recording",
  "language": "Auto",
  "name": "My Voice"
}

Response:

{
  "prompt_id": "550e8400-e29b-41d4-a716-446655440000",
  "name": "My Voice",
  "status": "success",
  "message": "Voice profile created successfully"
}

POST /api/synthesize

Generate speech using cloned voice. Automatically chunks text over 200 characters.

Request:

{
  "text": "Hello, this is synthesized speech",
  "language": "Auto",
  "prompt_id": "550e8400-e29b-41d4-a716-446655440000"
}

Response:

{
  "audio_base64": "base64_encoded_audio_string",
  "sample_rate": 24000,
  "duration": 3.5,
  "status": "success"
}

GET /api/health

Check system status.

Response:

{
  "model_loaded": true,
  "model_name": "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
  "gpu_available": true,
  "gpu_memory_used": 8.5,
  "gpu_memory_total": 24.0,
  "gpu_memory_cached": 10.2,
  "active_prompts": 1,
  "auto_offload_enabled": true,
  "auto_offload_minutes": 10
}

GET /api/prompts

List all active voice profiles.

Response:

{
  "prompts": [
    {
      "prompt_id": "550e8400-e29b-41d4-a716-446655440000",
      "name": "My Voice",
      "created_at": "2024-01-01T00:00:00"
    }
  ],
  "count": 1
}

GET /api/prompts/{prompt_id}

Get info about a specific voice profile.

PATCH /api/prompts/{prompt_id}

Update voice profile name.

Request:

{
  "name": "My New Voice Name"
}

Response:

{
  "status": "success",
  "message": "Profile name updated",
  "name": "My New Voice Name"
}

DELETE /api/prompts/{prompt_id}

Delete a voice profile.

GET /api/download-prompt/{prompt_id}

Download voice profile as JSON file.

POST /api/import-prompt

Import voice profile from uploaded JSON file (multipart form data).

POST /api/model/load

Load model into GPU memory.

POST /api/model/unload

Unload model to free GPU memory.

POST /api/model/config

Configure auto-offload settings.

Request:

{
  "enabled": true,
  "minutes": 10
}

Model Information

• Model: Qwen/Qwen3-TTS-12Hz-1.7B-Base
• Parameters: ~6.8B (non-embedding: ~5.9B)
• Audio Sample Rate: 24kHz
• Languages: Chinese, English, Japanese (auto-detect supported)
• GPU Memory: ~6-8GB (with flash-attn), ~8-10GB (without)
• Flash Attention 2: Automatically detected and used when available

Model Management

The model automatically unloads from GPU memory after 10 minutes of inactivity to free up VRAM.

Check Model Status

curl http://localhost:8000/api/health

Manually Unload/Load Model

# Unload to free GPU memory
curl -X POST http://localhost:8000/api/model/unload

# Load model back (takes 10-15 seconds)
curl -X POST http://localhost:8000/api/model/load

Configure Auto-Offload

# Disable auto-offload
curl -X POST http://localhost:8000/api/model/config \
  -H "Content-Type: application/json" \
  -d '{"enabled": false}'

# Set custom timeout (e.g., 5 minutes)
curl -X POST http://localhost:8000/api/model/config \
  -H "Content-Type: application/json" \
  -d '{"enabled": true, "minutes": 5}'

Troubleshooting

Server won't start

# Check if venv is activated
source venv/bin/activate

# Check if dependencies are installed
pip list | grep -E "(fastapi|qwen-tts|torch)"

# Try reinstalling dependencies
pip install -r requirements.txt

GPU not detected

# Check GPU status
nvidia-smi

# Check CUDA availability in Python
source venv/bin/activate
python -c "import torch; print(f'CUDA: {torch.cuda.is_available()}')"

Voice profile import/synthesis errors

• Ensure the voice profile was exported correctly
• Check that the backend is running and model is loaded
• Verify the profile ID is valid using /api/prompts

Performance

• First Run: 5-10 minutes (model download)
• Model Loading: 10-15 seconds on subsequent runs
• Voice Cloning: 2-5 seconds for 3+ second audio
• Speech Synthesis: 0.5-1 seconds per second of audio

API Documentation

Once backend is running, visit:

• API Docs: http://localhost:8000/docs
• Interactive API: http://localhost:8000/redoc

Development

The frontend is built with pure JavaScript (no build step required). Backend is FastAPI.

# Start development server
cd app
source ../venv/bin/activate
python main.py

# Frontend files are in app/static/
# Backend API is in app/main.py

License

This project uses Qwen3-TTS, which is licensed under Apache 2.0.

Acknowledgments

• Qwen3-TTS - Voice cloning model
• FastAPI - Python web framework