AdVoice

Academic Project — AI & Machine Learning Emotionally-aware Text-to-Speech system for marketing advertisements

---

Overview

Marketing TTS API generates voice advertisements from text. Given an ad description, the system detects the emotional tone, selects a matching speaker voice, synthesizes speech, and applies audio post-processing effects that match the ad's mood.

---

How It Works

Ad Text
  │
  ▼
Emotion Detection          (joy / sadness / anger / neutral)
  │                        HuggingFace DistilRoBERTa
  ▼
Category & Gender          (fashion / food / tech / clothing + M/F)
  │                        Keyword-based classifier
  ▼
Speaker Selection          (from VCTK 108-speaker dataset)
  │                        Matched by emotion + gender
  ▼
Speech Synthesis           Coqui TTS — VITS / YourTTS
  │
  ▼
Audio Post-Processing      Emotion effects + humanization
  │                        librosa / scipy
  ▼
WAV Audio Output

---

Features

• Emotion-aware voice selection — matches speaker mood to ad content
• Multi-category product detection — fashion, food, technology, clothing
• Gender-targeted voice casting — selects male or female speakers based on product context
• Audio emotion effects — pitch, speed, and distortion tuned per emotion
• Audio humanization — subtle volume variations to reduce robotic sound
• Async ticket system — submit a job, poll for status, download audio when ready
• Dockerized — single command to run

---

Tech Stack

Component	Technology
API Framework	FastAPI + Uvicorn
Speech Synthesis	Coqui TTS (tts_models/en/vctk/vits, YourTTS)
Emotion Classification	j-hartmann/emotion-english-distilroberta-base
Speaker Dataset	VCTK Multi-Speaker Corpus (108 speakers)
Audio Processing	librosa, soundfile, scipy
Containerization	Docker

---

Project Structure

AdVoice/
├── app.py               # Gradio UI — interactive demo interface
├── main.py              # FastAPI REST API routes and ticket management
├── tts_service.py       # Core logic: TTS, emotion, speaker selection, audio
├── requirements.txt     # Python dependencies
├── Dockerfile
├── data/
│   └── vctk_metadata_upd_cleaned_wsl.csv   # Speaker metadata
└── outputs/             # Generated audio files (git-ignored)

Two ways to run AdVoice:

• Gradio UI (app.py) — interactive browser interface, best for demos
• FastAPI REST API (main.py) — async ticket-based API, best for integration

---

API Endpoints

POST /create-ticket

Submit ad text for processing.

curl -X POST "http://localhost:8000/create-ticket" \
  -H "Content-Type: application/json" \
  -d '{"ad_description": "Stunning red dress perfect for elegant women!"}'

Response:

{
  "ticket_id": "4f2a5a9e-...",
  "status": "pending",
  "emotion": null
}

GET /ticket-status/{ticket_id}

Poll for status: pending → done or failed

GET /ticket-audio/{ticket_id}

Download the generated WAV file once status is done.

---

Running Locally

git clone https://github.com/rayenx2/advoice.git
cd advoice

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
mkdir -p outputs

Option 1 — Gradio UI (recommended for demos):

python app.py

Opens at http://localhost:7860

Option 2 — FastAPI REST API:

uvicorn main:app --host 0.0.0.0 --port 8000 --reload

Open http://localhost:8000/docs for the interactive API explorer.

---

Running with Docker

docker build -t marketing-tts-api .
docker run -p 8000:8000 marketing-tts-api

---

Academic Context

This project was developed as part of an AI & Machine Learning curriculum. It explores:

• Pre-trained transformer models for NLP (emotion detection)
• Neural text-to-speech synthesis using the VITS architecture
• Multi-speaker voice selection strategies
• Audio signal processing for post-production effects
• Asynchronous REST API design with FastAPI

The VCTK dataset (109 speakers) provides speaker metadata for voice selection. The dataset is not included in this repo — see VCTK Dataset.

---

Known Limitations

• Category detection uses keyword matching, not a semantic classifier
• Ticket storage is in-memory and resets on server restart
• English only (VITS model)

---

Author

Rayen Lassoued GitHub: @rayenx2

---

Academic use only.