A multitask model on a fine-tuned WavLM backbone produces both judgments — human vs. AI-generated, and accent profile — from a single audio clip.
🔗 Try it live | 🎬 Demo video
VoxShield developed an AI service platform that detects, in real time, whether a voice is generated by AI or spoken by a real person. As voice phishing and AI-generated voice scams become increasingly common, the platform provides an effective way to identify potentially fraudulent audio. In addition, when the speaker is a real person, the system can classify the speaker's accent or country of origin. This technology can help protect the general public from voice-based fraud while also providing valuable support to law enforcement and investigative agencies. Overall, VoxShield offers a practical AI solution for enhancing voice authentication and public safety.
| Human voice | AI-generated voice |
|---|---|
 |
 |
 |
 |
| Verified as human → language & accent stages run | Flagged as synthetic → accent stages skipped |
 |
 |
 |
 |
 |
|---|---|---|---|---|
| Jinwoong Lee | Hyunwoo Jung | Hyejin Park | Seongho Choi | Geonah Lee |
| Data Acquisition Lead | Data Systems | Presentation & Design | Engineering | Engineering |
| GitHub | GitHub | GitHub | GitHub | GitHub |
| Name | Major | Role |
|---|---|---|
| Jinwoong Lee | Robotics Engineering | Data Acquisition Lead — sourced, curated, and validated the training corpora underpinning both the detection and accent models |
| Hyunwoo Jung | Electronic Engineering | Data Systems — structured and maintained dataset documentation and technical records across the pipeline |
| Hyejin Park | Climate Environmental Engineering | Presentation & Design — crafted the visual narrative and deck that brings the system's results to life |
| Seongho Choi | Police Administration & Computer Science (Double Major) | Engineering — bridges investigative domain expertise with hands-on model and system development |
| Geonah Lee | Computer Science | Engineering — architected and built the core detection and classification pipeline |
Given a single audio clip, the service estimates two things at once:
- Accent (nationality) estimation — how close the speaker's English sounds to
each of several accent regions, expressed as per-class probabilities. It currently
covers 6 classes —
US / UK / CA / AU / IN / CN— and uses the raw softmax probabilities directly as "accent closeness" percentages (e.g., US 30% · UK 45% · IN 10% · ...). - AI synthetic speech (deepfake) detection — for the same input, whether the clip is a real human recording (REAL) or generated by TTS/voice cloning (FAKE).
Both tasks run through a single multitask model: a shared WavLM backbone with a country head and a real/fake head on top, so accent estimation and authenticity detection don't require two separate models.
- Estimating a speaker's accent region from audio alone is a recurring need in language learning feedback, call-center quality monitoring, and speaker profiling.
- At the same time, as AI-generated synthetic speech spreads, more use cases need to verify whether audio is actually human before trusting it. Solving both problems in one pipeline is why we chose a multitask design.
Audio input
│
▼
Pretrained audio encoder (WavLM, fine-tuned)
│
├──▶ Country (accent) classification head ──▶ US/UK/CA/AU/IN/CN probabilities
│
└──▶ Real/fake classification head ──▶ REAL/FAKE probabilities
- Training data lives in a GCS bucket (gcloud-first); training runs on Vertex AI Custom Jobs (T4/A100). Locally we only keep source code, environment variables, and docs.
- Results can be checked directly through a Cloud Run demo (model tester) — upload a clip and see accent percentages plus the REAL/FAKE verdict.
- One backbone, two heads — accent classification and synthetic-speech detection are trained jointly on a shared encoder, keeping inference cost low.
- Frame-level evidence — frame-level logits are preserved so a time-axis accent
heatmap can be rendered (
infer.py --plot). - Domain randomization augmentation — speed, band-limiting, reverb, and colored noise augmentations simulate channel/recording differences, improving generalization to unseen corpora.
- Reproducible training pipeline — curation → split → train → evaluate → deploy
is fully scripted, with a report written for every experiment
(
classifier/reports/).
| Area | Technology |
|---|---|
| Model | PyTorch, HuggingFace Transformers (WavLM backbone, fine-tuned) |
| Training infra | Google Cloud Vertex AI Custom Job (T4 / A100), Cloud Build, Artifact Registry |
| Data | Google Cloud Storage (GCS), speaker-disjoint train/val/test splits |
| Serving / demo | Google Cloud Run |
| Language | Python |
- PRD.md — Product Requirements Document