Model Evaluation

Why Evaluation Matters

A high test accuracy does not necessarily mean that a voice-cloning detector is reliable in real-world conditions.

VoxShield should evaluate how well the model handles:

• Unseen speakers
• Different microphones
• Background noise
• Recording compression
• Various audio lengths
• Different synthetic voice systems

Confusion Matrix

A confusion matrix summarizes classification results.

Actual / Predicted	REAL	FAKE
REAL	True Negative / Correct REAL	False Positive
FAKE	False Negative	True Positive / Correct FAKE

The exact positive-class terminology depends on how labels are encoded.

Important Metrics

Precision

• Of the samples predicted as FAKE, how many were actually FAKE?

Recall

• Of the actual FAKE samples, how many did the model identify?

F1 Score

• The harmonic mean of precision and recall.

Accuracy

• The proportion of correctly classified samples.

Security Relevance

• False negatives are particularly important because a synthetic voice may be classified as REAL.

• False positives also matter because genuine speakers may be incorrectly flagged.

• The relative importance of these errors depends on the intended use and operating threshold.