How much does Smart Turn lose on phone audio? #971
Charansripadi
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I measured Smart Turn v3.2 through a simulated phone channel (8 kHz mu-law, 300 to 3400 Hz). On 12,612 test clips, accuracy fell from 92.41% to 88.92%, and unfinished turns called finished rose from 9.86% to 18.05%, so callers get cut off about 1.8x as often.
Fine-tuning with phone-audio augmentation, compared with a control on the full 31,527 clips over that channel, took accuracy from 91.52% to 92.78% and that error rate from 11.09% to 8.64%. Accuracy improved in all 23 languages. The channel is simulated, so I haven't tested this on real calls.
Do you see more cut-offs on telephony than on web calls? I'd like to compare this with real traffic.
Weights (same ONNX interface as v3.2): https://huggingface.co/CharanSripadi/haan
All reactions