large-v3-turbo hallucinates repeated Chinese phrases during no-speech segments

[WangYuheng69]

Description

I encountered a reproducible Chinese transcription hallucination with large-v3-turbo.

During a screen-recorded online meeting, the host's microphone temporarily failed. I manually checked the original recording, and there was no speech during that interval.

However, Whisper still generated a long sequence of multilingual and corrupted-looking text, followed by the repeated Chinese phrase:

请不吝点赞 订阅 转发 打赏支持明镜与点点栏目

What makes this case interesting is that the hallucination occurred specifically during a no-speech / microphone-failure segment, and normal Chinese transcription resumed after the audio recovered.

Environment

• OS: Windows
• Library: openai-whisper
• Model: large-v3-turbo
• Model source: local large-v3-turbo.pt checkpoint
• GPU: NVIDIA GeForce GTX 1660 Ti
• Language: zh
• Task: transcribe
• FP16: enabled

Model loading:

import whisper

model = whisper.load_model(
    "large-v3-turbo.pt",
    device="cuda"
)

Transcription:

result = model.transcribe(
    video_path,
    language="zh",
    task="transcribe",
    fp16=True,
    initial_prompt=INITIAL_PROMPT,
    verbose=False,
)

The initial_prompt only contained technical terms related to spatiotemporal data and traffic forecasting.

It did not contain any words or phrases related to:

• 点赞
• 订阅
• 转发
• 打赏
• 明镜与点点栏目

Actual audio condition

I manually checked the original recording.

Immediately before the problematic interval, normal Chinese conversation was present.

Then the host's microphone failed, and there was no meaningful speech during the problematic interval.

After the microphone/audio recovered, normal Chinese transcription resumed.

Hallucinated output

The transcription around the no-speech interval was:

哎能听到吗你现在有你现在有代码了吗

不是一两�CA
把5 выступ cant得了吗
这就是웠那对那一个东西
那边的支持者
因为帮着了把4种区员的哗か
图博那辆酒
α勤抗强
他的钓冮

请不吝点赞 订阅 转发 打赏支持明镜与点点栏目
明镜与点点栏目
明镜与点点栏目
明镜与点点栏目
明镜与点点栏目
明镜与点栏目

After this interval, Whisper returned to normal Chinese transcription.

Second occurrence

A similar hallucination appeared again at the end of the recording.

The real conversation had already finished:

OK OK OK
嗯好拜拜

Whisper then continued generating:

就跟 потоп了
看个晒
请不吝点赞 订阅 转发 打赏支持明镜与点点栏目

There was no corresponding speech in the original recording.

Expected behavior

For an interval with no meaningful speech, I would expect Whisper to either:

1. classify the interval as no-speech and skip it, or
2. return no transcription for that interval.

Instead, the model generated multilingual text and a highly specific repeated Chinese phrase.

Questions

1. Is this a known hallucination pattern for large-v3 / large-v3-turbo?
2. Why can this pass the no-speech detection?
3. Could this fixed phrase be related to patterns learned from the training data?
4. Is there a recommended way to suppress this type of hallucination without removing legitimate low-volume speech?
5. Would adjusting parameters such as no_speech_threshold, logprob_threshold, or condition_on_previous_text help?

I can provide a short anonymized audio segment for reproduction if needed.

[misutoneko]

Hi, for [5] this might be worth a try:
#2766

I've only tried it with english samples and it usually helps (still not perfect, but pretty close).

[phineas-pta]

1. all whisper variants have hallucination problem
2. because of dataset issue
3. see Dataset bias ("❤️ Translated by Amara.org Community") #928
4. use dedicated VAD model (voice activity detection) example: SileroVAD
5. yes but it may not be reliable for all audio