Skip to content

fix(asr): 过滤 GLM-ASR 偶发的纯井号占位转写(#787)(同步 #916 到 beta) - #920

Merged
appergb merged 1 commit into
betafrom
fix/cherrypick-916-to-beta
Aug 5, 2026
Merged

fix(asr): 过滤 GLM-ASR 偶发的纯井号占位转写(#787)(同步 #916 到 beta)#920
appergb merged 1 commit into
betafrom
fix/cherrypick-916-to-beta

Conversation

@appergb

@appergb appergb commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

User description

main 上 #916 的 hotfix(直接合入 main,绕过 beta 流程)补进 beta:正式版发布前保持两条分支一致,避免 beta 后续版本缺失 GLM-ASR 纯井号过滤。cherry-pick 后 cargo check 通过。


PR Type

Bug fix, Tests


Description

  • Filter GLM-ASR pure # placeholder transcriptions.

  • Normalize placeholder chunks to empty transcripts, reusing the existing empty guard.

  • Apply normalization in both normal and confident-text fallback paths.

  • Add unit tests for single, mixed, and all-placeholder chunk scenarios.


Diagram Walkthrough

flowchart LR
  A["ASR response text"] --> B{"Pure # placeholder?"}
  B -- "yes" --> C["Empty transcript"]
  B -- "no" --> D["Normal transcript"]
  C --> E["Existing empty guard"]
Loading

File Walkthrough

Relevant files
Bug fix
whisper.rs
Filter pure hash placeholders in ASR transcription             

openless-all/app/src-tauri/src/asr/whisper.rs

  • Added is_placeholder_heading to detect text consisting only of #
    characters.
  • Applied placeholder normalization in transcribe_chunk and
    extract_confident_text.
  • Preserved real content such as C# and # 你好.
  • Added 4 unit tests covering pure, mixed, and all-placeholder cases.
+103/-2 

@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

916 - Partially compliant

Compliant requirements:

  • is_placeholder_heading added and correctly detects strings made only of # (including ##, ###, and whitespace-padded variants) while rejecting C#, # 你好, #hash, empty, and whitespace-only text.
  • The check is applied in the non-verbose branch of transcribe_chunk and in the no-segments fallback of extract_confident_text; matches are normalized to empty strings and reuse the existing empty-transcript guard.
  • Multi-chunk placeholder chunks are dropped while real chunks are preserved (["你好", "#", "世界"] -> 你好世界).
  • Four unit tests were added covering the required placeholder scenarios and normal-content preservation.

Non-compliant requirements:

None

Requires further human verification:

  • Whether GLM-ASR in verbose_json mode can return a segments array containing a pure-# segment. The new filter is only applied to the whole text field and to the no-segments fallback, not to individual segments in the extract_confident_text loop, so runtime verification with GLM-ASR is needed to confirm this path cannot leak placeholders.
⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Possible Issue

The placeholder filter is applied only to the whole-text path (non-verbose branch) and to the no-segments fallback in extract_confident_text. When verbose_json is enabled and the response contains a segments array, segment texts are still concatenated as before, so a segment whose text is pure # (the issue #787 placeholder) is kept whenever it passes the compression_ratio/avg_logprob heuristics, or kept unconditionally if the provider omits those metadata fields (missing metadata is treated as "don't discard"). In that path the placeholder would still reach user input, which is the exact bug this PR targets. Consider filtering pure-# segment texts inside the loop or normalizing the final kept string; whether GLM-ASR actually returns such segments in verbose mode needs runtime confirmation.

    let text = json["text"].as_str().unwrap_or("").trim();
    if is_placeholder_heading(text) {
        return String::new();
    }
    return text.to_string();
};

let mut kept = String::new();
for seg in segments {
    let text = seg.get("text").and_then(|t| t.as_str()).unwrap_or("");
    if text.trim().is_empty() {
        continue;
    }
    let no_speech = seg
        .get("no_speech_prob")
        .and_then(|v| v.as_f64())
        .unwrap_or(0.0);
    let avg_logprob = seg
        .get("avg_logprob")
        .and_then(|v| v.as_f64())
        .unwrap_or(0.0);
    let compression = seg
        .get("compression_ratio")
        .and_then(|v| v.as_f64())
        .unwrap_or(1.0);

    let is_hallucination =
        (no_speech > 0.6 && avg_logprob < -0.5) || compression > 2.4 || avg_logprob < -1.0;
    if is_hallucination {
        log::warn!(
            "[whisper] 丢弃疑似幻听段落: no_speech={:.2} avg_logprob={:.2} compression={:.2} text={:?}",
            no_speech,
            avg_logprob,
            compression,
            text.trim()
        );
        continue;
    }
    kept.push_str(text);
}

let kept = kept.trim().to_string();
if kept.is_empty() {
    // 全部段落被判为幻听(≈整段几乎是静音)。回退到原始 text 会把幻听又捡
    // 回来,所以返回空串;上层把空转写当「什么都没说」无害处理。
    return String::new();
}
kept

@appergb
appergb merged commit d31bf5b into beta Aug 5, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants