Migrate editdistance metrics to rapidfuzz with empty-reference guard

[LauraGPT]

Summary

Maintainer-owned replacement for #3105. Thanks to @excosy for starting the editdistance -> rapidfuzz migration.

This PR:

• replaces archived editdistance with rapidfuzz.distance.Levenshtein
• updates runtime and install dependencies to require rapidfuzz>=3.0.0
• preserves CER/WER behavior for normal non-empty references
• fixes the remaining empty-reference divide-by-zero case found while reviewing Migrate obsoleted editdistance to rapidfuzz #3105

Root Cause

The original migration guarded only on whether edit-distance entries existed. That still divides by zero when the input list is non-empty but every reference is empty or whitespace-only:

calculate_cer(["abc"], ["   "])
calculate_wer(["hello world"], ["   "])

This PR calculates total reference length explicitly and returns None when it is zero, matching the existing CTC metric no-valid-reference behavior.

Testing

python -m pytest tests/test_metrics_common_rapidfuzz.py -q
python -m py_compile funasr/metrics/common.py tests/test_metrics_common_rapidfuzz.py setup.py
git diff --check origin/main...HEAD

Additional validation on ind-gpu8:

• red before fix: tests/test_metrics_common_rapidfuzz.py failed with ZeroDivisionError
• green after fix: 2 passed
• temporary real rapidfuzz>=3.0.0 install verified string and token-list Levenshtein.distance() semantics

[gemini-code-assist]

Code Review

This pull request replaces the editdistance dependency with rapidfuzz in funasr/metrics/common.py and the setup configuration. It also handles division-by-zero scenarios in CER and WER calculations when the reference length is zero. A new test file is introduced to verify these changes. The reviewer recommends simplifying the test file by removing the complex mocking of rapidfuzz and dynamic module loading, and instead using standard imports and the real library.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

Mocking rapidfuzz with a custom pure-Python Levenshtein implementation and using dynamic module loading (importlib.util) is an anti-pattern here. Since rapidfuzz is a required dependency in setup.py, it is guaranteed to be installed in any standard test environment. Mocking it defeats the purpose of testing the actual integration with the library, hides potential import/compatibility issues, and adds unnecessary complexity to the test suite. Simplifying the test file to use standard imports and the real rapidfuzz library makes the tests cleaner, more maintainable, and more reliable.

from funasr.metrics.common import ErrorCalculator


def test_cer_and_wer_return_none_when_reference_length_is_zero():
    calc = ErrorCalculator(
        char_list=["<blank>", "a", "b", "c", "h", "e", "l", "o", "w", "r", "d", " "],
        sym_space=" ",
        sym_blank="<blank>",
        report_cer=True,
        report_wer=True,
    )

    assert calc.calculate_cer(["abc"], ["   "]) is None
    assert calc.calculate_wer(["hello world"], ["   "]) is None


def test_cer_and_wer_keep_normal_levenshtein_semantics():
    calc = ErrorCalculator(
        char_list=["<blank>", "a", "b", "c", "x", "y", " "],
        sym_space=" ",
        sym_blank="<blank>",
        report_cer=True,
        report_wer=True,
    )

    assert calc.calculate_cer(["abc"], ["axc"]) == 1 / 3
    assert calc.calculate_wer(["foo bar baz"], ["foo qux baz"]) == 1 / 3