Fix Fun-ASR-Nano CTC batch fallback#3187
Conversation
There was a problem hiding this comment.
Code Review
This pull request implements sequential processing for multi-segment inputs when CTC timestamps are active by introducing the _inference_llm_ctc_sequential method along with helper functions _slice_batch_value and _merge_inference_meta, supported by a new unit test. The review feedback identifies two key issues: a potential runtime crash when key is None or out of bounds during sequential execution, and incorrect fallback behavior in _slice_batch_value where the entire original value is returned if the index is out of bounds. Code suggestions are provided to resolve both issues.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| results = [] | ||
| meta_data = {} | ||
| for i, data_i in enumerate(data_in): | ||
| key_i = [key[i]] if key is not None and i < len(key) else None |
There was a problem hiding this comment.
If key is None or if its length is less than the number of segments in data_in, key_i will be set to None. When self.inference_llm is recursively called with key=None, it will crash with a TypeError: 'NoneType' object is not subscriptable when attempting to access key[0] (e.g., at lines 885 and 960). To prevent this runtime crash, we should provide a fallback key (such as a generated random key or index-based key) when key is unavailable or out of bounds.
| key_i = [key[i]] if key is not None and i < len(key) else None | |
| key_i = [key[i]] if key is not None and i < len(key) else [f"rand_key_{i}"] |
| def _slice_batch_value(value, index): | ||
| if value is None: | ||
| return None | ||
| if isinstance(value, torch.Tensor) and value.ndim > 0 and value.shape[0] > index: | ||
| return value[index : index + 1] | ||
| if isinstance(value, list) and len(value) > index: | ||
| return [value[index]] | ||
| if isinstance(value, tuple) and len(value) > index: | ||
| return (value[index],) | ||
| return value |
There was a problem hiding this comment.
In _slice_batch_value, if value is a multi-element Tensor, list, or tuple, but index is out of bounds (i.e., index >= len(value)), the function falls through and returns the entire original value. This is incorrect because the downstream code expects a single-item slice or None for the out-of-bounds index. We should explicitly return None when the index is out of bounds for these sequence types.
| def _slice_batch_value(value, index): | |
| if value is None: | |
| return None | |
| if isinstance(value, torch.Tensor) and value.ndim > 0 and value.shape[0] > index: | |
| return value[index : index + 1] | |
| if isinstance(value, list) and len(value) > index: | |
| return [value[index]] | |
| if isinstance(value, tuple) and len(value) > index: | |
| return (value[index],) | |
| return value | |
| def _slice_batch_value(value, index): | |
| if value is None: | |
| return None | |
| if isinstance(value, torch.Tensor) and value.ndim > 0: | |
| return value[index : index + 1] if value.shape[0] > index else None | |
| if isinstance(value, list): | |
| return [value[index]] if len(value) > index else None | |
| if isinstance(value, tuple): | |
| return (value[index],) if len(value) > index else None | |
| return value |
Summary
inference_prepareRoot Cause
inference_llmskipped_inference_llm_batchwhenctc_decoderexisted, but then passed all VAD segments intoinference_prepare, which intentionally rejects multi-item input withNotImplementedError("batch decoding is not implemented").Verification
python3 -m pytest tests/test_fun_asr_nano_ctc_batch_fallback.py -qfailed withNotImplementedError: batch decoding is not implementedbefore the fixpython3 -m pytest tests/test_fun_asr_nano_ctc_batch_fallback.py tests/test_fun_asr_nano_autocast_device.py tests/test_fun_asr_nano_openai_response.py tests/test_fun_asr_nano_repetition_penalty.py -q-> 13 passedpython3 -m py_compile funasr/models/fun_asr_nano/model.py tests/test_fun_asr_nano_ctc_batch_fallback.pygit diff --cached --check -- funasr/models/fun_asr_nano/model.py tests/test_fun_asr_nano_ctc_batch_fallback.py