Skip to content

fix: disable duplicate dynamic silence logic in DynamicStreamingVAD#3240

Merged
LauraGPT merged 5 commits into
modelscope:mainfrom
LiangRx0501:fix/forward-silence-schedule
Jul 16, 2026
Merged

fix: disable duplicate dynamic silence logic in DynamicStreamingVAD#3240
LauraGPT merged 5 commits into
modelscope:mainfrom
LiangRx0501:fix/forward-silence-schedule

Conversation

@LiangRx0501

@LiangRx0501 LiangRx0501 commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Disable the model-level dynamic silence logic when DynamicStreamingVAD manages the threshold through _apply_dynamic_threshold().

This prevents model.py from overwriting the caller-configured VAD threshold on every streaming chunk.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates DynamicStreamingVAD.feed to pass silence_schedule to self.model.generate(). The reviewer identified a conflicting dual-implementation of dynamic silence thresholding where the model's internal logic silently overrides the user-configured thresholds in DynamicStreamingVAD. They suggested passing dynamic_silence=False to self.model.generate() to disable the redundant internal logic and allow DynamicStreamingVAD's implementation to manage the thresholding as designed.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread funasr/models/fsmn_vad_streaming/dynamic_vad.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@LiangRx0501 LiangRx0501 changed the title fix: forward silence_schedule to streaming FSMN-VAD fix: disable duplicate dynamic silence logic in DynamicStreamingVAD Jul 16, 2026

@LauraGPT LauraGPT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing the duplicate threshold controller. I confirmed on the real CPU FSMN-VAD that dynamic_silence=False prevents the model-level schedule from overwriting DynamicStreamingVAD after the cache exists: a custom 3000 ms schedule now remains at a 2850 ms counter threshold instead of being reset to 1850 ms.

There is still a first-call correctness gap that needs to be fixed before merge. _apply_dynamic_threshold() runs before self.model.generate(), so the first call has no cache["stats"] to update. With this patch disabling the model schedule, generate() initializes the first call with the model's default 800 ms end-silence setting, regardless of the wrapper's configured schedule. Because feed() explicitly accepts an arbitrary-length chunk and the model internally slices it, this makes segmentation depend on how the caller chunks identical audio.

Real-model reproduction at ad3f3e402, using the repository's runtime/funasr_api/asr_example.wav plus two seconds of silence and silence_schedule=[(inf, 10000)]:

  • one feed(full_audio): returns [[610, 5440]]; counter threshold is 650 (800 ms total);
  • feed(first_60_ms) followed by feed(rest): returns []; counter threshold is 9850 (10000 ms total).

Please initialize the first model call with the wrapper's desired end-silence setting (for example through the existing max_end_silence_time initialization path, or an equivalent cache initialization) while keeping the model-level dynamic schedule disabled.

Please also add focused regression coverage for both parts of the contract:

  1. the wrapper's custom silence schedule and speech_noise_thres are not overwritten by the model-level defaults;
  2. identical audio produces the same result when supplied as one initial chunk or split after the first streaming chunk.

The current focused suite (tests/test_fsmn_vad_dynamic_silence.py plus tests/test_realtime_ws_service.py) passes 15 tests, but none exercises the DynamicStreamingVAD/model interaction changed by this PR.

Pass the wrapper silence and noise thresholds into the initial FSMN-VAD cache so first-call behavior does not depend on caller chunking. Add a regression that exercises the wrapper with the production cache initializer.

Assisted-by: Codex:gpt-5.6

@LauraGPT LauraGPT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the first-call fix on head 92b749cdf.

  • The initial FSMN-VAD cache now receives the wrapper-configured end-silence and speech_noise_thres values before processing audio.
  • The new wrapper/cache regression failed on the previous head (650 != 9850, and one-chunk [[0, 1000]] versus split []) and passes after the fix.
  • The focused suite passes 17/17 tests; compile, targeted Ruff, and diff checks pass.
  • A real CPU fsmn-vad run with the repository audio plus two seconds of silence produces identical one-chunk and 60 ms-split results ([]), with both caches at threshold 9850 and noise threshold 0.73.

The requested threshold-preservation and chunking-invariance contract is covered.

@LauraGPT
LauraGPT merged commit 33b0403 into modelscope:main Jul 16, 2026
LauraGPT added a commit that referenced this pull request Jul 16, 2026
Preserve the bounded streaming VAD buffer fix alongside the first-call dynamic threshold initialization from #3240.

Assisted-by: Codex:gpt-5.6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants