Skip to content

Can SenseVoice works with speaker diarization model like cam++ ? #2706

@shenxiangzhuang

Description

@shenxiangzhuang

With demo code, I got error like "can not get timestamp":

from funasr import AutoModel
from funasr.utils.postprocess_utils import rich_transcription_postprocess

model_dir = "iic/SenseVoiceSmall"

model = AutoModel(
    model=model_dir,
    vad_model="fsmn-vad",
    vad_kwargs={"max_single_segment_time": 30000},
    device="cuda:0",
)

# en
res = model.generate(
    input=f"{model.model_path}/example/en.mp3",
    cache={},
    language="auto",  # "zn", "en", "yue", "ja", "ko", "nospeech"
    use_itn=True,
    batch_size_s=60,
    merge_vad=True,  #
    merge_length_s=15,
)
text = rich_transcription_postprocess(res[0]["text"])
print(text)

Then I tried to set output_timestamp=True, the code like:

self.model = AutoModel(
    model="iic/SenseVoiceSmall",
    output_timestamp=True,
    # vad_model="fsmn-vad",
    # vad_kwargs={"max_single_segment_time": 30000},
    vad_model="fsmn-vad",
    # vad_kwargs={"max_single_segment_time": 180000, "merge_vad": True, "merge_length_s": 15},
    punc_model="ct-punc-c", punc_model_revision="v2.0.4",
    spk_model="cam++", spk_model_revision="v2.0.2",
    device="cuda:0",
)

But I got another error:

  File ".../python3.10/site-packages/funasr/models/campplus/utils.py", line 203, in distribute_spk
    overlap = max(min(sentence_end, spk_ed) - max(sentence_start, spk_st), 0)
TypeError: '>' not supported between instances of 'float' and 'NoneType'

So my question is can SenseVoice works with speaker diarization model like cam++ or my setting should be changed to make it work?

Metadata

Metadata

Assignees

No one assigned

    Labels

    questionFurther information is requested

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions