Skip to content

说话人分离好像没效果啊 #2944

@triumph

Description

@triumph

[我是这么启动程序的]
CUDA_VISIBLE_DEVICES=0 python examples/industrial_data_pretraining/fun_asr_nano/serve_vllm.py --port 8899 --model FunAudioLLM/Fun-ASR-Nano-2512 --gpu-memory-utilization 0.5

客户端这么测试的 2speakers_example.wav
curl -X POST http://localhost:8899/asr -F "file=@2speakers_example.wav" -F "language=中文" -F "spk=true"

返回的结果是【 全是 "speaker": "SPK0" 】,

{
"text": "嗯,那么今天我们就简单的进行一下那个新生招聘的嗯讨论吧。因为现在不是马上就新生到校嘛,然后我们社团呢也需要招聘一些新的社员,然后就今天就大概就讨论一下嗯怎么招聘的内容吧。嗯,我们就首先想一下那个招聘的地点在哪里吧。嗯,地点的话,我们现在可以有三个选择。嗯,第一个的话,我们可以选择在操场,因为那儿嗯学生流动量也挺大的。操场的话,这这段时间太热了,我怕那个人流量有点少。嗯,那我们还可以有第二个选择呀。嗯,我们可以在图书馆楼下那里有一块可以遮阴的地方。嗯,图书馆我觉得应该还可以吧。",
"segments": [
{
"text": "嗯,那么今天我们就简单的进行一下那个新生招聘的嗯讨论吧。因为现在不是马上就新生到校嘛,然后我们社团呢也需要招聘一些新的社员,然后就今天就大概就讨论一下嗯怎么招聘的内容吧。嗯,我们就首先想一下那个招聘的地点在哪里吧。嗯,地点的话,我们现在可以有三个选择。嗯,第一个的话,我们可以选择在操场,因为那儿嗯学生流动量也挺大的。操场的话,这这段时间太热了,我怕那个人流量有点少。嗯,那我们还可以有第二个选择呀。嗯,我们可以在图书馆楼下那里有一块可以遮阴的地方。嗯,图书馆我觉得应该还可以吧。",
"start": 0.08,
"end": 51.64,
"words": [
{
"word": "嗯",
"start": 0.44,
"end": 0.5
},
{
"word": ",",
"start": 0.5,
"end": 0.5599999999999999
},
{
"word": "那",
"start": 0.74,
"end": 0.7999999999999999
},
{
"word": "么",
"start": 0.86,
"end": 0.9199999999999999
},
{
"word": "今",
"start": 0.98,
"end": 1.04
},
{
"word": "天",
"start": 1.1600000000000002,
"end": 1.22
},
{
"word": "我",
"start": 1.34,
"end": 1.52
},
{
"word": "们",
"start": 1.52,
"end": 1.6400000000000002
},
{
"word": "就",
"start": 1.76,
"end": 1.82
},
{
"word": "简",
"start": 2.12,
"end": 2.18
},
{
"word": "单",
"start": 2.24,
"end": 2.3000000000000004
},
{
"word": "的",
"start": 2.42,
"end": 2.48
},
{
"word": "进",
"start": 2.66,
"end": 2.72
},
{
"word": "行",
"start": 2.7800000000000004,
"end": 2.84
},
{
"word": "一",
"start": 2.96,
"end": 3.08
},
{
"word": "下",
"start": 3.08,
"end": 3.2
},
{
"word": "那",
"start": 3.3200000000000005,
"end": 3.38
},
{
"word": "个",
"start": 3.5,
"end": 3.62
},
{
"word": "新",
"start": 3.86,
"end": 3.92
},
{
"word": "生",
"start": 4.1,
"end": 4.16
},
{
"word": "招",
"start": 4.46,
"end": 4.5200000000000009
},
{
"word": "聘",
"start": 4.64,
"end": 4.76
},
{
"word": "的",
"start": 4.94,
"end": 5.0600000000000009
},
{
"word": "嗯",
"start": 5.6,
"end": 5.66
},
{
"word": "讨",
"start": 6.32,
"end": 6.38
},
{
"word": "论",
"start": 6.5,
"end": 6.5600000000000009
},
{
"word": "吧",
"start": 6.74,
"end": 6.8
},
{
"word": "。",
"start": 6.86,
"end": 6.92
},
{
"word": "因",
"start": 6.98,
"end": 7.04
},
{
"word": "为",
"start": 7.1,
"end": 7.16
},
{
"word": "现",
"start": 7.28,
"end": 7.34
},
{
"word": "在",
"start": 7.4,
"end": 7.46
},
{
"word": "不",
"start": 7.64,
"end": 7.7
},
{
"word": "是",
"start": 7.76,
"end": 7.82
},
{
"word": "马",
"start": 8.48,
"end": 8.540000000000001
},
{
"word": "上",
"start": 8.66,
"end": 8.72
},
{
"word": "就",
"start": 8.78,
"end": 8.84
},
{
"word": "新",
"start": 9.02,
"end": 9.08
},
{
"word": "生",
"start": 9.2,
"end": 9.26
},
{
"word": "到",
"start": 9.44,
"end": 9.5
},
{
"word": "校",
"start": 9.68,
"end": 9.74
},
{
"word": "嘛",
"start": 9.98,
"end": 10.040000000000001
},
{
"word": ",",
"start": 10.22,
"end": 10.28
},
{
"word": "然",
"start": 10.46,
"end": 10.52
},
{
"word": "后",
"start": 10.58,
"end": 10.64
},
{
"word": "我",
"start": 10.7,
"end": 10.76
},
{
"word": "们",
"start": 10.76,
"end": 10.82
},
{
"word": "社",
"start": 10.94,
"end": 11.0
},
{
"word": "团",
"start": 11.12,
"end": 11.18
},
{
"word": "呢",
"start": 11.3,
"end": 11.36
},
{
"word": "也",
"start": 11.48,
"end": 11.540000000000001
},
{
"word": "需",
"start": 11.66,
"end": 11.72
},
{
"word": "要",
"start": 11.78,
"end": 11.9
},
{
"word": "招",
"start": 12.02,
"end": 12.08
},
{
"word": "聘",
"start": 12.14,
"end": 12.2
},
{
"word": "一",
"start": 12.32,
"end": 12.44
},
{
"word": "些",
"start": 12.44,
"end": 12.5
},
{
"word": "新",
"start": 12.68,
"end": 12.74
},
{
"word": "的",
"start": 12.86,
"end": 12.92
},
{
"word": "社",
"start": 13.1,
"end": 13.16
},
{
"word": "员",
"start": 13.34,
"end": 13.4
},
{
"word": ",",
"start": 13.7,
"end": 13.76
},
{
"word": "然",
"start": 14.06,
"end": 14.12
},
{
"word": "后",
"start": 14.18,
"end": 14.3
},
{
"word": "就",
"start": 14.42,
"end": 14.48
},
{
"word": "今",
"start": 14.72,
"end": 14.78
},
{
"word": "天",
"start": 14.9,
"end": 14.96
},
{
"word": "就",
"start": 15.08,
"end": 15.14
},
{
"word": "大",
"start": 15.32,
"end": 15.38
},
{
"word": "概",
"start": 15.5,
"end": 15.56
},
{
"word": "就",
"start": 15.68,
"end": 15.74
},
{
"word": "讨",
"start": 15.92,
"end": 15.98
},
{
"word": "论",
"start": 16.099999999999999,
"end": 16.159999999999998
},
{
"word": "一",
"start": 16.279999999999999,
"end": 16.4
},
{
"word": "下",
"start": 16.4,
"end": 16.459999999999999
},
{
"word": "嗯",
"start": 17.06,
"end": 17.119999999999999
},
{
"word": "怎",
"start": 17.36,
"end": 17.419999999999999
},
{
"word": "么",
"start": 17.479999999999998,
"end": 17.54
},
{
"word": "招",
"start": 17.72,
"end": 17.779999999999999
},
{
"word": "聘",
"start": 17.9,
"end": 17.959999999999999
},
{
"word": "的",
"start": 18.08,
"end": 18.2
},
{
"word": "内",
"start": 18.38,
"end": 18.439999999999999
},
{
"word": "容",
"start": 18.5,
"end": 18.619999999999999
},
{
"word": "吧",
"start": 18.74,
"end": 18.86
},
{
"word": "。",
"start": 19.4,
"end": 19.459999999999999
},
{
"word": "嗯",
"start": 19.459999999999999,
"end": 19.52
},
{
"word": ",",
"start": 19.52,
"end": 19.58
},
{
"word": "我",
"start": 19.639999999999998,
"end": 19.759999999999999
},
{
"word": "们",
"start": 19.759999999999999,
"end": 19.88
},
{
"word": "就",
"start": 19.939999999999999,
"end": 20.0
},
{
"word": "首",
"start": 20.24,
"end": 20.299999999999998
},
{
"word": "先",
"start": 20.36,
"end": 20.419999999999999
},
{
"word": "想",
"start": 20.659999999999998,
"end": 20.72
},
{
"word": "一",
"start": 20.84,
"end": 20.959999999999999
},
{
"word": "下",
"start": 20.959999999999999,
"end": 21.02
},
{
"word": "那",
"start": 21.139999999999998,
"end": 21.2
},
{
"word": "个",
"start": 21.259999999999999,
"end": 21.38
},
{
"word": "招",
"start": 21.799999999999998,
"end": 21.86
},
{
"word": "聘",
"start": 21.979999999999998,
"end": 22.04
},
{
"word": "的",
"start": 22.159999999999998,
"end": 22.22
},
{
"word": "地",
"start": 22.279999999999999,
"end": 22.4
},
{
"word": "点",
"start": 22.459999999999999,
"end": 22.58
},
{
"word": "在",
"start": 22.759999999999999,
"end": 22.819999999999998
},
{
"word": "哪",
"start": 22.939999999999999,
"end": 23.0
},
{
"word": "里",
"start": 23.06,
"end": 23.119999999999999
},
{
"word": "吧",
"start": 23.24,
"end": 23.299999999999998
},
{
"word": "。",
"start": 24.38,
"end": 24.439999999999999
},
{
"word": "嗯",
"start": 24.439999999999999,
"end": 24.5
},
{
"word": ",",
"start": 24.5,
"end": 24.56
},
{
"word": "地",
"start": 24.68,
"end": 24.74
},
{
"word": "点",
"start": 24.799999999999998,
"end": 24.86
},
{
"word": "的",
"start": 24.979999999999998,
"end": 25.04
},
{
"word": "话",
"start": 25.04,
"end": 25.099999999999999
},
{
"word": ",",
"start": 25.099999999999999,
"end": 25.159999999999998
},
{
"word": "我",
"start": 25.279999999999999,
"end": 25.34
},
{
"word": "们",
"start": 25.34,
"end": 25.4
},
{
"word": "现",
"start": 25.52,
"end": 25.58
},
{
"word": "在",
"start": 25.7,
"end": 25.819999999999998
},
{
"word": "可",
"start": 26.06,
"end": 26.119999999999999
},
{
"word": "以",
"start": 26.119999999999999,
"end": 26.24
},
{
"word": "有",
"start": 26.299999999999998,
"end": 26.36
},
{
"word": "三",
"start": 26.479999999999998,
"end": 26.54
},
{
"word": "个",
"start": 26.599999999999999,
"end": 26.659999999999998
},
{
"word": "选",
"start": 26.84,
"end": 26.9
},
{
"word": "择",
"start": 27.02,
"end": 27.139999999999998
},
{
"word": "。",
"start": 27.619999999999999,
"end": 27.68
},
{
"word": "嗯",
"start": 27.74,
"end": 27.799999999999998
},
{
"word": ",",
"start": 27.86,
"end": 27.919999999999999
},
{
"word": "第",
"start": 27.979999999999998,
"end": 28.099999999999999
},
{
"word": "一",
"start": 28.099999999999999,
"end": 28.159999999999998
},
{
"word": "个",
"start": 28.159999999999998,
"end": 28.279999999999999
},
{
"word": "的",
"start": 28.34,
"end": 28.4
},
{
"word": "话",
"start": 28.4,
"end": 28.459999999999999
},
{
"word": ",",
"start": 28.459999999999999,
"end": 28.52
},
{
"word": "我",
"start": 28.58,
"end": 28.7
},
{
"word": "们",
"start": 28.7,
"end": 28.759999999999999
},
{
"word": "可",
"start": 28.759999999999999,
"end": 28.88
},
{
"word": "以",
"start": 28.88,
"end": 29.0
},
{
"word": "选",
"start": 29.06,
"end": 29.119999999999999
},
{
"word": "择",
"start": 29.24,
"end": 29.299999999999998
},
{
"word": "在",
"start": 29.599999999999999,
"end": 29.72
},
{
"word": "操",
"start": 30.439999999999999,
"end": 30.5
},
{
"word": "场",
"start": 30.68,
"end": 30.74
},
{
"word": ",",
"start": 30.919999999999999,
"end": 30.979999999999998
},
{
"word": "因",
"start": 31.099999999999999,
"end": 31.159999999999998
},
{
"word": "为",
"start": 31.22,
"end": 31.279999999999999
},
{
"word": "那",
"start": 31.52,
"end": 31.58
},
{
"word": "儿",
"start": 31.7,
"end": 31.759999999999999
},
{
"word": "嗯",
"start": 32.66,
"end": 32.72
},
{
"word": "学",
"start": 33.08,
"end": 33.14
},
{
"word": "生",
"start": 33.26,
"end": 33.32
},
{
"word": "流",
"start": 33.5,
"end": 33.559999999999998
},
{
"word": "动",
"start": 33.62,
"end": 33.68
},
{
"word": "量",
"start": 33.8,
"end": 33.86
},
{
"word": "也",
"start": 33.98,
"end": 34.04
},
{
"word": "挺",
"start": 34.1,
"end": 34.16
},
{
"word": "大",
"start": 34.339999999999999,
"end": 34.4
},
{
"word": "的",
"start": 34.519999999999999,
"end": 34.58
},
{
"word": "。",
"start": 34.879999999999998,
"end": 34.94
},
{
"word": "操",
"start": 35.059999999999998,
"end": 35.12
},
{
"word": "场",
"start": 35.3,
"end": 35.36
},
{
"word": "的",
"start": 35.54,
"end": 35.6
},
{
"word": "话",
"start": 35.66,
"end": 35.72
},
{
"word": ",",
"start": 36.44,
"end": 36.5
},
{
"word": "这",
"start": 36.5,
"end": 36.559999999999998
},
{
"word": "这",
"start": 36.8,
"end": 36.86
},
{
"word": "段",
"start": 36.98,
"end": 37.04
},
{
"word": "时",
"start": 37.16,
"end": 37.22
},
{
"word": "间",
"start": 37.28,
"end": 37.4
},
{
"word": "太",
"start": 37.58,
"end": 37.64
},
{
"word": "热",
"start": 37.82,
"end": 37.879999999999998
},
{
"word": "了",
"start": 38.059999999999998,
"end": 38.12
},
{
"word": ",",
"start": 38.18,
"end": 38.239999999999998
},
{
"word": "我",
"start": 38.3,
"end": 38.36
},
{
"word": "怕",
"start": 38.48,
"end": 38.54
},
{
"word": "那",
"start": 39.199999999999999,
"end": 39.26
},
{
"word": "个",
"start": 39.32,
"end": 39.379999999999998
},
{
"word": "人",
"start": 39.44,
"end": 39.5
},
{
"word": "流",
"start": 39.62,
"end": 39.68
},
{
"word": "量",
"start": 39.8,
"end": 39.86
},
{
"word": "有",
"start": 40.1,
"end": 40.16
},
{
"word": "点",
"start": 40.22,
"end": 40.339999999999999
},
{
"word": "少",
"start": 40.519999999999999,
"end": 40.58
},
{
"word": "。",
"start": 40.82,
"end": 40.879999999999998
},
{
"word": "嗯",
"start": 41.18,
"end": 41.239999999999998
},
{
"word": ",",
"start": 41.239999999999998,
"end": 41.3
},
{
"word": "那",
"start": 41.48,
"end": 41.54
},
{
"word": "我",
"start": 41.6,
"end": 41.72
},
{
"word": "们",
"start": 41.72,
"end": 41.78
},
{
"word": "还",
"start": 41.9,
"end": 41.96
},
{
"word": "可",
"start": 42.019999999999999,
"end": 42.08
},
{
"word": "以",
"start": 42.08,
"end": 42.14
},
{
"word": "有",
"start": 42.199999999999999,
"end": 42.26
},
{
"word": "第",
"start": 42.32,
"end": 42.379999999999998
},
{
"word": "二",
"start": 42.379999999999998,
"end": 42.5
},
{
"word": "个",
"start": 42.5,
"end": 42.62
},
{
"word": "选",
"start": 42.739999999999998,
"end": 42.8
},
{
"word": "择",
"start": 42.86,
"end": 42.92
},
{
"word": "呀",
"start": 43.1,
"end": 43.16
},
{
"word": "。",
"start": 43.82,
"end": 43.879999999999998
},
{
"word": "嗯",
"start": 43.879999999999998,
"end": 43.94
},
{
"word": ",",
"start": 44.239999999999998,
"end": 44.3
},
{
"word": "我",
"start": 44.42,
"end": 44.48
},
{
"word": "们",
"start": 44.54,
"end": 44.6
},
{
"word": "可",
"start": 44.66,
"end": 44.72
},
{
"word": "以",
"start": 44.78,
"end": 44.9
},
{
"word": "在",
"start": 45.019999999999999,
"end": 45.08
},
{
"word": "图",
"start": 45.32,
"end": 45.379999999999998
},
{
"word": "书",
"start": 45.44,
"end": 45.5
},
{
"word": "馆",
"start": 45.62,
"end": 45.68
},
{
"word": "楼",
"start": 45.86,
"end": 45.92
},
{
"word": "下",
"start": 46.04,
"end": 46.1
},
{
"word": "那",
"start": 46.64,
"end": 46.699999999999999
},
{
"word": "里",
"start": 46.82,
"end": 46.879999999999998
},
{
"word": "有",
"start": 46.94,
"end": 47.0
},
{
"word": "一",
"start": 47.059999999999998,
"end": 47.12
},
{
"word": "块",
"start": 47.239999999999998,
"end": 47.3
},
{
"word": "可",
"start": 47.48,
"end": 47.54
},
{
"word": "以",
"start": 47.54,
"end": 47.66
},
{
"word": "遮",
"start": 47.72,
"end": 47.78
},
{
"word": "阴",
"start": 47.9,
"end": 47.96
},
{
"word": "的",
"start": 48.08,
"end": 48.14
},
{
"word": "地",
"start": 48.199999999999999,
"end": 48.26
},
{
"word": "方",
"start": 48.32,
"end": 48.379999999999998
},
{
"word": "。",
"start": 48.739999999999998,
"end": 48.8
},
{
"word": "嗯",
"start": 48.98,
"end": 49.04
},
{
"word": ",",
"start": 49.339999999999999,
"end": 49.4
},
{
"word": "图",
"start": 49.4,
"end": 49.46
},
{
"word": "书",
"start": 49.519999999999999,
"end": 49.58
},
{
"word": "馆",
"start": 49.699999999999999,
"end": 49.76
},
{
"word": "我",
"start": 50.059999999999998,
"end": 50.12
},
{
"word": "觉",
"start": 50.18,
"end": 50.239999999999998
},
{
"word": "得",
"start": 50.3,
"end": 50.36
},
{
"word": "应",
"start": 50.48,
"end": 50.54
},
{
"word": "该",
"start": 50.6,
"end": 50.66
},
{
"word": "还",
"start": 50.78,
"end": 50.839999999999999
},
{
"word": "可",
"start": 51.019999999999999,
"end": 51.08
},
{
"word": "以",
"start": 51.14,
"end": 51.199999999999999
},
{
"word": "吧",
"start": 51.32,
"end": 51.379999999999998
},
{
"word": "。",
"start": 51.559999999999998,
"end": 51.62
}
],
"speaker": "SPK0"
}
],
"duration": 51.66275,
"processing_time": 1.108,
"rtf": 0.0214
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingneeds triageNeeds maintainer triage and routing

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions