Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/deployment_matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Use this page to choose the shortest deployment path for a product, demo, benchm
| Runtime WebSocket service | Live captions, meetings, call-center streams | [Runtime service docs](../runtime/readme.md) | Use when partial results, endpointing, or long-lived audio streams matter. |
| ONNX/C++ runtime | High-concurrency CPU services or embedded realtime ASR | [ONNX runtime docs](../runtime/onnxruntime/readme.md) | Keep this path when latency/concurrency is already proven; add text post-processing for fixed business terms before moving to GPU LLMs. |
| vLLM acceleration | Higher-throughput LLM-based ASR with Fun-ASR-Nano | [vLLM guide](./vllm_guide.md) | Use for LLM decoder throughput; does not apply to non-autoregressive Paraformer. |
| MOSS-Transcribe-Diarize | Long-form multi-speaker transcription with timestamps and speaker labels | [Third-party MOSS deployment guide](./moss_transcribe_diarize.md) | OpenMOSS Apache-2.0 model integrated with FunASR `AutoModel`; choose local HF (`backend="hf"`) or vLLM (`backend="vllm"`), or serve it independently through native SGLang Omni. SGLang Omni is not an `AutoModel` backend. The model remains published and maintained by OpenMOSS. |
| MOSS-Transcribe-Diarize | Long-form multi-speaker transcription with timestamps and speaker labels | [Third-party MOSS deployment guide](./moss_transcribe_diarize.md) | OpenMOSS Apache-2.0 model integrated with FunASR `AutoModel`; choose local HF (`backend="hf"`), vLLM (`backend="vllm"`), or SGLang Omni (`backend="sglang"`). The model remains published and maintained by OpenMOSS. |
| MCP server | Claude/Cursor/desktop agent speech tools | [MCP example](../examples/mcp_server/) | Good when the ASR result should be exposed as a local tool. |
| Subtitle generator | SRT/VTT from long audio or video | [Subtitle example](../examples/subtitle/) | Use verbose segments and speaker labels when readability matters. |
| Batch ASR script | Archives, meetings, datasets, repeated offline runs | [Batch example](../examples/batch_asr_improved.py) | Add queueing, manifests, and retry logs for production use. |
Expand Down
2 changes: 1 addition & 1 deletion docs/deployment_matrix_ja.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
| Kubernetes API | Cluster service 向け internal speech API | [Kubernetes template](../examples/openai_api/kubernetes/) | private `ClusterIP` から開始。公開範囲を広げる前に auth、TLS、network policy、GPU scheduling を追加します。 |
| Runtime WebSocket service | Live captions、meeting、call-center stream | [Runtime service docs](../runtime/readme.md) | partial result、endpointing、long-lived audio stream が重要な場合に使います。 |
| vLLM acceleration | Fun-ASR-Nano の LLM-based ASR throughput 向上 | [vLLM guide](./vllm_guide.md) | LLM decoder throughput 向け。non-autoregressive Paraformer には適用しません。 |
| MOSS-Transcribe-Diarize | 長時間の複数話者 transcription、timestamp、speaker label | [Third-party MOSS guide](./moss_transcribe_diarize.md) | OpenMOSS の Apache-2.0 model を FunASR `AutoModel` に統合済みです。local HF(`backend="hf"`)または vLLM(`backend="vllm"`)を選択でき、native SGLang Omni から独立して serve することもできます。SGLang Omni は `AutoModel` backend ではなく、model の公開・保守主体は OpenMOSS のままです。 |
| MOSS-Transcribe-Diarize | 長時間の複数話者 transcription、timestamp、speaker label | [Third-party MOSS guide](./moss_transcribe_diarize.md) | OpenMOSS の Apache-2.0 model を FunASR `AutoModel` に統合済みです。local HF(`backend="hf"`)vLLM(`backend="vllm"`)、または SGLang Omni(`backend="sglang"`)を選択できます。model の公開・保守主体は OpenMOSS のままです。 |
| MCP server | Claude/Cursor/desktop agent の speech tool | [MCP example](../examples/mcp_server/) | ASR 結果を local tool として Agent に渡したい場合に便利です。 |
| Subtitle generator | 長時間 audio/video から SRT/VTT 作成 | [Subtitle example](../examples/subtitle/) | readability が重要な場合は verbose segment と speaker label を使います。 |
| Batch ASR script | Archive、meeting、dataset、繰り返し offline run | [Batch example](../examples/batch_asr_improved.py) | production では queue、manifest、retry log を追加してください。 |
Expand Down
2 changes: 1 addition & 1 deletion docs/deployment_matrix_ko.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
| Kubernetes API | Cluster service용 internal speech API | [Kubernetes template](../examples/openai_api/kubernetes/) | private `ClusterIP`부터 시작합니다. 범위를 넓히기 전에 auth, TLS, network policy, GPU scheduling을 추가하세요. |
| Runtime WebSocket service | Live captions, meeting, call-center stream | [Runtime service docs](../runtime/readme.md) | partial result, endpointing, long-lived audio stream이 중요할 때 사용합니다. |
| vLLM acceleration | Fun-ASR-Nano의 LLM-based ASR throughput 향상 | [vLLM guide](./vllm_guide.md) | LLM decoder throughput용입니다. non-autoregressive Paraformer에는 적용되지 않습니다. |
| MOSS-Transcribe-Diarize | 긴 다중 화자 transcription, timestamp, speaker label | [Third-party MOSS guide](./moss_transcribe_diarize.md) | OpenMOSS의 Apache-2.0 model이며 FunASR `AutoModel`에 통합되어 있습니다. local HF(`backend="hf"`) 또는 vLLM(`backend="vllm"`)을 선택하거나 native SGLang Omni에서 독립적으로 serve할 수 있습니다. SGLang Omni는 `AutoModel` backend가 아니며, model의 공개 및 유지 관리는 계속 OpenMOSS가 담당합니다. |
| MOSS-Transcribe-Diarize | 긴 다중 화자 transcription, timestamp, speaker label | [Third-party MOSS guide](./moss_transcribe_diarize.md) | OpenMOSS의 Apache-2.0 model이며 FunASR `AutoModel`에 통합되어 있습니다. local HF(`backend="hf"`), vLLM(`backend="vllm"`) 또는 SGLang Omni(`backend="sglang"`)를 선택할 수 있습니다. model의 공개 및 유지 관리는 계속 OpenMOSS가 담당합니다. |
| MCP server | Claude/Cursor/desktop agent speech tool | [MCP example](../examples/mcp_server/) | ASR 결과를 local tool로 Agent에 전달할 때 유용합니다. |
| Subtitle generator | 긴 audio/video에서 SRT/VTT 생성 | [Subtitle example](../examples/subtitle/) | readability가 중요하면 verbose segment와 speaker label을 사용합니다. |
| Batch ASR script | Archive, meeting, dataset, 반복 offline run | [Batch example](../examples/batch_asr_improved.py) | production에서는 queue, manifest, retry log를 추가하세요. |
Expand Down
2 changes: 1 addition & 1 deletion docs/deployment_matrix_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
| Runtime WebSocket 服务 | 实时字幕、会议、客服流式音频 | [Runtime 服务文档](../runtime/readme_cn.md) | 需要中间结果、断句或长连接音频流时选择。 |
| ONNX/C++ Runtime | 高并发 CPU 服务或嵌入式实时 ASR | [ONNX Runtime 文档](../runtime/onnxruntime/readme.md) | 如果延迟和并发已经验证,不要轻易替换;固定业务词优先做文本后处理。 |
| vLLM 加速 | Fun-ASR-Nano 等 LLM-based ASR 高吞吐 | [vLLM 指南](./vllm_guide.md) | 适合 LLM 解码吞吐;不适用于非自回归 Paraformer。 |
| MOSS-Transcribe-Diarize | 长音频多人转写、时间戳和说话人标签 | [第三方 MOSS 部署指南](./moss_transcribe_diarize.md) | OpenMOSS Apache-2.0 模型,已接入 FunASR `AutoModel`;可选本地 HF(`backend="hf"`)vLLM(`backend="vllm"`),也可通过原生 SGLang Omni 独立服务。SGLang Omni 不是 `AutoModel` backend。模型仍由 OpenMOSS 发布和维护。 |
| MOSS-Transcribe-Diarize | 长音频多人转写、时间戳和说话人标签 | [第三方 MOSS 部署指南](./moss_transcribe_diarize.md) | OpenMOSS Apache-2.0 模型,已接入 FunASR `AutoModel`;可选本地 HF(`backend="hf"`)vLLM(`backend="vllm"`) SGLang Omni(`backend="sglang"`)。模型仍由 OpenMOSS 发布和维护。 |
| MCP 服务 | Claude/Cursor/桌面 Agent 语音工具 | [MCP 示例](../examples/mcp_server/) | 适合把 ASR 结果暴露成一个本地工具。 |
| 字幕生成 | 从长音频或视频生成 SRT/VTT | [字幕示例](../examples/subtitle/) | 需要可读性时使用 verbose segments 和说话人标签。 |
| 批处理脚本 | 录音归档、会议纪要、数据集处理 | [批处理示例](../examples/batch_asr_improved.py) | 生产使用时建议增加队列、manifest 和重试日志。 |
Expand Down
39 changes: 37 additions & 2 deletions docs/moss_transcribe_diarize.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ This guide connects the third-party
[OpenMOSS/MOSS-Transcribe-Diarize](https://github.com/OpenMOSS/MOSS-Transcribe-Diarize)
model to the FunASR deployment ecosystem. The model is published by OpenMOSS
under Apache-2.0; it is not a FunASR model. FunASR provides an adapter for its
public Transformers and vLLM interfaces while retaining the OpenMOSS model
name, license, and upstream revision.
public Transformers, vLLM, and SGLang Omni interfaces while retaining the
OpenMOSS model name, license, and upstream revision.

MOSS-Transcribe-Diarize jointly generates transcription, timestamps, and
speaker labels such as `[S01]`. An application therefore does not need to
Expand Down Expand Up @@ -259,6 +259,16 @@ claims as measurements of your host.
Follow the pinned upstream SGLang Omni installation guide for CUDA 13, then
download the immutable model snapshot and serve the local directory:

```bash
git clone https://github.com/sgl-project/sglang-omni.git
git -C sglang-omni checkout 3f819f9cdae3d4eeec22f73306c9067a1ec2542e
```

This source pin includes the transcription API's `max_new_tokens` forwarding.
The original #914 merge predates that request field, so it is not sufficient
for the long-audio command below even though its published H100 benchmark
remains useful upstream evidence.

```bash
hf download OpenMOSS-Team/MOSS-Transcribe-Diarize \
--revision e8681d68e7042738ffca8ac8212bc8fcb1131ab8 \
Expand Down Expand Up @@ -287,6 +297,31 @@ current `verbose_json` contract, the speaker identifier is retained as the
Parse and validate that prefix before wiring the response into subtitles,
meeting notes, or analytics.

The FunASR adapter performs that validation and maps the official SGLang
segments into the same `sentence_info` contract as the HF and vLLM paths:

```python
from funasr import AutoModel

model = AutoModel(
model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
backend="sglang",
sglang_base_url="http://127.0.0.1:8898/v1",
sglang_model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
max_new_tokens=65536,
disable_update=True,
)
result = model.generate(input="audio.wav", max_new_tokens=65536)[0]
for segment in result["sentence_info"]:
print(segment["start"], segment["end"], segment["spk"], segment["text"])
```

Do not pass `vad_model` or `spk_model`: MOSS performs segmentation and
anonymous speaker attribution jointly, and external splitting can destroy
speaker consistency across long turns. The adapter preserves the upstream
tagged transcript in `raw_text`, strips only the validated `[Sxx]` prefix from
each normalized segment, and fails closed if SGLang omits that prefix.

The native runtime was merged in SGLang Omni
[#914](https://github.com/sgl-project/sglang-omni/pull/914). Its single-H100
Seed-TTS EN benchmark completed 1088/1088 clips with no request failures. WER
Expand Down
37 changes: 35 additions & 2 deletions docs/moss_transcribe_diarize_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
本文把第三方
[OpenMOSS/MOSS-Transcribe-Diarize](https://github.com/OpenMOSS/MOSS-Transcribe-Diarize)
模型接入 FunASR 部署生态。模型由 OpenMOSS 以 Apache-2.0 发布,不是 FunASR
自有模型。FunASR 只为其公开的 Transformers 与 vLLM 接口提供适配器,并保留
OpenMOSS 模型名称、许可证和上游 revision。
自有模型。FunASR 为其公开的 Transformers、vLLMSGLang Omni 接口提供适配器,
并保留 OpenMOSS 模型名称、许可证和上游 revision。

MOSS-Transcribe-Diarize 会联合生成转写、时间戳和 `[S01]` 等说话人标签,应用侧
不必再拼接外部 VAD、ASR 和 diarization 管线。这里描述的是部署形态,不表示模型
Expand Down Expand Up @@ -231,6 +231,15 @@ LocalAI `master@a7cc5873ef5b7c909fc9ff7d349d51738ba9bb05` 已包含

按固定上游 SGLang Omni 安装指南准备 CUDA 13 环境,然后下载不可变模型快照并从本地目录启动:

```bash
git clone https://github.com/sgl-project/sglang-omni.git
git -C sglang-omni checkout 3f819f9cdae3d4eeec22f73306c9067a1ec2542e
```

该源码 pin 已把 `max_new_tokens` 传入 transcription 生成请求。最初的 #914
merge 早于这个请求字段,因此虽然它的 H100 benchmark 仍可作为上游证据,却不能
支撑下方长音频命令。

```bash
hf download OpenMOSS-Team/MOSS-Transcribe-Diarize \
--revision e8681d68e7042738ffca8ac8212bc8fcb1131ab8 \
Expand All @@ -257,6 +266,30 @@ curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions \
`verbose_json` 合同把说话人编号保留为 `segments[].text` 的 `[Sxx]` 前缀,
并没有单独的 `speaker` 字段。接入字幕、会议纪要或分析系统前,应解析并校验此前缀。

FunASR 适配器会完成该校验,并把 SGLang 官方 segments 映射为与 HF、vLLM
一致的 `sentence_info` 合同:

```python
from funasr import AutoModel

model = AutoModel(
model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
backend="sglang",
sglang_base_url="http://127.0.0.1:8898/v1",
sglang_model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
max_new_tokens=65536,
disable_update=True,
)
result = model.generate(input="audio.wav", max_new_tokens=65536)[0]
for segment in result["sentence_info"]:
print(segment["start"], segment["end"], segment["spk"], segment["text"])
```

不要传入 `vad_model` 或 `spk_model`:MOSS 在一次生成中联合完成分段和匿名说话人
归属,外部分段会破坏长轮次中的说话人一致性。适配器把上游带标签原文保存在
`raw_text`,只从标准化 segment 中移除已经校验的 `[Sxx]` 前缀;如果 SGLang
没有返回此前缀,则明确失败,不会伪造说话人身份。

原生 runtime 已通过 SGLang Omni
[#914](https://github.com/sgl-project/sglang-omni/pull/914) 合并。其单张 H100
Seed-TTS EN benchmark 完成 1088/1088 条请求且无请求失败。WER 是在移除
Expand Down
Loading
Loading