FAQ

Spark-X2.5 FAQ · 常见问题

本页整理常见问题、维护者答复和社区经验，最初迁自 issue #10，此后持续补充社区交流群中的高频问题。各条目附有来源，具体环境与后续讨论请查看原链接。

This FAQ collects common questions, maintainer answers and community reports. It was first migrated from issue #10 and is updated with recurring questions from the community support groups. Follow each source link for the environment, limitations and ongoing discussion.

最后核对 / Last checked: 2026-10-06.

中文

1. vLLM 启动失败，提示不认识 `Spark2_5ForCausalLM`

通用 vLLM 镜像不会自动注册 Spark2_5 架构，必须在运行 vLLM 的同一个 Python 环境里安装 Spark 插件：

uv pip install vllm-spark2_5-plugin
# 或从源码安装
git clone https://github.com/XHToken/Spark-plugin.git
cd ./Spark-plugin && pip install .

插件仓库：https://github.com/XHToken/Spark-plugin
PyPI：https://pypi.org/project/vllm-spark2-5-plugin
昇腾（Ascend）环境建议加 --no-deps 安装，避免覆盖镜像内已有的 vLLM-Ascend。
官方 Docker 镜像 vllm/vllm-openai 也是如此：直接挂载模型启动会无法加载，需要先在容器内安装插件（或基于该镜像构建一个装好插件的镜像）。

来源：#6；Spark-X2.5-4B Discussion #23

2. llama.cpp 报 `unknown model architecture: 'spark2_5'`

上游 PR ggml-org/llama.cpp#27868 已于 2026-09-06 合入，从 release b10828 起原生支持 Spark2_5ForCausalLM。

升级到 b10828 或更新版本并重新构建，可获得上游的 Spark2_5 架构支持。具体后端、量化文件和运行结果仍需在目标环境验证。

来源：#4

3. 怎么用 LLaMA-Factory 微调？

Spark-X2.5 的支持尚未合入上游 LLaMA-Factory，请使用官方 fork：

微调指南：examples/torch_spark2_5/Spark2_5_Finetune.md

来源：#7

4. 1M 上下文能跑通，但"大海捞针"检索准确率很低？

维护者对 Spark-X2.5-1.7B 使用 200 条中英文 probe 复测时，关闭 thinking 明显改善了检索准确率：

Prompt tokens	thinking=true, max_tokens=2048	thinking=false, max_tokens=2048
32,000	59.5%	98.0%
320,000	68.0%	96.5%
1,000,000	52.5%	82.0%

对"大海捞针"这类简单信息提取任务：

建议关闭 thinking——问题本身不需要长推理，开启后反而有相当比例的回答被 max_tokens 截断而失分；
如果确实需要 thinking，请把 max_tokens 调大，避免思考过程把答案挤掉。

在 1M 上下文下，thinking=true、max_tokens=32768 的复测准确率为 60.5%。这些结果对应上述测试配置，不代表所有长上下文任务；模型、注意力配置与推理环境也会影响结果。

来源：#2 维护者复测及完整讨论。

5. 能在 RK3588 上运行吗？NPU 支持确认了吗？

已有社区开发者反馈在 RK3588 上成功运行：

硬件：RK3588（4×Cortex-A76 + 4×Cortex-A55，内置 6 TOPS NPU），8GB 内存
模型：Spark-X2.5-4B，Q8 量化
生成速度：约 5.9 tokens/s

这是维护者转述的社区运行反馈；具体推理框架、部署步骤和 NPU 适配细节仍待确认，尚不足以提供可复现的 NPU 部署方案。

来源：#8

6. 推荐的采样参数是什么？

temperature=1.0、top_p=0.95、top_k=-1。README 中的评测结果均在 thinking 模式下取得。

来源：README 评测说明。

7. 国内网络怎么下载模型？

除 Hugging Face 外还有多个镜像渠道，按环境选择：

渠道	适用场景
ModelScope / AtomGit	国内网络
Modelers	昇腾芯片
SCNet	海光芯片
Ollama / LM Studio	本地快速运行

来源：README 下载链接。

8. 推理速度和显存大概什么水平？

社区对 Spark-X2.5-1.7B BF16 在 H200（batch=1，TP=1）上的测试结果供参考：

1M 上下文的端到端吞吐（含 prefill）：SGLang 30.20 tokens/s，Transformers 11.84 tokens/s。
对应显存读数：SGLang 63.34 GiB，Transformers 124.51 GiB。两者分别用 nvidia-smi 与 torch.cuda.max_memory_allocated 测量，且存在内存池预分配差异，不能直接视为严格同口径的显存节省比例。
同一次测试中，4B 模型在 1M 上下文下显存不足（OOM）。

社区实测数据，非官方性能指标。来源：#5

9. 怎么用 Ollama、LM Studio、Unsloth 或手机 App 直接运行？

以下工具已原生支持 Spark-X2.5，无需再自行编译 fork 或替换运行时：

Ollama：更新到 v0.34.1 或以上（2026-09-15 起原生支持），然后一键运行：
```
ollama run SparkLLM/Spark-X2.5-1.7B
# 或
ollama run SparkLLM/Spark-X2.5-4B
```
Ollama 模型库中的这两个模型目前只有 latest 一个标签，为未量化权重（4B 约 8.2 GB）。显存或内存有限时，可以用 Ollama 直接拉取 Hugging Face 上的官方 GGUF 量化，例如 ollama run hf.co/XHToken/Spark-X2.5-4B-GGUF:Q4_K_M。
LM Studio：0.4.0 或以上，搜索 "Spark-X2.5" 下载后即可对话；命令行可用 lms chat spark-x2.5。
Unsloth Studio：最新版支持 Spark-X2.5 GGUF 原生推理。
PocketPal AI：iOS（App Store）与 Android（Google Play）版已原生支持，可在手机上本地运行。

来源：README Release 记录与本地运行说明。

10. 显存 / 内存有限，该选哪个版本？模型文件大小就是需要的显存吗？

官方 GGUF 文件大小参考：

模型	Q4_K_M	Q8_0	未量化（BF16）
Spark-X2.5-4B	2.6 GB	4.4 GB	8.2 GB
Spark-X2.5-1.7B	1.1 GB	1.8 GB	3.4 GB

同一种运行时只需下载一个量化文件，不必把所有版本都下载下来。
文件大小不等于所需显存：运行时还要为上下文的 KV Cache 等分配内存，而且上下文越长占用越多。显存较小时请先限制上下文长度（如 llama.cpp 的 -c、Ollama 的 num_ctx）。
4 GB 左右显存建议先试 4B Q4_K_M 或 1.7B Q8_0；显存不够时，运行时可能退回 CPU 推理，速度会明显下降。
使用 vLLM / SGLang 时，也可以选择 Hugging Face 上的 FP8 / INT8 权重。

来源：Spark-X2.5-4B-GGUF、Spark-X2.5-1.7B-GGUF 文件列表；社区交流群答疑。

11. 16 GB 显存大约能跑多长的上下文？

社区经验：4B 量化模型配合 KV Cache Q8 量化，16 GB 显存大约可以跑到 500K 上下文。实际上限还与推理框架、量化方式和并发数有关，请以自己环境的实测为准。手机等低内存端侧设备目前还没有可参考的长上下文实测数据，欢迎补充。

社区实测经验，非官方性能指标。来源：社区交流群反馈。

12. 接入 OpenClaw 等需要工具调用的 Agent 时，提示模型不支持 tools？

有社区用户（2026-09-08）通过 Ollama 接入时发现，模型的 capabilities 只有 completion、没有 tools，因此只能用于纯对话，无法驱动需要工具调用的 Agent；当时的服务器仅用 CPU 推理，单次响应约 80 秒，也难以支撑 Agent 的多轮调用。

需要工具调用时，建议使用已配置工具调用解析器的 OpenAI 兼容服务：

SGLang：README 中的启动示例已包含 --tool-call-parser spark25；
vLLM：安装 Spark 插件后，启动时加上 --enable-auto-tool-choice --tool-call-parser spark25（解析器名称是 spark25，不是 spark2_5）。

更新（2026-10）：在 Ollama 0.35.0 上，用官方 Q4_K_M GGUF 创建的模型（Modelfile 中只有 FROM Spark-X2.5-1.7B-Q4_K_M.gguf）实测：/api/show 返回的 capabilities 包含 tools、thinking 和 completion，/api/chat 带工具调用时能正常返回 tool_calls。也就是说，新版 Ollama 上的工具调用本身是可用的。不过 ollama.com 上 SparkLLM/Spark-X2.5-* 的模型页目前没有显示 tools / thinking 标记，按工具调用筛选时搜不到，这一点在 #13 中跟进。

来源：社区交流群反馈；#13；README SGLang 部署示例；Spark-plugin README。

13. 日常对话时 thinking 会反复自问、陷入循环？

thinking 默认开启。社区用户反馈，在日常问答中它偶尔会反复自我追问、陷入循环，而关闭 thinking 后回答更稳定。对于不需要长推理的请求，可以按请求关闭：

"chat_template_kwargs": {"enable_thinking": false}

这与第 4 条"简单信息提取任务建议关闭 thinking"的结论一致。另有用户反馈，在 Q4_K_M 量化、128K 上下文下开启 thinking 做编码任务时，模型会在细节决策上反复"语义循环"，浪费上万 token（Spark-X2.5-4B Discussion #26）；这类场景可以对比 Q8_0 量化或关闭 thinking 的表现。如果能稳定复现循环，请附最小 prompt、模型版本、推理框架与原始输出提交 issue。

来源：README 关于关闭 thinking 的说明；社区交流群反馈。

English

1. vLLM fails to start — `Spark2_5ForCausalLM` not recognized

A stock vLLM image does not auto-register the Spark2_5 architecture. Install the Spark plugin into the same Python environment that runs vLLM:

uv pip install vllm-spark2_5-plugin
# or from source
git clone https://github.com/XHToken/Spark-plugin.git
cd ./Spark-plugin && pip install .

Plugin repo: https://github.com/XHToken/Spark-plugin
PyPI: https://pypi.org/project/vllm-spark2-5-plugin
On Ascend, install with --no-deps so the plugin does not overwrite the vLLM-Ascend build already in the image.
The official vllm/vllm-openai Docker image is no exception: mounting the model and starting it directly fails to load. Install the plugin inside the container first, or build an image on top of it with the plugin installed.

Source: #6; Spark-X2.5-4B Discussion #23

2. llama.cpp reports `unknown model architecture: 'spark2_5'`

Upstream PR ggml-org/llama.cpp#27868 was merged on 2026-09-06; Spark2_5ForCausalLM is natively supported from release b10828 onward.

Upgrade to b10828 or newer and rebuild for upstream Spark2_5 architecture support. Backend compatibility, quantized files and runtime behavior still need validation in your environment.

Source: #4

3. How do I fine-tune with LLaMA-Factory?

Spark-X2.5 support is not merged upstream yet, so use the official fork:

Guide: examples/torch_spark2_5/Spark2_5_Finetune.md

Source: #7

4. 1M-token requests succeed, but needle-in-a-haystack accuracy is low

In a maintainer retest of Spark-X2.5-1.7B with 200 EN/ZH probes, disabling thinking substantially improved retrieval accuracy:

Prompt tokens	thinking=true, max_tokens=2048	thinking=false, max_tokens=2048
32,000	59.5%	98.0%
320,000	68.0%	96.5%
1,000,000	52.5%	82.0%

For simple retrieval/extraction tasks like needle-in-a-haystack:

Turn thinking off — the task needs no extended reasoning, and with thinking on a significant share of answers get truncated by max_tokens and score as misses;
If you do need thinking, raise max_tokens so the reasoning does not crowd out the answer.

At 1M context, the retest with thinking=true and max_tokens=32768 scored 60.5%. These results apply to that test setup, not all long-context tasks; the model, attention configuration and inference environment also matter.

Source: maintainer retest in #2 and full discussion.

5. Can it run on RK3588, and is NPU support confirmed?

A community developer reported running it on RK3588:

Hardware: RK3588 (4×Cortex-A76 + 4×Cortex-A55, 6 TOPS NPU), 8GB RAM
Model: Spark-X2.5-4B, Q8 quantization
Throughput: roughly 5.9 tokens/s

This is a community report relayed by a maintainer. The inference framework, deployment steps and NPU adaptation details remain unconfirmed; a reproducible NPU deployment procedure is not yet available.

Source: #8

6. What sampling parameters are recommended?

temperature=1.0, top_p=0.95, top_k=-1. All benchmark numbers in the README were produced in thinking mode.

Source: README evaluation notes.

7. Where can I download the models from China?

Besides Hugging Face, pick the mirror that matches your environment:

Channel	Best for
ModelScope / AtomGit	Users in China
Modelers	Ascend chips
SCNet	Hygon chips
Ollama / LM Studio	Running locally

Source: README download links.

8. What throughput and memory should I expect?

Community results for Spark-X2.5-1.7B BF16 on an H200 (batch=1, TP=1), for reference:

At 1M context, end-to-end throughput (including prefill) was 30.20 tokens/s for SGLang and 11.84 tokens/s for Transformers.
The corresponding memory readings were 63.34 GiB and 124.51 GiB. SGLang was measured with nvidia-smi, while Transformers used torch.cuda.max_memory_allocated; the metrics and memory-pool preallocation differ, so these are not strictly comparable memory-saving ratios.
In the same test, the 4B model ran out of memory at 1M context.

Community-measured, not an official performance claim. Source: #5

9. How do I run it directly with Ollama, LM Studio, Unsloth or a phone app?

These tools now support Spark-X2.5 natively, so you no longer need to build a fork or replace the runtime:

Ollama: update to v0.34.1 or later (native support since 2026-09-15), then run:
```
ollama run SparkLLM/Spark-X2.5-1.7B
# or
ollama run SparkLLM/Spark-X2.5-4B
```
Both models in the Ollama library currently have a single latest tag with unquantized weights (about 8.2 GB for 4B). If VRAM or RAM is limited, have Ollama pull an official GGUF quantization from Hugging Face instead, for example ollama run hf.co/XHToken/Spark-X2.5-4B-GGUF:Q4_K_M.
LM Studio: 0.4.0 or later. Search for "Spark-X2.5", download it and start chatting; from the command line, use lms chat spark-x2.5.
Unsloth Studio: the latest version supports native Spark-X2.5 GGUF inference.
PocketPal AI: the iOS (App Store) and Android (Google Play) apps support Spark-X2.5 natively for on-device use.

Source: README release log and local-run instructions.

10. VRAM or RAM is limited. Which version should I use? Is the file size the memory I need?

Official GGUF file sizes:

Model	Q4_K_M	Q8_0	Unquantized (BF16)
Spark-X2.5-4B	2.6 GB	4.4 GB	8.2 GB
Spark-X2.5-1.7B	1.1 GB	1.8 GB	3.4 GB

Download one quantized file for your runtime; you do not need every variant.
File size is not the memory you need. The runtime also allocates memory for the KV cache and other context state, and this grows with context length. On small GPUs, limit the context length first (for example -c in llama.cpp or num_ctx in Ollama).
With around 4 GB of VRAM, start with 4B Q4_K_M or 1.7B Q8_0. If the model does not fit, the runtime may fall back to CPU inference, which is much slower.
For vLLM or SGLang, FP8 and INT8 weights are also available on Hugging Face.

Source: file lists of Spark-X2.5-4B-GGUF and Spark-X2.5-1.7B-GGUF; community support groups.

11. How much context can I run with 16 GB of VRAM?

Community experience: a quantized 4B model with a Q8-quantized KV cache reaches about 500K tokens of context in 16 GB of VRAM. The actual limit depends on the inference framework, quantization and concurrency, so measure it in your own environment. There is no long-context data yet for phones and other low-memory edge devices; contributions are welcome.

Community report, not an official performance claim. Source: community support groups.

12. A tool-calling agent such as OpenClaw says the model does not support tools

A community user connecting through Ollama (2026-09-08) found that the model listed only the completion capability and not tools, so it could be used for plain chat but not to drive a tool-calling agent. Their server was also CPU-only, at about 80 seconds per response, which is too slow for multi-step agent loops.

For tool calling, use an OpenAI-compatible server with the tool-call parser configured:

SGLang: the README launch example already includes --tool-call-parser spark25;
vLLM: install the Spark plugin and start with --enable-auto-tool-choice --tool-call-parser spark25 (the parser name is spark25, not spark2_5).

Update (2026-10): on Ollama 0.35.0, a model built from the official Q4_K_M GGUF (a Modelfile containing only FROM Spark-X2.5-1.7B-Q4_K_M.gguf) reports tools, thinking and completion in the capabilities from /api/show, and /api/chat with a tool returns proper tool_calls. In other words, tool calling itself works on recent Ollama. However, the SparkLLM/Spark-X2.5-* model pages on ollama.com do not show a tools / thinking badge yet, so the models are missing from the tools filter; this is tracked in #13.

Source: community support groups; #13; README SGLang example; Spark-plugin README.

13. In everyday chat, thinking mode sometimes loops, questioning itself repeatedly

Thinking is on by default. Community users report that in everyday Q&A it occasionally keeps questioning itself in a loop, and that answers are more stable with thinking turned off. For requests that do not need extended reasoning, disable it per request:

"chat_template_kwargs": {"enable_thinking": false}

This matches the advice in entry 4 to turn thinking off for simple extraction tasks. Another user reported that with Q4_K_M, a 128K context and thinking enabled, the model got stuck in "semantic loops" over small decisions in a coding task and wasted more than 10K tokens (Spark-X2.5-4B Discussion #26); in such cases, compare against Q8_0 or thinking turned off. If you can reproduce the loop reliably, please open an issue with a minimal prompt, the model revision, your inference framework and the raw output.

Source: README note on disabling thinking; community support groups.

如果问题尚未覆盖，请提交新 issue，附上硬件、推理框架及版本、模型及量化方式、复现步骤和完整报错。

If your question is not covered, please open a new issue with your hardware, inference framework and version, model variant and quantization, reproduction steps and full error output.

Navigation Menu

Repository navigation

FAQ

Spark-X2.5 FAQ · 常见问题

中文

1. vLLM 启动失败，提示不认识 `Spark2_5ForCausalLM`

2. llama.cpp 报 `unknown model architecture: 'spark2_5'`

3. 怎么用 LLaMA-Factory 微调？

4. 1M 上下文能跑通，但"大海捞针"检索准确率很低？

5. 能在 RK3588 上运行吗？NPU 支持确认了吗？

6. 推荐的采样参数是什么？

7. 国内网络怎么下载模型？

8. 推理速度和显存大概什么水平？

9. 怎么用 Ollama、LM Studio、Unsloth 或手机 App 直接运行？

10. 显存 / 内存有限，该选哪个版本？模型文件大小就是需要的显存吗？

11. 16 GB 显存大约能跑多长的上下文？

12. 接入 OpenClaw 等需要工具调用的 Agent 时，提示模型不支持 tools？

13. 日常对话时 thinking 会反复自问、陷入循环？

English

1. vLLM fails to start — `Spark2_5ForCausalLM` not recognized

2. llama.cpp reports `unknown model architecture: 'spark2_5'`

3. How do I fine-tune with LLaMA-Factory?

4. 1M-token requests succeed, but needle-in-a-haystack accuracy is low

5. Can it run on RK3588, and is NPU support confirmed?

6. What sampling parameters are recommended?

7. Where can I download the models from China?

8. What throughput and memory should I expect?

9. How do I run it directly with Ollama, LM Studio, Unsloth or a phone app?

10. VRAM or RAM is limited. Which version should I use? Is the file size the memory I need?

11. How much context can I run with 16 GB of VRAM?

12. A tool-calling agent such as OpenClaw says the model does not support tools

13. In everyday chat, thinking mode sometimes loops, questioning itself repeatedly

Uh oh!

Clone this wiki locally

FAQ

Spark-X2.5 FAQ · 常见问题

中文

1. vLLM 启动失败，提示不认识 Spark2_5ForCausalLM

2. llama.cpp 报 unknown model architecture: 'spark2_5'

3. 怎么用 LLaMA-Factory 微调？

4. 1M 上下文能跑通，但"大海捞针"检索准确率很低？

5. 能在 RK3588 上运行吗？NPU 支持确认了吗？

6. 推荐的采样参数是什么？

7. 国内网络怎么下载模型？

8. 推理速度和显存大概什么水平？

9. 怎么用 Ollama、LM Studio、Unsloth 或手机 App 直接运行？

10. 显存 / 内存有限，该选哪个版本？模型文件大小就是需要的显存吗？

11. 16 GB 显存大约能跑多长的上下文？

12. 接入 OpenClaw 等需要工具调用的 Agent 时，提示模型不支持 tools？

13. 日常对话时 thinking 会反复自问、陷入循环？

English

1. vLLM fails to start — Spark2_5ForCausalLM not recognized

2. llama.cpp reports unknown model architecture: 'spark2_5'

3. How do I fine-tune with LLaMA-Factory?

4. 1M-token requests succeed, but needle-in-a-haystack accuracy is low

5. Can it run on RK3588, and is NPU support confirmed?

6. What sampling parameters are recommended?

7. Where can I download the models from China?

8. What throughput and memory should I expect?

9. How do I run it directly with Ollama, LM Studio, Unsloth or a phone app?

10. VRAM or RAM is limited. Which version should I use? Is the file size the memory I need?

11. How much context can I run with 16 GB of VRAM?

12. A tool-calling agent such as OpenClaw says the model does not support tools

13. In everyday chat, thinking mode sometimes loops, questioning itself repeatedly

Uh oh!

Clone this wiki locally

1. vLLM 启动失败，提示不认识 `Spark2_5ForCausalLM`

2. llama.cpp 报 `unknown model architecture: 'spark2_5'`

1. vLLM fails to start — `Spark2_5ForCausalLM` not recognized

2. llama.cpp reports `unknown model architecture: 'spark2_5'`