【腾讯犀牛鸟2026】Tencent-Hunyuan/HunyuanOCR ncnn 移植与多平台部署

[HGinkgo]

HunyuanOCR-ncnn 发布说明

Tencent HunyuanOCR 的 ncnn C++17 推理运行时：图片输入、OCR 文本输出、动态 vision、KV cache 解码和跨平台构建验证

作者：HGinkgo
日期：2026 年 7 月 8 日
版本：v0.2
项目仓库：HGinkgo/HunyuanOCR-ncnn

---

项目概览

HunyuanOCR-ncnn 是一个面向 Tencent HunyuanOCR 的 ncnn C++ runtime。项目将 HunyuanOCR 推理链路拆成 ncnn 子模块，并用 C++17 串起图片解码、图像预处理、vision encoder、LLM KV cache 解码、lm_head 和 tokenizer decode。

当前版本的目标是先提供一个可以复现、可以编译、可以端到端运行的 fp32 CPU 版本，并用公开图片做严格 token/text 回归。

当前交付能力：

能力	状态
输入	PNG/JPEG 图片
输出	OCR / 文档解析文本
构建	CMake + C++17
推理后端	ncnn fp32 CPU
Vision	支持不同尺寸图片输入，保留固定尺寸回退路径
Text decoder	KV cache 贪心解码
Prompt	内置 spotting / document，支持自定义 --prompt
平台	Linux 本地验证，Windows CI 编译，Windows 实机带模型运行验证
公开回归	28 张图片 strict token/text 一致

---

Runtime 结构

运行时模型目录使用以下结构：

hunyuan_ocr_ncnn_model/
  model.json
  tokenizer/
  text_embed/
  text_decoder/
  lm_head/
  vision/
    vision.ncnn.param
    vision.ncnn.bin
    pos_embed.bin
    grid_<grid_h>x<grid_w>/

C++ 代码结构：

include/hunyuan_ocr/
  hunyuan_ocr.h
  image_preprocessor.h
  model_layout.h
  prompt_builder.h
  text_runtime.h
  tokenizer.h
  vision_runtime.h

src/
  hunyuan_ocr.cpp
  image_preprocessor.cpp
  main.cpp
  model_layout.cpp
  prompt_builder.cpp
  text_runtime.cpp
  tokenizer.cpp
  vision_runtime.cpp

第三方代码只保留 stb_image.h 负责 PNG/JPEG 解码，核心推理依赖 ncnn。

---

验证结果

当前公开回归集包含 28 张图片，来源包括 HunyuanOCR upstream assets、HunyuanOCR README 示例图和 OmniDocBench 少量公开样本。图片来源记录在仓库的 examples/IMAGE_SOURCES.md。

验证口径：

项目	配置
PyTorch reference	fp32
ncnn runtime	fp32 CPU
Image processor	min_pixels=262144, max_pixels=524288
Decode	greedy argmax, do_sample=False
Repetition penalty	1.03
Prompt modes	spotting, document, custom prompt

28 张公开样本回归结果：

summary: 28/28 passed

覆盖情况：

项目	数值
图片数	28
spotting 样本	10
document 样本	18
generated tokens 总数	11801
unique grids	19

覆盖的 grid：

16x92, 20x94, 24x84, 28x70, 32x60, 32x62, 34x56,
36x54, 36x56, 38x52, 40x40, 40x48, 44x44, 44x46,
46x42, 52x38, 54x36, 58x22, 58x34

回归检查项包括：

• prompt ids 一致；
• position ids 一致；
• generated token ids 一致；
• 最终 decode 文本一致。

Windows 侧也完成了两类验证：

验证	状态
GitHub Actions MSVC compile-only	通过
Windows 实机 packaged-model runtime	通过

---

示例

仓库内置 28 张公开示例图：

python tools/run_example.py --list

运行单张图：

python tools/run_example.py \
  --model ./hunyuan_ocr_ncnn_model \
  --case hf_demo

运行全部示例图：

python tools/run_examples.py \
  --model ./hunyuan_ocr_ncnn_model

严格回归：

python tools/run_regression.py \
  --package \
  --package-vision-backend dynamic

部分公开样本统计：

Case	Prompt	Grid	New tokens
hf_demo	spotting	38x52	411
chinese_doc	spotting	54x36	593
en_book	document	58x34	1024
formula	document	58x34	979
table	document	54x36	325
hunyuan_vis_parsing_table_2	document	46x42	1024
hunyuan_translation2	document	40x48	844
hunyuan_show_res_parsing_fig	document	44x46	1024

完整列表见仓库：

• examples/regression_cases.json
• examples/EXPECTED_OUTPUTS.md
• examples/IMAGE_SOURCES.md

---

示例效果

下面选了几类不同图片做展示。输出来自 ncnn runtime，展示时只做换行排版，不做内容截断；spotting 类保留坐标，便于核对检测结果。完整样本列表和来源见仓库中的 examples/ 目录。

字幕 / 双语文字

ncnn 输出：

对不起 我是诺曼·斯佩尔曼(298,852),(699,918)
Sorry, I'm Norm. Spellman.(340,926),(657,974)

中文文档页

ncnn 输出：

304(143,37),(185,50)
求医不求求已家庭医学全书(143,50),(307,62)
7.脾虚呕吐(126,106),(246,123)
【表现】(171,137),(242,154)
脾胃虚弱，神疲乏力。(171,162),(385,180)
【家庭复方调理法】(171,202),(353,219)
推拿法：每天做一次。(171,227),(385,245)
①补脾经300下（图4-1-5）。(171,252),(462,271)
②揉板门300下（图4-1-11）。(171,278),(473,297)
③捏脊5遍（图4-1-21）。(171,305),(430,323)
④按揉足三里穴150下（图4-7-1）。(171,331),(529,350)
气和嗝立消——小儿打嗝的复方调理法(124,440),(650,466)
打嗝就是膈肌痉挛。婴儿与成年人都有膈肌，但功能略有不同，婴(217,507),(817,526)
儿以腹式呼吸为主，膈肌还是呼吸肌的一部分。当婴儿吃奶过快或吸入(169,530),(819,548)
冷空气时，就会使膈肌突然收缩，引起迅速吸气并发出“喘”的声音。(169,552),(719,570)
这种情况一般都能自行缓解，但如果持续性发作或间歇性发作，就需要(169,574),(819,592)
调理了。(169,596),(242,613)
【原因】(169,656),(240,674)
胃寒、胃热、食滞、气郁和正气亏虚等造成膈肌痉挛。(169,682),(691,701)
（一）各种类型打嗝的通用复方调理法(124,733),(527,754)
推拿法：每天做一次。(169,767),(383,785)
①开天门300~500下（图4-1-1）。(169,792),(525,812)
②揉膻中60下（图4-8-1）。(169,818),(446,837)
③揉中脘300下（图4-8-1）。(169,844),(458,863)
④摩腹300~500下。(169,870),(379,889)

英文文档页

ncnn 输出：

Fig. 10.3 Solved sudoku puzzle.

## 10.4 Hybrid Optimization

Hybrid methods may be required to solve particularly difficult real-world optimization problems. Implementation of hybrid methods typically requires non-trivial scripting because hybrids generally require a custom optimization workflow process. To illustrate this point, we describe a hybrid optimization method to solve a parameter estimation problem arising in the context of a model for childhood infectious disease transmission.

This parameter estimation model is a difficult nonconvex, nonlinear program for which no efficient solution algorithm exists to find global optima. To facilitate solution, the problem is reformulated using a MIP under-estimator and an NLP over-estimator. Information is exchanged between the two formulations, and the process is iterated until the two solutions converge.

Example 10.A.14 provides a Pyomo script that implements the hybrid search. Note that some initialization code is omitted from this example for clarity. In this example, user-defined Python functions are used to organize and modularize the optimization workflow. For example, the optimization model in this application is constructed via a function, disease.mdl. Example 10.A.14 includes a sketch of this function.

Beyond the standard Pyomo model constructs, we observe the ability to dynamically augment Pyomo models with arbitrary data, e.g., the definition of the pts.LS and pts.LI attributes; these are not model components, but rather raw Python data types. Pyomo itself is unaware of of these attributes, but other components of a user-defined script can access and manipulate these attributes. Such a mechanism is invaluable when information is being propagated across disparate components of a

网页 / UI 截图

ncnn 输出：

Lab mouse portrait(142,15),(251,33)
huggingface.co/chat/conversation/6655bacb6dfed1a136...(156,76),(545,99)
R(644,78),(654,93)
Relaunch to update(844,76),(964,97)
Generate the portrait of a scientific mouse in its laboratory.(151,176),(557,199)
Called tool Image Generation(214,590),(408,612)
The mouse stands proudly in its laboratory, dressed in a crisp white lab coat and a pair of(180,645),(827,671)
round, thick-framed glasses. It holds a clipboard in one paw, ready to make notes on its(180,684),(812,710)
latest experiment. Its whiskers twitch slightly as it sniffs at a beaker on the desk, curious(180,722),(820,749)
and inquisitive. The laboratory is filled with all sorts of interesting equipment, from(180,761),(776,788)
Tools (6) 1g flasks to complicated-looking machines with buttons and dials. The n(162,803),(748,829)
Upload file(780,803),(852,826)
seems right at home in this environment, its tail curled neatly around a nearby stool(180,840),(784,863)
Ask anything(146,878),(236,901)
Model: CohereForAI/c4ai-command-r-plus(137,935),(380,954)
Generated content may be inaccurate or false.(402,935),(664,954)
Share this conversation(727,935),(860,952)

公式 / 论文页

ncnn 输出：

Let $ g,h $ be the inverse Fourier transforms of $ \chi_{U},\chi_{UK} $ (as given by the Plancherel theorem), and let $ f=|U|^{-1}gh $. Then $ f\in L^{1} $ and $ \widehat{f}=\widehat{g}\star\widehat{h} $ by Proposition (4.36); thus $ \widehat{f}(\xi)=|U|^{-1}\int_{U}\chi_{UK}(\eta^{-1}\xi)\,d\eta $ has the desired properties.

(4.51) Theorem. If $ N\subset\widehat{G} $ is closed, then
$ \nu(\iota(N))=N $.

Proof: If $ \xi\notin N $, take $ K=\{\xi\} $ and
$ W=\widehat{G}\setminus N $ in Lemma (4.50) to obtain
$ f\in\iota(N) $ such that $ \widehat{f}(\xi)\neq 0 $.

When G is compact, the other half of the correspondence is easily analyzed. First, a simple lemma that will also be useful elsewhere.

(4.52) Lemma. If $ f\in L^{1}(G) $ and $ \xi\in\widehat{G} $ ($ \subset L^{\infty}(G) $) then $ f\star\xi=\widehat{f}(\xi)\xi $.

Proof: For any $ x\in G $,

$$ f\star\xi(x)=\int f(y)\langle y^{-1}x,\xi\rangle\,dy=\langle x,\xi\rangle\int f(y)\langle\overline{y},\overline{\xi}\rangle\,dy=\widehat{f}(\xi)\langle x,\xi\rangle. $$

(4.53) Theorem. If G is compact, then $ \iota(\nu(\mathcal{I}))=\mathcal{I} $ for every closed ideal $ \mathcal{I}\subset L^{1}(G) $.

Proof: Since G is compact, we have $ \widehat{G}\subset L^{\infty}\subset L^{2}\subset L^{1} $. Suppose $ f\in\iota(\nu(\mathcal{I})) $. Then $ f\star\xi=f(\xi)\xi $ by Lemma (4.52), and either $ \widehat{f}(\xi)=0 $ or $ \xi\notin\nu(\mathcal{I}) $. In the first case, $ f\star\xi=0 $; in the second case, there exists $ g\in\mathcal{I} $ such that $ \widehat{g}(\xi)=1 $, so that $ \xi=g*g\in\mathcal{I} $ by Lemma (4.52) again. In either case we have $ f\star\xi\in\mathcal{I} $, and hence $ f*g\in\mathcal{I} $ for any $ g $ in the linear span of $ \widehat{G} $. The latter is dense in $ L^{2} $ by Corollary (4.26), so $ f*g\in\mathcal{I} $ for all $ g\in L^{2} $ since $ \mathcal{I} $ is closed. Finally, we can take $ g $ to be an approximate identity to conclude that $ f\in\mathcal{I} $.

When G is noncompact, the question of whether $ \iota(\nu(\mathcal{I}))=\mathcal{I} $ is much more delicate. We now exhibit a simple example to show that the answer can be negative.

(4.54) Theorem. Let $ G=R^{n} $ with $ n\geq 3 $, and let S be the unit sphere in $ R^{n} $. There is a closed ideal $ \mathcal{I} $ in $ L^{1}(R^{n}) $ such that $ \nu(\mathcal{I})=S $ but $ \mathcal{I}\neq\iota(S) $.

Proof: First we observe that if f and $ x_{1}f $ (= the function whose value at x is $ x_{1}f(x) $) are in $ L^{1}(R^{n}) $ then

$$ \begin{align*}(4.55)\qquad&=\int\frac{\partial e^{-2\pi i\xi\cdot x}}{\partial\xi_{1}}f(x)\,dx=\frac{\partial\widehat{f}}{\partial\xi_{1}}(\xi).\end{align*} $$

Hence $ \partial\widehat{f}/\partial\xi_{1} $ exists and is continuous.

Let I be the set of all $ f\in L^{1} $ such that $ x_{1}f\in L^{1} $ and $ \widehat{f}|S=(\partial\widehat{f}/\partial\xi_{1})|S=0 $, and let $ \mathcal{I} $ be the closure of I in $ L^{1} $. Since $ (L_{y}f)^{-}(\xi)= $

---

构建方式

使用已安装的 ncnn CMake package：

cmake -S . -B build -Dncnn_DIR=/path/to/ncnn/lib/cmake/ncnn
cmake --build build -j

或者直接指定本地 ncnn build tree：

cmake -S . -B build \
  -DHUNYUAN_OCR_USE_NCNN_PACKAGE=OFF \
  -DNCNN_INCLUDE_DIR=/path/to/ncnn/src \
  -DNCNN_BUILD_INCLUDE_DIR=/path/to/ncnn/build/src \
  -DNCNN_LIBRARY=/path/to/ncnn/build/src/libncnn.a
cmake --build build -j

打包 runtime 模型目录：

python tools/package_model.py \
  --workspace <workspace> \
  --output ./hunyuan_ocr_ncnn_model \
  --vision-backend dynamic \
  --copy \
  --force

---

当前边界

当前版本聚焦 fp32 correctness 和跨平台可运行性：

• 当前验证和导出实验使用 NVIDIA GeForce RTX 3090；max_pixels=524288 是在这台机器上选择的稳定验证配置，不是模型结构上限。
• 用户可以根据显存、内存和速度需求调整 HunyuanOCR processor 的 min_pixels / max_pixels，但需要重新生成 PyTorch reference、runtime 模型包和 regression fixtures。
• 当前主线是 ncnn fp32 CPU runtime。
• 低精度、Vulkan、int8 和性能优化还没有作为本版本 gate。
• 示例脚本用于公开图片运行；strict token/text 对齐通过 fixture regression 验证。

---

后续计划

下一步会继续补充：

• CPU benchmark：不同线程数、不同 grid、不同输出长度；
• 更多类别图片的可视化展示和长文本输出摘录；
• 更大规模公开样本回归；
• Vulkan fp16 / int8 量化评估。

---

参考链接

1. HunyuanOCR: https://github.com/Tencent-Hunyuan/HunyuanOCR
2. HunyuanOCR HuggingFace model: https://huggingface.co/tencent/HunyuanOCR
3. ncnn: https://github.com/Tencent/ncnn
4. pnnx: https://github.com/Tencent/ncnn/tree/master/tools/pnnx
5. OmniDocBench: https://github.com/opendatalab/OmniDocBench
6. 本项目仓库：HGinkgo/HunyuanOCR-ncnn