Skip to content

SimdYolo 0.11.4

Choose a tag to compare

@github-actions github-actions released this 11 Nov 13:19
· 2 commits to main since this release

SimdYolo 0.11.4

本版集中优化 CPU / Vulkan 推理中的线程等待、临时数据、池化和 Winograd 卷积,继续使用 FP32。SimdYolo 与 SimdYolo.Training 同步发布 0.11.4,目标框架为 .NET 10,现有公开 API 保持兼容。

更新内容

  • CPU 调度:作业完成时只等待已实际参与的工作线程,减少因线程迟到而产生的空等;保留正在执行的回调、局部状态清理及异常传播的完成屏障。异常队列按需创建。
  • CPU SIMD 与内存访问:最大池化为每个工作线程复用单个平面工作区,保持水平/垂直处理的缓存局部性,并展开常见五点向量归约;批次卷积直接写入最终输出,减少中间缓冲区和复制。
  • 检测头:CPU 和 Vulkan 的受支持多尺度检测头直接写入合并输出,减少中间张量及拼接;CPU 按尺度及时解码。保留其他后端及跟踪路径的回退。
  • Vulkan 池化与提交:增加 5×5 池化共享内存分块内核,复用相邻窗口的数据,并减少提交路径的临时分配。
  • Vulkan Winograd:缓存连续四通道权重布局,以 vec4 读取配合共享内存复用;为大、小空间分别提供矩阵乘内核,保持递增输入通道的 FP32 FMA 顺序。非适用形状使用原路径,也可用 SIMDYOLO_WINOGRAD_PACKED=0 关闭新路径。
  • 工具与文档:中英文 README 更新安装版本和性能说明;基准工具增加可选 Windows HighQoS 测试模式;发布流水线增加新池化和 Winograd 内核的 Mesa Vulkan 回归测试。
  • 训练包:同步升级推理依赖并验证训练、保存和恢复流程。本版没有重新测量训练速度,不把推理收益当作训练收益。

性能实测

Windows 11、Core i7-12700、RTX 3060 12 GB(驱动 610.88),FP32、640×640、batch 1、12 CPU 线程。使用固定真实图片输入,串行 ABBA 独立进程对照,统计各窗口稳态中位数的中位数。PredictRaw 计时包含 GPU 输入上传和全部输出回读,不含图片预处理、NMS 或最终掩码生成。

下面是发布前最后一轮优化的增量对照。基线已包含本版前期的池化、检测头和批次写入改动,不是 NuGet 0.11.3 原包,因此这些百分比不是 0.11.3 → 0.11.4 的完整版本差值。 两个测量程序集当时仍标记 0.11.3,准确身份以报告中的 SHA256为准。

显式 HighQoS:两版本使用相同测试进程设置

模型 / Model 后端 / Backend 基线 / Baseline 优化后 / Optimized 耗时变化 / Latency change
YOLOv8n PT CPU 23.036 ms 23.164 ms +0.56%
YOLO11n PT CPU 21.928 ms 21.844 ms -0.38%
YOLO26n-seg PT CPU 30.705 ms 30.544 ms -0.53%
YOLOv8n PT Vulkan 4.204 ms 3.768 ms -10.38%
YOLO11n PT Vulkan 4.270 ms 4.076 ms -4.55%
YOLO26n-seg PT Vulkan 6.157 ms 5.846 ms -5.05%
YOLOv8n ONNX Vulkan 4.126 ms 3.684 ms -10.70%
YOLOv8n ONNX,初测 / initial CPU 21.076 ms 22.225 ms +5.45%
YOLOv8n ONNX,加长复测 / longer repeat CPU 21.251 ms 20.919 ms -1.56%

Vulkan 在这些配置中降低 4.55%~10.70% 的稳态耗时。CPU PT 中位耗时基本持平,每帧托管分配减少 3.07%~4.04%;ONNX CPU 两轮方向不同,未确认稳定提速。CPU 的 YOLOv8n / YOLO26n-seg P95 分别增加 3.43% / 5.90%,不宣称尾延迟全面改善。

继承本机后台启动环境的 CPU 调度策略

两版本均未主动请求 HighQoS:YOLOv8n PT 为 125.406 → 67.954 ms(-45.81%),YOLO26n-seg PT 为 182.732 → 104.455 ms(-42.84%)。此组说明减少等待迟到线程在该启动条件下有收益;绝对耗时及改善比例不能推广到前台推理,不能与上表混用。主库不改变应用的调度策略、系统电源计划或 CPU 亲和性。

开销与测量范围

  • 新 GPU 权重布局需要额外主机缓存。本机 GPU 测试中的整进程工作集增加约 48~75 MiB;这不是显存测量,也不能将全部增量归因于权重缓存。CPU 托管分配减少不代表驻留内存减少。
  • 共 72 个独立进程窗口、23,642 次稳态推理。主测预热 80 帧、至少测量 3 秒 / 40 帧;ONNX CPU 复测为 100 帧、4 秒。PT 主测每版本每配置四个窗口,ONNX 初测和继承策略组各两个。
  • 本轮不宣称启动耗时、其他显卡、Metal、全部模型或训练性能都有同样收益。保留 FP32,没有通过降精度换取提速。
  • 完整报告与复现命令 · 所有窗口、逐帧样本和验证记录。原始张量和冻结 DLL 保存在实验机器,JSON 包含其路径及哈希。

正确性与发布验证

发布前 Windows 本机通过 101 项 CPU、11 项训练和 20 项相关 Vulkan 测试,覆盖线程休眠唤醒、迟到参与者、张量尾部、批次、输入变化后的命令重放及独立标量卷积参考。184 项首帧 / 稳态输出回归核对通过:GPU 输出逐字节一致;CPU 最大绝对差约 0.002350,符合既有逐元素容限 0.003 + 0.0003*abs(reference)。这不是新一轮数据集 mAP 评估。

发布流水线重新构建、运行 CPU / 训练及选定 Mesa Vulkan 测试,并从打包产物消费两个 NuGet 包。NuGet 上传后回读包内容(排除 NuGet 添加的签名),再发布 GitHub Release 包附件及 SHA256SUMS.txt。

已知限制:显式启用通用 ONNX 末轴 GPU TopK 的同分索引用例在基线也失败;通用 ONNX TopK 默认走 CPU。本轮相关 Vulkan 测试通过不等同于该已知用例修复。

安装 / Install

dotnet add package SimdYolo --version 0.11.4
dotnet add package SimdYolo.Training --version 0.11.4

训练包会传递引入同版推理包。SimdYolo on NuGet · SimdYolo.Training on NuGet。本次附件为两个 NuGet 包和 SHA-256 清单;模型通过仓库 Git LFS 获取。

English release notes

SimdYolo 0.11.4 improves CPU and Vulkan inference while retaining FP32 and the existing public APIs. Both NuGet packages target .NET 10 and use version 0.11.4.

  • The CPU executor waits only for workers that actually joined a job, while still draining active callbacks and cleanup before returning. Exception queues are allocated only when needed.
  • CPU max pooling reuses one workspace plane per worker and unrolls common five-sample SIMD reductions. Batched convolutions write directly into their final output.
  • Supported CPU and Vulkan multi-scale detection heads write into joined outputs, reducing intermediate tensors and concatenation. Other backend and tracing fallbacks remain available.
  • Vulkan adds a shared-memory 5×5 max-pooling kernel and packed Winograd weights with vectorized reads, shared-memory reuse and separate kernels for small and large spatial sizes. FP32 accumulation order is retained. Set SIMDYOLO_WINOGRAD_PACKED=0 to use the previous Winograd middle path.
  • The training package updates its inference dependency; training, checkpoint saving and resuming are validated. Training speed was not remeasured.

The table above measures the final optimization round, not the full difference from the published 0.11.3 package. Its frozen baseline already includes earlier pooling, detection-head and batch-write changes in this release. On the tested i7-12700 / RTX 3060 system, Vulkan steady inference latency decreased by 4.55–10.70%. CPU PT medians were essentially unchanged with explicit HighQoS; managed allocations fell by 3.07–4.04%. The initial ONNX CPU result regressed by 5.45%, while a longer repeat improved by 1.56%; no stable CPU ONNX speedup is claimed. CPU P95 increased for two PT cases.

Under this machine's inherited background launch environment, CPU latency fell by 42.84–45.81% for two models. This result is specific to that scheduling condition and must not be generalized to foreground inference. Only the optional benchmark worker requests HighQoS; the library does not change application scheduling policy or system power settings.

The packed GPU path stores an extra host-side weight layout. Measured whole-process working sets increased by approximately 48–75 MiB; this is not a VRAM measurement. Tests retain FP32 and include GPU transfers and complete raw output readback, but exclude image preprocessing, NMS and final mask generation. The reports preserve all 72 process windows, 23,642 steady inferences and both ONNX CPU runs. No universal startup, device, model, training or Metal performance claim is made.

Before release, 101 CPU, 11 training and 20 relevant Vulkan tests passed locally, along with 184 raw output comparisons. GPU outputs were bit-identical to the frozen baseline; CPU differences stayed within the existing elementwise tolerance. These are regression checks, not a new dataset accuracy evaluation. The existing opt-in generic last-axis GPU TopK tie-index limitation remains; generic ONNX TopK defaults to CPU.

GitHub Actions builds and tests the release, consumes both packaged libraries, publishes them through NuGet Trusted Publishing, verifies public package contents and then publishes the GitHub Release with both packages and their SHA-256 manifest.