Skip to content

KataGo ONNX Runtime backend (OpenVINO EP) — v4: Multi-device CPU/GPU/NPU inference

Latest

Choose a tag to compare

@seniorfish seniorfish released this 02 Aug 15:57

Prebuilt KataGo.exe with ONNX Runtime 1.29.0 and OpenVINO 2026.2.1.

Run CPU, GPU, and NPU simultaneously in a single process. On Intel Core Ultra 285H, combining GPU+NPU achieves ~260 visits/s—a 79% performance boost over GPU alone (145 visits/s).

Performance (Intel Core Ultra 285H, model kata1-b18c384nbt)

Config Best visits/s Threads
GPU only 145.3 8
NPU only 80.8 8
GPU + NPU (4:4 batch) 259.9 10
GPU + NPU (6:2 batch) 205.9 8
GPU + NPU + CPU (4:4:1) 181.5 10

Detailed benchmarks: (lightvector#1222 (comment))

This release introduces new settings for multi-device setups, including per-thread device assignment (onnxOpenVINODeviceTypeThread), per-device tuning (onnxOpenVINODeviceConfig__), and thread batch limits (nnMaxBatchSizeThread).

Please refer to gtp_gpu_npu_example.cfg for a complete list and demonstration of these new configuration options.

Package Contents

  1. katago.exe (64-bit, ONNX backend, commit 815378d)
  2. OpenVINO 2026.2.1 and ONNX Runtime 1.29.0 DLLs
  3. Config files: gpu_optimized.cfg (single-GPU) and gtp_gpu_npu_example.cfg (multi-device) gtp_example.cfg (comprehensive)