NEAT Core 0.6.0

Neat Library v0.6.0

Neat Library v0.6.0 moves Neat to Platform 3.0.0 and its DMA-BUF runtime. It adds hardware video encoding, USB camera input, LiDAR and BEV perception nodes, oriented box decoding for YOLOv8, YOLO11 and YOLO26, detailed model benchmarking, board telemetry, PCIe GenAI APIs, and offline installers.

Important: v0.6.0 requires Platform 3.0.0 (Palette SDK 3.0.0.0, DevKit software 3.0.0). Native applications and plugins must be rebuilt against the v0.6.0 headers and libsima_neat.so.6. Python applications should reinstall the matching PyNeat package. Applications that pass outputs from one model graph into another graph's EV74 stage must now call .cvu() on them first; see Breaking changes and migration.

Highlights

Hardware video encoding and sending

• nodes::SimaEncode encodes raw frames to H.264, H.265, or MJPEG, configured through SimaEncodeOptions (#936, #943).
• VideoSender sends all three codecs over RTP/UDP: VideoSenderOptions::FromRaw() encodes raw frames and VideoSenderOptions::Passthrough() forwards already-encoded video (#937).
• An RTSP source can feed a decoder and a passthrough VideoSender in one pipeline without a realtime link (#986).
• PyNeat exposes SimaEncodeOptions, nodes.sima_encode, VideoSenderOptions.from_raw, and VideoSenderOptions.passthrough.

USB cameras

• nodes::UsbCameraInput opens a USB (UVC) MJPEG camera by a stable USB ID from usb_camera_ids(), and usb_camera_info() reports how its frames decode (#1174).
• 4:2:2 MJPEG webcams decode to NV16, and models accept NV16 input directly through PreprocessColorFormat::NV16 (#1161).
• Unsupported camera resolutions or pixel formats fail with gstreamer.camera_configuration_unsupported, which names the camera and the requested format (#974).

LiDAR and BEV perception

SimaVoxelize, SimaPillarScatter, SimaBEVPoolPlan, and SimaBEVPool provide the point-cloud and bird's-eye-view stages for BEVFusion and PointPillars pipelines, on A65 or EV74 (#1212). SimaBEVPoolPlan runs on A65 only.

Oriented bounding boxes

• BoxDecodeType::YoloV26Obb decodes native YOLO26 OBB models (#1142).
• BoxDecodeType::YoloV8Obb decodes YOLOv8 and YOLO11 OBB models with rotated NMS (#1173).
• PyNeat reads both with decode_obb().
• YOLOX segmentation and pose decoding from v0.5.0 is available again on Platform 3.0.0.

Model benchmarking and execution plans

• Model::benchmark() reports p50, p95, and p99 latency, per-core CPU use, and board telemetry (#1080, #1073).
• Each measured pass runs for at least BenchmarkOptions::min_duration_ms, 10 seconds by default; set it to 0 for count-only timing (#1086).
• BenchmarkOptions::inputs benchmarks with your own tensors instead of synthetic input (#1159).
• Advanced: include_mla_layer_stats and include_mla_stage_stats add MLA layer and stage latencies. They need mla-rt and an otherwise idle DevKit (#1082, #1117).
• BenchmarkReport::save() writes the report as YAML, or JSON for a .json path (#1082).
• Model::execution_plan() shows, without running the model, where each step runs (A65, EV74, or MLA), its kernel and settings, and each tensor's shape, type, and quantization (#1070).

Board telemetry

TelemetryMonitor records board telemetry from Sentinel between start() and stop(), including CPU, EV74 memory, and estimated power (#1103, #1131, #1153). Benchmark power figures also come from Sentinel. Power values are estimates (#1237).

GenAI

• PCIe hosts can run LLM, VLM, and Whisper speech recognition on a Modalix card through pcie::genai::GenAIModel in C++ and pyneatpcie.genai.GenAIModel in Python (#1019).
• gpt-oss mixture-of-experts models run on the DevKit, and GenerationRequest::reasoning_effort selects low, medium, or high reasoning (#924).
• Prompts longer than the model context return HTTP 400 context_length_exceeded instead of an empty success, and token-limited responses report finish_reason: length (#1252).

Installation

• Offline installer bundles install Core on the SDK or DevKit without downloading packages during installation: sima-cli neat install core/offline@v0.6.0 --download-only, then ./install.sh (#1155).
• PCIe hosts have an offline installer with preflight checks and automatic SSH setup (#1135, #1207).
• Existing Python environments are kept and checked; --recreate-venv rebuilds one (#1175).
• neat status reports PCIe host and attached card software (#1164), and shows the code-server password and correct URLs in the SDK (#1107).

Important fixes

• Native EV74 firmware stays active after a reboot. Previously the first Neat request after a reboot hung for 5 seconds and later runs failed until the firmware was reactivated (#1203).
• Applications that keep an output while pulling the next one no longer stall; YOLOv8n throughput rose from 58 to about 459 FPS in that pattern (#1182).
• Zero-copy buffer stalls log a named warning instead of stopping the stream silently (#1229, #1253).
• OutputOptions.drop=true is honored, so live graphs no longer time out when outputs back up (#1209).
• Models whose tensor order differs from the executable port order prepare correctly and return outputs in the right order (#955).
• Batched models and YOLO26n batch-four postprocessing work again (#1200, #1221).
• Model::build() without a seed input starts correctly, and seeded models accept larger frames within input_max_* (#1232, #1024).
• benchmark() works with dynamic-image models (#1109).
• Native INT32 ArgMax outputs report the correct type, and stateful models with shared cache tensors no longer hit slot limits (#1136).
• Child graphs keep their parent's graph_options, which fixes routed RF-DETR and BoxDecode metadata (#1128).
• Closed graphs and Python node wrappers no longer leak (#1140).
• Containers sharing a model extraction folder no longer overwrite each other's files (#1199).
• Model Compiler 3.0 models with a single unnamed I/O section load (#1053).
• PCIe source-driven throughput is no longer capped near 360 FPS for YOLOv8n (#1130), and PCIe co-processing reads non-dense card outputs at the correct offsets (#993).
• MetoakDepth works on Platform 3.0.0 (#1187).
• PyNeat installs on boards without python3-venv (#1061), and works with numpy 2 (#1036).
• FORCE_INSTALL and -f bypass the installer's platform check and reach EV74 firmware activation (#1089, #1091).
• A failed Sentinel installation no longer stops a Core installation (#1096).
• sima-cli neat install core installs from a fresh bundle copy, which fixes the "manifest differs from complete DEB set" failure (#1239).

Breaking changes and migration

Move to Platform 3.0.0

Core runs on Platform 3.0.0 with one DMA-BUF execution plan per model (#877). Internals is no longer used. Core now runs through Scheduler, which coordinates Runner, Kernels, and Codecs; GenAI continues to use LLiMa (#1095). Install Core with its matching runtime set, as described under Compatibility.

Call .cvu() before passing outputs to another graph's EV74 stage

To reduce CMA use, outputs of steps that run only on the MLA or A65 are now allocated in DMS memory instead of CMA. EV74 can read only CMA, so passing such an output directly into a separately built graph whose first step runs on EV74 now fails with EV74 requires admitted CMA or contiguous camera memory. Core cannot choose the right memory across separately built graphs; it places memory automatically only within one graph.

Applications that chain models, such as an encoder feeding a decoder, must call Tensor::cvu() (PyNeat tensor.cvu()) on the output first. It returns a tensor already in EV74 memory unchanged and copies any other tensor. ZeroCopy keeps the producer's memory and does not make it readable by EV74. .cvu() copies only the selected tensor into independent memory, which fixes tensor span exceeds runtime segment errors in these handoffs (#1245).

Rebuild native applications

The C++ ABI moves from 5 to 6 (libsima_neat.so.6). Tensor, GenAI, graph-link, benchmark, TessSpec, and SimaArgMaxOptions layouts changed. The Cast, CastTess, Quant, QuantTess, Tess, and Preproc option layouts now match Core's internal layouts (#1250). Rebuild C++ applications, plugins, and Python extensions against the v0.6.0 headers. Do not link ABI 5 binaries to the ABI 6 library.

These headers were removed: gst/ProcessMlaRuntimeConfig.h, gst/SimaPreparedRuntimeAbi.h, and neat/PreparedRuntimeBridge.h. gst/SimaaiGstErrors.h is now provided by Scheduler at the same include path.

Copy Owned outputs to the device before reuse

OutputMemory::Owned outputs, and Auto outputs that take the copy path, are now independent system-memory copies on the CPU instead of CVU buffers (#1236, #1243). Call Tensor::to() before pushing such an output directly into a device input such as Preproc, Quant, Tess, or a model. Tensor::cpu() no longer copies; use clone() for a copy.

RTSP drops late packets by default

drop_on_latency now defaults to true in nodes::RTSPInput, RtspEncodedInputOptions, RtspDecodedInputOptions, and pyneat.nodes.rtsp_input (#1234). Live pipelines keep moving, but H.264 and H.265 can show artifacts until the next keyframe. Set drop_on_latency to false for recorded or lossless streams.

Benchmark timing and power

Benchmark passes now last at least 10 seconds by default (#1086). Set min_duration_ms = 0 to restore count-only timing. PowerMonitor, enable_board_power(), the I2C power helpers, the boolean benchmark() overloads, and the plugin-latency options are deprecated; power comes from Sentinel.

BoxDecode options

BoxDecode requires a positive top-K, and retired non-default options raise an error instead of silently using legacy execution (#1095).

Deprecated video output groups

H264RtpUdpFromRaw and H264RtpUdpFromEncoded are deprecated. Use VideoSender with VideoSenderOptions::FromRaw() or VideoSenderOptions::Passthrough().

Local source builds

build.sh installs the Scheduler package set under deps/scheduler and only LLiMa into the SDK sysroot (#1126, #1166). Pass local builds with --scheduler PATH and --llima PATH, or --no-llima. The NEAT_KERNELS_*, NEAT_RUNNER_*, NEAT_SCHEDULER_*, and NEAT_CODECS_* variables now stop the build; to test a Kernels, Runner, or Codecs change, build Scheduler against it and pass that folder. See the build and dependency guide.

Known issues

• Decoder buffer-pool starvation can still stop a stream without an error (#1267).
• Installation can fail Scheduler admission when the install folder contains older .deb files (#1240).
• SimaArgMax applies a non-zero input view offset twice (#1211).
• Two MIPI camera pipelines run at about 23–24 FPS each (#1113).
• Model::execution_plan() omits BoxDecode settings (#1256).
• There is no supported uninstall or rollback for Core (#1242).

Install

Install or update Neat Library with:

sima-cli neat install core@v0.6.0

To download only the PyNeat wheel:

sima-cli neat install core@v0.6.0 -t pyneat

To prepare an offline installation:

sima-cli neat install core/offline@v0.6.0 --download-only

Compatibility

• Core package version: 0.6.0.
• Compatible Platform release: 3.0.0 (Palette SDK 3.0.0.0, DevKit software 3.0.0).
• Public C++ ABI: 6.
• Runtime: Scheduler v0.6.0, with its matching Runner, Kernels, and Codecs, and LLiMa v0.6.0.

Contributors

@manuel-roldan, @Onat-Inak, @shanguangzai, @florianvoss-commit, @nicholasxu-wq, @dotimothy, @naik24, @karimguirguis-sima, @Yukkta-Seelam

Full Changelog: v0.5.0...v0.6.0