Skip to content

Release v0.3.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 17 Jul 10:48
v0.3.0
13aa476
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Neat Apps v0.3.0

Neat Apps v0.3.0 expands what users can build with live video on Modalix. The release adds high-density 16-, 24-, and 48-stream object detection, RTSP MJPEG and HTTP MJPEG inputs, more efficient encoded video delivery, improved Insight synchronization, and clearer GenAI application workflows.

Highlights

16-, 24-, and 48-Stream Object Detection

  • Adds a high-density YOLO26 object-detection application in C++ and Python.
  • Includes ready-to-use profiles for:
    • 16 × 720p streams at 25 FPS
    • 24 × 720p streams at 20 FPS
    • 48 × 720p streams at 10 FPS
  • Runs one shared detector across all streams instead of creating a separate detector for every input.
  • Publishes encoded video and detection metadata to separate Insight channels for every stream.
  • Detects inactive or stalled streams and reports metadata-delivery failures.
  • Lets users control decoder and inference pressure through inference.max_inflight_per_stream and inference.max_inflight_total.

See the High-Density Multi-Stream Object Detector.

RTSP H.264, RTSP MJPEG, and HTTP MJPEG Inputs

  • The single-stream object detector and instance segmenter now support:
    • RTSP H.264
    • RTSP MJPEG
    • HTTP and HTTPS MJPEG
  • Users select the source type, codec, URL, and frame rate through configuration instead of changing application code.
  • The applications can probe source resolution and frame rate when the source exposes them.
  • source.fps can be provided explicitly for cameras or streams that do not expose a usable frame rate.
  • Existing H.264 RTSP configurations using source.rtsp_url remain supported.

See the:

More Efficient RTSP and Insight Delivery

  • Updated detector and tracker applications reuse the original encoded H.264 stream for Insight.
  • Each RTSP source is opened once and shared between the Insight video path and the decode-and-inference path.
  • This removes the second RTSP connection, decoded-frame CPU copies, and board-side re-encoding from the updated applications.
  • Insight video delivery keeps the latest frame so a slow viewer or congested network does not block decoding and inference.
  • Encoded stream capabilities are preserved automatically instead of being overridden manually.
  • Optional debug-frame saving remains outside the main runtime path when disabled.

Improved Video and Metadata Synchronization

  • Applications preserve source timestamps and frame IDs through detection, segmentation, tracking, and GenAI processing.
  • Metadata uses Sample.pts_ns instead of unrelated wall-clock or placeholder timestamps.
  • High-density applications include the source RTP timestamp so Insight can associate detections with the correct video frame.
  • Existing VideoSender and MetadataSender APIs remain unchanged.

Improved GenAI Applications

  • Detection-to-VLM is separated into a detector application and a local GenAI server.
  • Users can select the VLM and change the system and user prompts through configuration.
  • The detector can run with GenAI disabled while continuing to publish video and detections to Insight.
  • GenAI requests run through a bounded background worker so slow VLM responses do not stop the live detection pipeline.
  • Multimodal Assistant setup validates the model directory before installing dependencies or downloading models.
  • Model storage can use LLIMA_MODELS_PATH, the default NVMe location, or another writable directory.

See the Detection-to-VLM Assistant.

Cleaner Application Package

  • The Apps package contains all 13 application examples.
  • Internal tests, test executables, test models, and test-only configuration are excluded from customer installations.
  • Applications are packaged independently of which internal tests are enabled.
  • The package records and installs its matching Neat Core dependency.

APIs Demonstrated by the Applications

The v0.3.0 examples show how to use the following Neat Core APIs:

  • RtspDecodedInput for decoded RTSP H.264 and MJPEG sources.
  • HttpMjpegDecodedInput for HTTP and HTTPS MJPEG sources.
  • RtspEncodedInput for encoded RTSP H.264.
  • SimaDecode for hardware video decoding.
  • VideoSender for encoded or raw video delivery to Insight.
  • MetadataSender for detection, segmentation, and tracking metadata.
  • Graph::connect() and Graph::build() for connected live-video applications.
  • GraphLinkOptions and RealtimeLatestByStream for bounded multistream processing.
  • source_fps for declaring the source frame rate.
  • memory_policy for public memory-placement configuration.

Implementation examples:

Installation

sima-cli neat install apps@v0.3.0

Compatibility

Component Version
SiMa.ai Platform 2.1.2
Neat Library >= 0.3.0
Neat Insight >= 0.0.6

Migration Notes

  • New codec-aware configurations should use source.type, source.codec, source.url, and source.fps.
  • Existing H.264 RTSP configurations using source.rtsp_url remain supported.
  • Applications now use the public memory_policy API instead of allocator-specific options.
  • Use matching Core, Internals, Apps, and Insight releases.
  • High-density profiles require fixed-rate 1280×720 H.264 RTSP sources, no B-frames, and a short, regular IDR interval.
  • The 48-stream profile uses the throughput/low-latency decoder configuration and requires no-B-frame H.264 sources.
  • Video and metadata timestamp matching requires the companion Insight synchronization support.

Key Included Pull Requests

Full Changelog: v0.2.2...v0.3.0

Contributors

Thank you to everyone who contributed to this release:

Assets 2
Loading