Skip to content

Release v0.5.0

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 21 Aug 01:32
v0.5.0
96b5959
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Neat Apps v0.5.0

Neat Apps v0.5.0 adds eight applications for camera, vision, PCIe, and GenAI workflows. It also adds H.265 RTSP support, faster model loading through Neat Library 0.4.0, better video and metadata synchronization, and safer installation and upgrades.

The package targets SiMa.ai Platform 2.1.3 and uses Model Zoo 2.1.3.

Highlights

Eight new applications

  • MIPI camera capture captures NV12 frames through CameraInput, exercises the strict zero-copy DMA-BUF path, and reports timing, delivery, and luminance diagnostics. The application was validated with an IMX568 camera.
  • Thermal face detection runs a custom YOLOv5s-face model in C++ and Python and publishes face boxes and five facial landmarks to Insight.
  • Multi-stream pose estimation runs one shared YOLO26 pose model across as many as four RTSP streams and publishes complete 17-keypoint COCO metadata.
  • SuperPoint feature extraction produces sparse keypoints and descriptors for matching, tracking, mapping, and localization. Core-managed preprocessing maps coordinates back to the original image.
  • SSD-MobileNet detection adds C++ and Python applications for SSD-MobileNet V1/V2 and SSDlite-MobileNetV3 with model-managed preprocessing and decoding.
  • PCIe high-density detection runs shared YOLO26 inference across 16, 24, or 48 RTSP streams using an x86 host and Modalix PCIe card.
  • Neat GenAI Studio provides a local interface for language and vision-language chat, image and camera analysis, model management, Whisper transcription, speech output, RAG, and diagnostics. (Multi-Modal Assistant is deprecated)
  • YOLO26 tiny-drone tracking tracks small UAVs across as many as four RTSP streams and publishes annotated video with stable per-stream tracking IDs.

H.265 RTSP support

Detection, tracking, segmentation, and pose applications can accept H.265/HEVC RTSP sources alongside H.264. Select h265 or hevc in the application configuration without changing source code.

The applications choose the matching decoder, encoded format, RTP payload, and Insight sender. Multi-stream applications can forward the original encoded H.265 stream to Insight while decoded frames continue through inference, avoiding a board-side re-encode.

H.264 remains the default. H.265 playback through Insight requires host hardware HEVC decoding because Chromium does not provide a WebRTC H.265 software-decoder fallback.

Faster model loading

Apps 0.5.0 uses the optimized model archive loader in Neat Library 0.4.0. The loader decompresses each archive once, validates and extracts from the same snapshot, and uses eligible NVMe storage for private extracted packages.

Modalix measurements from Core #653 and Core #669 include:

  • YOLO26 model loading reduced from 32.47 seconds to 1.63 seconds.
  • A 150 MB reference model reduced from 278.65 seconds to approximately 4.96 seconds.

Results depend on model size, storage, and page-cache state. Existing model archives and the public Model API remain compatible.

Synchronization and scale

  • The single-stream object detector and Detection-to-VLM assistant produce video and metadata from one graph timeline, improving Insight frame correlation.
  • Instance segmentation sends polygons as structured metadata. Insight renders the masks and contours while the underlying video remains reusable.
  • The default high-density profile increases from 16 streams at 720p25 to 16 streams at 720p30. The updated profile sustained 30 FPS per channel on Modalix, or 480 aggregate FPS.
  • The 48-stream detector uses inference progress for liveness checks, preventing healthy burst-scheduled workloads from being reported as stalled.
  • RTSP probing honors the configured TCP transport.
  • The model benchmark compares YOLO26 standard and BoxDecode postprocessing routes and records the selected route and settings.

Installation and model delivery

  • The package installs Neat Library v0.4.0 at artifact revision 7dc2231af8aa, matching the dependency used to build and validate Apps.
  • Model Zoo commands and SDK model URLs select version 2.1.3.
  • Models remain separate from the Apps package and use the download instructions in each example.
  • Upgrades install the exact selected Apps package, request confirmation before replacement, and preserve prebuilt-apps/models/.
  • The installer retains a recoverable backup and restores the previous installation if dependency installation or runtime promotion fails.
  • Installed bundles record the Apps ref and exact commit alongside the selected Neat Library dependency.
  • Insight remains independently installed through its default sima-cli target.

Installation

Install Apps v0.5.0 on a supported Modalix or DevKit target:

sima-cli neat install apps@v0.5.0
cd prebuilt-apps

Compatibility

Component Version
SiMa.ai Platform 2.1.3
Neat Library v0.4.0 (7dc2231af8aa)
Neat Insight Latest release (v0.0.7 at release time)

Apps 0.5.0 uses Model Zoo 2.1.3 for documented model downloads.

Migration notes

  • Installing Apps 0.5.0 may replace the installed Neat Library package with v0.4.0:7dc2231af8aa.
  • H.264 configurations remain compatible.
  • H.265 playback requires host hardware HEVC decoding.
  • User-managed models under prebuilt-apps/models/ survive reinstallation and upgrades.
  • Store custom configurations and persistent user files outside the installer-managed bundle.
  • Insight is not pinned by Apps. The installer selects the latest available Insight release.

Validation

  • The exact release-preparation candidate passed the complete Vulcan build, Modalix runtime, packaging, and publication workflow.
  • The merged release commit passed the SDK build and all activated Apps runtime tests on Modalix in the main-branch Vulcan workflow.
  • Hardware coverage includes twelve SSD configurations, four SuperPoint accuracy variants, four-stream pose estimation at approximately 28 to 30 FPS per stream, an 836-frame IMX568 MIPI capture with zero timeouts, and repeated strict thermal-face runs in C++ and Python.
  • The 48-stream detector completed a five-minute run with every stream advancing and no metadata failures.
  • Segmentation validation matched every received metadata message to video with zero reassembly drops.

Key pull requests

Full changelog: v0.4.0...v0.5.0

Contributors

Thank you to everyone who contributed to this release:

Assets 2
Loading