v2.0.0

v2.0.0 — Masks Leave the CPU, Builds Get Out of Your Way, Multi-Stream Turns On

v1.3 removed the CPU from the data path. v2.0 removes it from the result path.

Every segmentation mask, depth map and semantic map used to be resized, binarized and alpha-blended
by hand on the host — for every instance, every frame, in floating point. That work now happens on
the device, in the same CUDA kernels the models themselves use, and both renderers share one
implementation of it. The pictures did not change. Not by a single byte.

Alongside it: three build layouts instead of one, six one-line build scripts, twenty-six sample
programs that finally look like each other, and multi-stream inference that runs up to nine 1080p
feeds on one GPU — the last of those at 60% and climbing.

---

TL;DR

	v1.3.0 (Oct 21)	v2.0.0 (today)
Registered models	34 (3 of them legacy duplicates)	31 — one implementation each
Build layouts	1	3 (+ optional OpenCV=OFF build)
Mask / depth / sem composition	per-pixel loops on the host	CUDA kernels, on the device
Multi-stream	pipeline only, no CLI	up to 9× 1080p verified, 60% shipped
Build entry point	one README full of flags	six scripts, one line each
Tests	always built	off by default, opt-in
Rendering backends	OpenCV	OpenCV + OpenGL (CUDA interop)

---

🧠 Masks stopped crossing the bus

A 1080p segmentation frame with a dozen instances used to cost, per instance: one full-frame bilinear
upsample, one threshold pass, and one alpha-blended pixel loop — all on the CPU, all in float, all
inside the renderer. That was the last place where a GPU pipeline quietly handed its work back to the
host, and it showed: with rendering enabled, seg and det were within 13 ms of each other over
520 frames, because the renderer was the cost.

Now:

• seg_mask_post does binarization, box cropping, in-place 0/1 float conversion and candidate
  counting in a single pass over the device buffer
• CvScaleMaskPlan is a kernel-for-kernel port of ATen's bilinear upsample — the same
  coefficients, the same rounding, the same edge handling
• overlay_stage composites masks, depth maps and semantic maps into the frame on the device,
  shared by the OpenCV and the OpenGL renderer

The host is left with a cudaMemcpy, a memcpy, and four box coordinates per instance. No pixel
loop survives anywhere on the CPU.

The proof it is not a different picture:

Check	Result
Masks vs F.interpolate + > 0.5 (PyTorch)	0 / 2,064,960 pixels differ
Rendered output vs the old host path	MD5 identical, byte for byte

⚙️ Three builds, six scripts, no flags to remember

The pipeline now ships as three coherent layouts, each with a name that says exactly what is inside:

Build directory	TensorRT	OpenCV	FFmpeg	OpenGL	What it is
build	✅	✅	—	—	default: images, folders, cameras, videos
build-ffm	✅	✅	✅	—	+ NVDEC hardware decoding
build-gl	✅	✅	✅	✅	+ zero-copy multi-stream display

Six scripts, named after the operating system and the libraries they link — nothing to decode:

scripts/build_windows_tensorrt-opencv.ps1
scripts/build_windows_tensorrt-opencv-ffmpeg.ps1
scripts/build_windows_tensorrt-opencv-ffmpeg-opengl.ps1
scripts/build_ubuntu_tensorrt-opencv.sh
scripts/build_ubuntu_tensorrt-opencv-ffmpeg.sh
scripts/build_ubuntu_tensorrt-opencv-ffmpeg-opengl.sh

Two switches do the rest:

-DTRT_ALPHA_WITH_OPENCV=OFF      # optional degraded build: video sources + --show only
-DTRT_ALPHA_BUILD_TESTS=ON       # tests are OFF by default; samples are ON
-DTRT_ALPHA_BUILD_SAMPLES=OFF    # skip samples/ and demos/ entirely

🎥 Multi-stream: 9 feeds, one GPU, 60% done

Nine 1080p (1912×1080) H.264 streams, decoded by NVDEC, batched through one engine, laid out in a
single window and drawn straight from GPU memory through CUDA-GL interop:

• one decode thread and one result queue per source — a slow camera cannot stall the other eight
• frames reach the renderer as device pointers; host copy OFF means the pixels never touch the host
• the overlay (boxes, masks, keypoints) is composed on the device before the tile is drawn
• click a tile to zoom, click again to go back to the grid

Where it stands — honest status:

	Status
Per-source decode, batching, inference	✅ done
Per-source result queues + drop accounting	✅ done
CUDA-GL interop display, tiled layout, click-to-zoom	✅ done
Device-side overlay composition for all task families	✅ done
9× 1080p, 30 fps, hours-long soak	🚧 in progress
Reconnect / stream-drop policy	🚧 planned
Per-tile HUD (FPS, latency, dropped frames)	🚧 planned
Encoder path (NVENC) for recording streams	🚧 planned

Call it 60%. The interesting half works; the boring half — reconnect, soak, HUD — is what makes
it something you leave running overnight. The OpenGL guide is held back until that half lands.

🛡️ It survives being squeezed

Nine 1080p streams want nine page-locked host buffers of 12.39 MB each. RLIMIT_MEMLOCK is 64 MB by
default on WSL2 and stock Linux, and it is a hard limit — a non-root process cannot raise it. The
old behaviour: cudaMallocHost fails, the pool throws, the source thread dies, and one tile goes
NO SIGNAL in the middle of playback.

Pinned memory is a performance optimisation, not a correctness requirement, so it now degrades:
tryAllocate() returns an empty block, the buffer falls back to pageable host memory, and a single
warning explains why. The failed allocation's CUDA last-error is cleared on the way out — otherwise
it resurfaced later as a phantom nv12ToBgr failed: out of memory from an unrelated kernel.

📦 Twenty-six samples that finally agree with each other

Every sample is now one directory, one task, one shape:

sample_yolov26_det  "<engine.trt>" "<input>"  [--save]

Same ModelConfig → InferencePool → OpenCVSource → OpenCVRenderer → Pipeline wiring
underneath, so reading one sample teaches you all twenty-six.

---

⚠️ Breaking changes

Three legacy models are gone. yolov8_old, yolov8_pose_old and yolov8_seg_old — kept as
A/B references for the fused rewrite — are removed, along with their configs, tests and engines.
They were duplicates of yolov8_det / yolov8_pose / yolov8_seg, which are faster and
bit-exact.

Before	After
34 registered models, 3 of them legacy	31 registered models
ModelType::yolov8_old	ModelType::yolov8
configs/yolov8_seg_old.ini	configs/yolov8_seg.ini

If you have an .engine built for one of the _old types, re-export it from the current model —
the ONNX is different, the shapes are different, and the old engine will not load.

🚀 Upgrading

git pull
rm -rf build build-ffm build-gl          # old trees carry the removed targets
./scripts/build_ubuntu_tensorrt-opencv.sh        # or the Windows .ps1

Nothing else moves. CLI flags, INI keys, InferParams and the Infer API are unchanged; the
FFmpeg and OpenGL paths are still opt-in at compile time, and a build without FFmpeg still throws
rather than silently downgrading when asked for --decode ffmpeg.

✅ Verify it in sixty seconds

./build/bin/Release/trt_alpha list          # 31 models, plus the header row
./build/bin/Release/trt_alpha run --net yolov26 --source data/bus.jpg --show
./build/bin/Release/trt_alpha run --net yolov26 --source data/people.mp4 --save output/det/

./build/bin/Release/trt_alpha run --net yolov26 --source data/people.mp4 \
      --show --decode ffmpeg --hwaccel nvdec     # needs build-ffm

The config box tells you what the engine actually declares; the source line tells you which decoder
actually ran; the last line tells you whether a single byte of pixel data touched the host. No
guessing. (On Windows, swap ./build/bin/Release/trt_alpha for .\build\bin\Release\trt_alpha.exe.)

🗺️ What's next

• Multi-stream to 100% — soak, reconnect, HUD, encoder. This is the head of the queue.
• NVENC — --save currently runs a JPEG farm on the CPU while the GPU has spare silicon
• Zero-copy for every model — today the fused detection family reads device pixels; the rest
  still upload, and every family that joins gets the same win
• Tracker — architecture is designed and written up; the association stage is the blocker
• GStreamer — the data source interface is ready for it

🔍 Known limitations

• Hardware acceleration covers decoding; pre/post-processing runs on the existing CUDA kernels
• 4K / HEVC sources are not yet in the measured matrix (H.264 1080p and 720p are)
• The OpenGL renderer does not draw segmentation masks when the build has OpenCV disabled
• Under WSLg the GL path is Mesa llvmpipe (software rasterization) and will be slow — native
  desktops only
• Multi-stream is exposed through the CLI but not yet through the high-level Infer API

---

_{31 models · 3 builds · 6 scripts · 26 samples · one pipeline that does not let the CPU see the pixels}