v1.4.0

📢 New chapter in Workflows begins now

This release introduces the biggest change to the Workflows execution stack since its inception: an experimental tensor-native execution mode that keeps video frames and model predictions on the GPU end to end, paired with hardware-accelerated video decoding. It ships disabled by default — nothing changes for existing deployments until you opt in.

⚡ Tensor-native Workflows execution (experimental) — #2357

Until now, every stage of a video Workflow — decoding, pre-processing, inference, post-processing, visualization — passed images and predictions through CPU memory as numpy arrays and sv.Detections, even when both the decoder and the model ran on the GPU. Tensor-native mode removes those round-trips: video frames are decoded straight into CUDA tensors, flow through Workflow blocks as torch tensors and native prediction objects, and only touch the CPU when something genuinely needs them there (a sink writing to disk, an HTTP response being serialized). This is an experimental mode, but it delivers promising speed-ups on GPU-equipped machines — most visibly in multi-camera InferencePipeline deployments on Jetson-class devices, where memory bandwidth is the scarcest resource.

To enable it, set one environment variable on the server or pipeline process:

docker run --rm --gpus all --network host \
  -e ENABLE_TENSOR_DATA_REPRESENTATION=True \
  roboflow/roboflow-inference-server-gpu:1.4.0

The mode is designed for and tested against InferencePipeline (video processing) — that is where the GPU-resident data path pays off. The HTTP server tolerates the flag, but video pipelines are the intended consumer:

from inference import InferencePipeline

pipeline = InferencePipeline.init_with_workflow(
    video_reference="rtsp://...",
    workspace_name="your-workspace",
    workflow_id="your-workflow",
    on_prediction=my_sink,
)
pipeline.start()

With the flag enabled, pipelines also default to a new self-tuning AUTO video-processing mode that matches the frame-collection rhythm to your cameras' actual frame rates. If you need the exact pre-1.4.0 collection behavior, pass video_processing_mode="legacy" — it is the explicit escape hatch and preserves the old semantics byte for byte.

Your Workflow definitions do not change: the same JSON runs in both modes, block names and kinds are identical, and with the flag off (the default) execution is unchanged from v1.3.x.

🎥 Hardware-accelerated video decoding

Video sources can now decode on the GPU instead of the CPU: NVDEC-backed producers land decoded frames directly in CUDA memory on both Jetson (a dedicated zero-copy tensor bridge) and x86 GPUs (GStreamer/CUDA and PyNvVideoCodec producers). Producer selection is automatic per platform and source type, with a transparent fallback to the classic OpenCV decoder whenever hardware decoding is unavailable — and the selection decision is now logged, so you can always tell which decoder served a source.

🤖 JetPack 7.2 (Thor) support

This release adds a server image for JetPack 7.2 (roboflow/roboflow-inference-server-jetson-7.2.0), bringing the full stack — including tensor-native mode and hardware decoding — to NVIDIA's newest Jetson generation.

⚠️ Breaking changes when tensor-native mode is enabled

If you do not set the flag, there are no breaking changes — flag-off behavior matches v1.3.x. When you do enable it, review the following before flipping it on a production deployment:

• In-process consumers receive tensors, not numpy. Custom on_prediction sinks and any code reading video_frame.image or prediction objects inside the pipeline process will receive torch tensors and native prediction objects instead of numpy arrays and sv.Detections. Built-in sinks were adapted; custom sinks that call OpenCV or numpy functions directly need to materialize first (helpers are provided in the codebase, e.g. for converting a tensor video frame back to numpy at the sink boundary).
• Some serialized payloads change shape. Most notably, instance segmentation masks may be serialized as compact run-length encodings (rle_mask) instead of polygon points for some block versions, and embedding/tensor outputs serialize as plain lists. If your downstream code parses serialized Workflow responses, verify it against a flag-on deployment before rolling out.
• Custom Python blocks keep working unchanged by default. Dynamic blocks declare a tensor_compatibility contract that defaults to legacy_compatibility: the engine converts native tensor objects to the documented sv.Detections/numpy representations at your block's input boundary and converts your returned values back, so existing user code runs as-is (locally and via remote execution). Blocks that want the GPU-resident data path can opt into tensor_compatibility: tensor_native and receive native objects directly (local execution only, for now).

🛠️ For contributors

Tensor-native mode changes how Workflow blocks are authored and tested, so please read this before opening a pull request that touches blocks:

• Blocks that consume or produce images/predictions now come in pairs: the classic numpy implementation (v1.py) and a tensor-native sibling (v1_tensor.py) registered in the block loader under the flag. Siblings re-implement the numpy behavior against native objects — they are deliberately standalone files, not wrappers around the numpy code. New blocks in these categories should ship both implementations; scalar/text/flow-control blocks typically need only one.
• Tests must pass in both flag directions. CI runs the Workflow suites with the flag on and off; use the established per-file _TENSOR_ONLY / _NUMPY_ONLY marker pattern for tests that only make sense in one mode, and mirror assertions across siblings.
• Manifest-level declarations (output kinds, dependent-resource discovery for model pre-loading) must be present on both siblings — a declaration only on the numpy side silently disappears under the flag.

This is a large and still-moving surface — if anything about the sibling pattern, the conversion boundary, or the testing conventions is unclear while you are contributing, please open a GitHub issue and ask. We would much rather answer questions early than review a pull request built on a wrong assumption.

📦 Side note — Cosmos 3 GPU image

This release ships a dedicated -cosmos3 variant of the GPU server image with preview support for NVIDIA Cosmos 3. The variant exists because Cosmos 3 required transformers and diffusers builds that were not officially released at the time the image was cut, so it pins pre-release snapshots of both. Official releases of those packages are now available downstream, so you can expect Cosmos support to arrive in the standard upstream GPU image in an upcoming release — at which point the dedicated variant will no longer be necessary.

Full Changelog: v1.3.10...v1.4.0