v1.5.2

🚀 Added

🎬 Action Recognition — a new video task

Action Recognition labels frame ranges in a clip: the output is a list of (start_frame_idx, end_frame_idx, class) tuples, and ranges may overlap. The temporal contract travels with the model — a VideoSampling description states the window length, the sample rate and the limits the model was trained with, so a caller sends a clip and nothing else, with no window or frame-rate knobs to get wrong. Two kinds of checkpoint share the one output type: fine-tunes trained on Roboflow use sliding windows and a constrained decoder, while zero-shot models read a whole clip in one call and name the events they find in their own words (@leeclemnet, #2854).

How to use it. Four surfaces reach the task: the roboflow_core/roboflow_action_recognition_model@v1 Workflow block, the new POST /infer/action_recognition route, the legacy /{model}/{version} route that serverless serves today, and client.infer_on_video in the SDK. Zero-shot checkpoints are endpoint-only — a stream has no end to span, so the block refuses a whole-video package and points at the endpoint instead.

🧠 Roboflow Cosmos 3 Edge fine-tunes

Cosmos 3 Edge fine-tunes trained on Roboflow ship as LoRA adapter packages, and the server now loads them over the base checkpoint the same way the other fine-tuned VLMs do — until now Cosmos3EdgeReasoner could only load a full checkpoint. The roboflow_core/cosmos3_edge@v1 block accepts a fine-tune's model id in model_version and no longer injects its own default system prompt, so a fine-tune reached through Workflows is prompted exactly as it was trained. As part of this, InferenceConfig no longer requires a training input size for models that accept any input size, and the model gained the documentation page it never had (@probicheaux, #2905).

📍 RF-DETR keypoint detection on TensorRT

The registry had an ONNX backend for RF-DETR keypoints and nothing else. This release adds RFDetrForKeyPointsTRT: AutoModel.from_pretrained(..., backend=BackendType.TRT, device="cuda") loads a local TRT package and returns KeyPoints with the same contract as ONNX, reusing the instance-segmentation TRT engine-loading path with CUDA graphs (@MehdiH7, #2900).

We've tested the model locally, but registration of platform weights is going to be completed soon. Nevertheless, we would like to highlight @MehdiH7 contribution.

🤖 More VLM options in Workflows

The Anthropic Claude blocks gain Claude Fable 5.1, the Google Gemini v5 block gains Gemini 3.8 Flash and the Meta VLM v2 block gains Muse Spark 1.3. OpenRouter-backed VLM blocks are now routed to native-precision providers, and GLM 5.3 Flash detection switched to the validated bbox_2d prompt. These landed on the post-v1.5.1 fast-track branches and are upstreamed here (@SkalskiP, @Erol444, #2910, #2913, #2926).

🔍 Camera Focus v2 runs on the GPU

When the frame is already a device tensor, roboflow_core/camera_focus@v2 computes grayscale, Sobel and the focus measure with torch ops and pulls the overall value plus every per-box mean to the host in a single sync, instead of materialising the full frame and copying it several more times for the overlays. The numpy path is untouched, and the device path is bit-exact against it — overlays included (@shntu, #2901).

🩺 Proxy health probe — GET /secure-gateway/health

Deployments that reach Roboflow through the secure gateway (or the legacy license server) had no way to tell a broken proxy from a broken server: a model download failing behind the gateway looked exactly like the inference server being unhealthy. The server now exposes an opt-in GET /secure-gateway/health route that probes the configured proxy's own /health endpoint from inside the inference server and reports a verdict, so monitoring can point at the right component (@PawelPeczek-Roboflow, #2908).

How to configure it.
The route is disabled by default and only makes sense on a server that has a proxy configured. Set SECURE_GATEWAY_HEALTH_ENDPOINT_ENABLED=True next to your existing SECURE_GATEWAY value (the legacy LICENSE_SERVER variable is still honoured). Additionally SECURE_GATEWAY_HEALTH_CHECK_TIMEOUT determines the timeout requesting response from the proxy

docker run -d --name inference -p 9001:9001 \
  -e SECURE_GATEWAY=https://gateway.internal \
  -e SECURE_GATEWAY_HEALTH_ENDPOINT_ENABLED=True \
  -e SECURE_GATEWAY_HEALTH_CHECK_TIMEOUT=5 \
  roboflow/roboflow-inference-server-cpu:1.5.2

Try it out with

curl -s http://localhost:9001/secure-gateway/health
# {"status": "healthy", "reason": null, "gateway_status_code": 200, "latency_ms": 12.4}

📊 Nsight trace analysis in the profiling tools

The profiling harness gains a scriptable analysis step: it runs nsys stats on a captured .nsys-rep, joins host-side NVTX ranges with GPU-projected ranges by process, thread and range id, validates the iteration set declared by the run manifest, and writes a versioned analysis.json next to the trace — no desktop UI needed to review a capture (@Silas-Asamoah, #2878).

🔧 Fixed

• Custom Python blocks over WebSocket stay connected — the webexec server closed idle sockets after ten seconds while the client's keepalive pings were answered below the application layer and never reset that timer, which sat behind the recent video-processing incidents. The protocol moves to v2 with app-level heartbeats that actually reset the server's idle timer, typed frame handling so a stray text frame is treated as a dead connection instead of parsed, request ids with per-container dedup so a resend after a lost response is answered from cache, an announced graceful close, and an opt-in loud failure when a reconnect lands on a container that lost the Python session. WEBEXEC_WS_MAX_CONNECTION_SECONDS drops from 3600 to 600 so the server always closes cleanly before Modal's per-input timeout, and websocket-client is now a declared dependency (@rafel-roboflow, #2879).
• Custom Python blocks fed with semantic segmentation predictions work again — every run failed with a lost WebSocket and no traceback, because the transport dropped the RLE masks and sent an image with no dimensions. RLE masks and image size now survive both Modal transports, and the confidence map is stored so boolean filtering of the detections works too (@Erol444, #2923).
• VLM output parsing in the vlm_as_* blocks is lenient and model-agnostic — replaying two weeks of playground parse failures drove the change: single detection objects emitted without their list, list bodies with the opening bracket dropped, one empty array per line, and classifier outputs without their wrapper are all recovered now, while truncated output still fails loudly (@Erol444, #2921 via #2926, #2930).
• Cache Get, Cache Set and the ONVIF sink no longer refuse to run based on step_execution_mode — that flag only says where model steps execute and was the wrong proxy for "is this a long-lived process". The cache blocks now carry the same soft multi-replica caveat every tracker has, and the ONVIF sink keeps its hard restriction on hosted runtimes that cannot reach a customer LAN (@rafel-roboflow, #2918).
• OWLv2 cache test kept off torch.compile — torch 2.14 rejects the mocked model (@probicheaux, #2914).
• Jetson 6.0.0 build repaired — it copied libnvdla_compiler.so from a directory that only exists at container runtime and had failed on every push since June (@probicheaux, #2924); superseded by the deprecation below.
• Docs sitemap no longer lists the homepage twice (@Erol444, #2899).

🚧 Maintenance

• Security dependency refresh (2026-09-04) — inference-models moves to 0.37.0 (from 0.37.0rc3) and transformers to the 5.15 line in its lock, alongside hydra-core, tornado and mkdocs-material (@PawelPeczek-Roboflow, #2929, #2931). See the JetPack 5 notice below for the one image this refresh does not cover.
• CI hygiene — the e2e CI suite is green again and obsolete AWS references are gone from the GitHub Actions workflows (@grzegorz-roboflow, #2895, #2894); linter cleanup (@PawelPeczek-Roboflow, #2911).

⚠️ Jetson platform notices

• JetPack 6.0 and JetPack 7.1 images are discontinued. Starting with v1.5.2 we no longer build roboflow-inference-server-jetson-6.0.0 and roboflow-inference-server-jetson-7.1.0 (#2932). If you are on JetPack 6, move to roboflow-inference-server-jetson-6.2.0; if you are on JetPack 7, move to roboflow-inference-server-jetson-7.2.0. Both are drop-in replacements within the same JetPack generation.
• JetPack 5 and JetPack 6.2 reach end of life at the end of 2026. The jetson-5.1.1 and jetson-6.2.0 images keep receiving builds until then and stop afterwards. JetPack 7.2 is the platform going forward, so plan the migration now — and if your fleet cannot move on its...

v1.5.1

🚀 Added

🪣 Optional S3-compatible shared cache for model files

Multi-node deployments no longer have to download the same model weights from origin once per node. An opt-in cache layer sits between the local filesystem cache and the original download source: on a local miss, inference checks a shared S3-compatible object store first, so an artifact downloaded by one node is reused by every other node — cutting cold-start latency and repeated origin downloads (@ecarrara, #2769).

How to use it. Set INFERENCE_MODELS_MODEL_BLOB_CACHE_ENABLED=True (default is off) and point INFERENCE_MODELS_MODEL_BLOB_CACHE_BUCKET at your bucket; endpoint URL, region, prefix, credentials, timeouts and circuit-breaker thresholds are all configurable through the INFERENCE_MODELS_MODEL_BLOB_CACHE_* family. The layer is designed to never take a deployment down: cache misses, timeouts, corrupted objects and service failures all fall back to the original source, and misconfiguration fails open.

🔢 Token-usage outputs on remote VLM Workflow blocks

Remote VLM blocks now tell you what they cost: new input_tokens and output_tokens outputs report billed token counts, with both Roboflow-key pass-through and user-supplied API keys (@SkalskiP, #2858, fast-tracked into the v1.5.0 post-releases). Shipped as new block versions — openrouter@v2, google_gemma@v3, meta_vlm@v2, qwen_vlm@v3, anthropic_claude@v4, google_gemini@v5, open_ai@v6, spacexai@v2 — so existing workflows keep their behavior byte-for-byte. The OpenRouter v2 and Gemma v3 blocks additionally gain a reasoning_effort parameter (none/low/medium/high/xhigh), OpenRouter v2 exposes the reasoning trace as a thinking output, and both raise the default max_tokens from 500 to 2048 so reasoning models don't burn the whole budget on internal thinking.

🧪 Experimental CUDA 13 server build

A new x86 GPU server image, roboflow/roboflow-inference-server-gpu-cu13, built on a CUDA 13.2.1 / Ubuntu 22.04 base with CUDA-13 builds of FFmpeg, GStreamer and OpenCV (@PawelPeczek-Roboflow, #2884 and #2885). Verified end-to-end on an RTX 6000; published with the same versioned tagging as the other server images. Experimental for now — one known limitation is that YOLACT ONNX models show systemic errors on this build, to be resolved separately.

📦 Custom file names for the Write Vision Event Bundle sink

The Write Vision Event Bundle sink gains an optional file_name field — a literal or a selector — so exported bundles can carry meaningful, downstream-friendly names instead of generated ones (@rvirani1, #2887). Names are strictly validated (safe character set, no path separators, length-capped) at both the manifest and runtime layers, and the sink refuses to overwrite an existing bundle of the same name — the collision check is atomic, so concurrent writers cannot clobber each other even on removable media.

📹 Grab-cadence telemetry for the GStreamer CUDA video producer

The GStreamer CUDA producer now reports per-producer frame-gap statistics (count/mean/max plus under/over-period buckets) and bounded source-stream metadata through tensor_bridge_stats, and a new ROBOFLOW_GSTREAMER_CUDA_APPSINK_SYNC env var can opt the appsink into clock-synced delivery for controlled comparisons — the low-latency default is unchanged, and the frame path itself is untouched (@hansent, #2868).

🔧 Fixed

• Batch- and list-shaped data now cross the Modal remote-execution boundary intact — Custom Python blocks executed remotely on Modal received broken data whenever a batch shape crossed the wire: a Batch[...] input (for example Batch[WorkflowImageData] from a crop step) arrived in the sandbox as a stringified placeholder, and a list-shaped BlockResult returned over the HTTP transport failed to deserialize on the way back. Both directions are fixed, on both the HTTP and msgpack transports, with batch indices preserved exactly so downstream index-based filtering keeps working (@jeku46, #2870). The fix spans the inference client and the hosted execution sandbox; the hosted side is rolled out alongside this release.
• Workflow step workers now run in the request's full context — steps executed in parallel (including nested block worker pools) run inside a per-task snapshot of the submitting thread's contextvars context, so block code and instrumentation observe the request's values for every ContextVar, and context set inside a step no longer leaks into later requests that reuse the same pool thread (@SolomonLake, #2843).
• MQTT Writer sink lifecycle rebuilt — the enterprise MQTT sink's client management is rewritten, closing 13 confirmed defects: a failed first connect permanently poisoning the client, rejected CONNACKs logged as "Connected", reconnect races, silently publishing to the wrong broker after host or credential changes, skipped username-only auth, and hangs on invalid timeout or port values. Failures now log and return error_status=True without stopping the workflow, and a new opt-in fail_fast field raises instead (@grzegorz-roboflow, #2876).
• TRT CUDA-graph capture no longer trips over concurrent pipelines — graph capture ran in process-wide mode, so any other thread touching the default CUDA stream during a capture failed with CUDA error 906. Capture is now thread-local and serialized under a lock; same graph, same replay path, no change to predictions (@shntu, #2866).
• MemoryCache per-key locks validate their generation — a waiter could acquire a lock whose cache entry had already expired and been replaced, letting two callers into a critical section meant for one. Stale-generation locks are now released and retried within the original timeout budget, at under 0.25 µs of overhead per operation (@voropaevv, #2804).
• Cache Set / Cache Get blocks clean up every namespace they touch — an instance serving multiple videos only released the last video's namespace, and cleanup from one instance could wipe keys still in use by another instance on the same video. Namespaces are now tracked per instance and reference-counted, making cross-instance sharing safe (@davidnichols-ops, #2834).
• from inference import Model works again — Model was advertised in __all__ but missing from the lazy loader's registry, so the import passed type checking and failed at runtime (@davidnichols-ops, #2844).
• google_gemini blocks forward temperature to thinking-level models — previously an explicitly set temperature was silently dropped for Gemini 3.x models, so structured-extraction workflows ran at Gemini's default temperature 1.0 with no way to opt out (@gavwin, #2813).
• Jetson runtime env compatibility — JetPack 6+ images set RUNNING_ON_JETSON, but the legacy VideoSource path only checked RUNS_ON_JETSON; the latter now falls back to the former, restoring correct RTSP/GStreamer producer selection on those images (@hfsc2004, #2806).
• Secure-gateway gate applies to per-run step_execution_mode — with SECURE_GATEWAY set, a caller passing step_execution_mode=remote per run could bypass the gate that blocks remote step execution against the hosted Roboflow API. The restriction is now enforced where the parameter is consumed, and such runs raise a clear error; self-hosted remote targets behind the gateway remain allowed (@adhavan18, #2805).
• Block-documentation links repaired — old /workflows/blocks/<slug> gallery URLs on inference.roboflow.com forward to the right docs.roboflow.com page instead of dropping the slug (@Erol444, #2839), and generated block-page slugs no longer break markdown cross-references for block names containing brackets like PTZ Tracking (ONVIF) (@adhavan18, #2830).

⚙️ Execution Engine v1.15.1

The Workflows Execution Engine version moves from v1.15.0 to v1.15.1, claiming the Modal boundary serialization fix (#2870) and the step-worker context propagation change (#2843) described above. The full version-by-version record lives in the Execution Engine changelog.

🚧 Maintenance

• Quieter serverless logs — the duplicated per-request "Request received" line and health/probe access-log lines are demoted to DEBUG; access logs for real requests stay at INFO, since the dedicated-deployment auto-pause daemon uses them as its activity signal (@bigbitbus, #2877).
• Webexec Modal deployment CI — deployment workflow and environment wiring for the hosted Custom Python execution app (@grzegorz-roboflow, #2848, #2849).
• Post-v1.5.0 fast-track packaging (@PawelPeczek-Roboflow, #2863, #2874, #2881), linter cleanup (#2867), a CI fix (#2890), and the PR review agent moves to Claude Opus 4.8 (@grzegorz-roboflow, https://github.com/roboflow/infer...

v1.5.0

🚀 Added

🔐 Header-based API-key authentication

Until now, the Roboflow API key travelled to the inference server as the api_key query parameter (API v0) or as a JSON-body field (API v1) — which means it could end up in access logs, proxy logs and browser histories. Starting with this release, the server also accepts the key as a standard Authorization: Bearer <api_key> header, and the SDK can send it that way (@PawelPeczek-Roboflow, #2810).

How to use it. The SDK gains a new api_key_transport field on InferenceConfiguration with three modes:

• legacy keeps today's behaviour byte-for-byte — key in the query parameter or body, works against every server version
• both keeps the legacy channels untouched and adds the Authorization: Bearer header on top — safe against every server version, since older servers simply ignore the header and newer servers read it.
• header sends the key only in the header — no key in URLs or request bodies, but it requires a server from release 1.5.0 onward; against an older server your requests arrive keyless and fail auth.

from inference_sdk import InferenceHTTPClient, InferenceConfiguration

client = InferenceHTTPClient(api_url="http://localhost:9001", api_key="<KEY>").configure(
    InferenceConfiguration(api_key_transport="both")
)

What are the defaults
Nothing changes on upgrade unless you opt in. The SDK default remains legacy (you will see a one-time guidance warning nudging you towards the header), the server accepts header-carried keys by default (ALLOW_API_KEY_FROM_HEADERS=True), and Workflows blocks that call remote Roboflow APIs default to both (WORKFLOWS_REMOTE_API_KEY_TRANSPORT, values legacy/both/header). When several channels carry a key, the server resolves them in the order:

• query parameter
• Authorization header
• body field
• API_KEY env var.

Why both is the right mode for now. header-only is the destination, but flipping straight to it couples your client upgrade to your server upgrade and to every piece of infrastructure in between — some proxies and gateways strip or rewrite Authorization headers, and hosted endpoints migrate on their own schedule. both gives you the security benefit wherever the header is honoured while remaining compatible with everything else, and it costs nothing: the header carries the same key the legacy channel already delivers. Run both for the transition period, and switch to header once you have confirmed the whole path speaks it.

Migration sketch

1. Upgrade your servers to 1.5.0.
2. Switch clients to api_key_transport="both" — this is safe immediately, even against servers you have not upgraded yet.
3. Once every server in the path is ≥ 1.5.0 and you have verified that your proxies pass Authorization headers through, switch to "header" and enjoy key-free URLs and bodies.
4. One caveat for self-hosted deployments sitting behind an auth proxy that forwards its own Authorization: Bearer <JWT> header (oauth2-proxy, GCP IAP and similar): the server will read that JWT as an API-key candidate ranked above body-carried keys. If that is your topology, set ALLOW_API_KEY_FROM_HEADERS=False on the server or stop the proxy from forwarding its token and raise an issue here, we will try to help.

📦 OFFLINE_MODE in inference-models, rebuilt around an explicit registry

OFFLINE_MODE in inference-models (now at 0.36.0) got a ground-up redesign of how offline model availability is decided (@PawelPeczek-Roboflow, #2833). Previously, offline serving inferred what was usable by inspecting and validating whatever it found in the model cache — an implicit contract that was hard to reason about and, in edge cases, stricter than intended. The new design makes the contract explicit: an offline-weights registry inside INFERENCE_HOME records exactly which models and packages were proven to load, and offline serving reads that registry through the same auto-negotiation path used online. What is registered loads; what is not, does not — and the error tells you precisely how to fix it.

How to use it
Warm-up and serving are now two explicit, mutually exclusive phases. On a machine with connectivity, run your models once with OFFLINE_MODE_WARM_UP=True — every successful load records the model, its packages and its platform-attested metadata in the registry. Then ship INFERENCE_HOME to the air-gapped target and run with OFFLINE_MODE=True. Setting both flags at once fails loudly at model load, by design. New AutoModel classmethods round out the workflow: list_offline_models() shows what the registry will serve, verify_offline_model() checks a record against the materialized files (optionally with hash verification), and purge_offline_model() removes one cleanly.

What you gain
TensorRT engine caching now works under OFFLINE_MODE, so warm restarts on air-gapped Jetson-class devices skip engine recompilation — that is minutes saved per model. Warm loads are much faster across the board, because cache-hit loads no longer re-hash every artifact on disk on every startup. Packages compiled and installed with the inference-compiler CLI are now first-class citizens of offline serving. And offline cache trees can be mounted read-only — offline serving treats the cache as immutable input and never writes to it.

Migration guide
The registry is the single source of offline truth, which has one important consequence: a cache that was warmed by "just running the models once online" — without OFFLINE_MODE_WARM_UP=True — has no registry records, and offline serving will not use it. Before upgrading an air-gapped fleet, re-warm on a connected machine with the flag set and ship the resulting INFERENCE_HOME.

Warning

Clients running in OFFLINE_MODE are responsible for consistency of data on their volumes. Additionally, since it is not possible to verify access credentials in Roboflow API w/o access to the Internet - security posture of the configuration must be ensured externally from the low-level engine running models.

🤖 Qwen VLM v2 workflow block

The unified qwen_vlm@v2 block brings Qwen-tuned OpenRouter plumbing and reasoning control to Workflows, shipped in a fast-track deployment post 1.4.1 release was aligned with main and is released now. (@SkalskiP, #2825).

🎯 Visual prompts for SAM3 Video Tracker

The SAM3 Video Tracker now accepts visual prompts, extending prompt-based tracking beyond text (@leeclemnet, #2787).

🧩 String Template block and SequenceJoin UQL operation

Workflows gain a String Template block for assembling text from step outputs, together with a SequenceJoin UQL operation (@JoeWayne, #2812).

📊 Usage rows now record the model variant

Usage-tracking rows can be attributed to the exact platform model variant (e.g. yolov8-n instead of just the architecture), resolved once at model load so the inference hot path stays untouched (@SolomonLake, #2811).

🏎️ Models that arrived before their invitations

We were apparently moving so fast this release that the code overtook the platform: three new model families are fully implemented in inference_models, while their packages are still making their way through platform registration. Consider this the trailer — Qwen3.8 27B VL with its /infer adapter (@hansent, #2801), the Qwen3.8 vLLM proxy with a qwen_vlm workflow variant for base-only serving (@hansent, #2802), and Mage-VL, a codec-native video VLM (@Erol444, #2820). The engines are on the tarmac; boarding passes are being printed. Coming soon.

🔧 Fixed

• Legacy RF-DETR preprocessing normalized BGR inputs in the wrong channel order — numpy (BGR) inputs are now normalized in RGB order, matching training (@probicheaux, #2828).
• Triton kernel runtime errors now fall back gracefully instead of failing the request (@dkosowski87, #2815).
• Tensor painters hardened against a CUDA SIGABRT in overlap owner resolution under the tensor-native Workflows path (@hansent, #2832).
• runtime_compatibility_hash removed from cache-manifest identity, so runtime-environment drift no longer invalidates otherwise-valid cached packages (@rafel-roboflow, #2809).
• Wheel builds no longer import the full runtime — building the inference wheels pulled in the whole package (torch, OpenCV) after the import changes, which broke image builds in slim stages; version resolution in the setup scripts is now import-free (@PawelPeczek-Roboflow, #2842).

🚧 Maintenance

• GPU build immune to PYTHONPATH changes (@PawelPeczek-Roboflow, #2823).
• Post-v1.4.1 fast-track follow-ups (@PawelPeczek-Roboflow, #2837).

🏅 New Contributors

• @JoeWayne made their first contribution in #2812

Full Changelog: v1.4.1...v1.5.0

v1.4.1

Warning

⚠️ Installing inference in Google Colab? Read this first

Starting with this release, pip install inference in a fresh Google Colab runtime can crash on import inference with RuntimeError: Detected that PyTorch and TorchAudio were compiled with different CUDA versions. Colab preinstalls a torchaudio built for an older CUDA than the torch that dependency resolution installs, and new transformers imports torchaudio when present in env.
Fix: remove the stale torchaudio before importing — it takes one cell:

!pip uninstall -y torchaudio

Run it right before pip install inference and before the first import — then no restart is needed. If you already hit the error, uninstall and restart the runtime (Runtime → Restart session) so the failed import is not cached. If you actually use torchaudio in the same notebook, reinstall a build matching your torch version instead of removing it.

🚀 Added

🤖 SpaceXAI Grok workflow block (Grok 4.6 / 4.5)

The new roboflow_core/spacexai@v1 block brings xAI's Grok vision models into Workflows, with Grok 4.6 as the default and Grok 4.5 selectable. It speaks xAI's OpenAI-compatible Responses API and supports the full VLM task lineup — unconstrained prompting, VQA, OCR, captioning, classification and structured answering — plus object detection using the percent-of-image box_2d contract that won the vlm-exam benchmark for Grok, with vlm_as_detector@v2 shipping the matching model_type="spacexai" parser. (@SkalskiP, #2799).

⚡ Gemini 3.7 Flash in the Google Gemini blocks

Google released Gemini 3.7 Flash on Aug 13 and it is selectable in google_gemini@v3 and google_gemini@v4 the day after, with thinking-level control and native code execution enabled — validated by full-benchmark runs across all six VLM tasks before flipping the switch (@SkalskiP, #2794).

📊 Deeper usage and cache telemetry

Usage rows can now be attributed by model architecture (resource_details.model_type, e.g. rfdetr-seg-nano) and by the input resolution the model actually ran at, bucketed in megapixels — resolved from a process-local cache at model load so the hot path never does a registry lookup (@SolomonLake, #2782). SAM3's visual-segment encoder fast path also gains a bounded Prometheus counter reporting embedding-cache hit / miss / not_attempted outcomes (@hansent, #2779).

🔧 Fixed

• Vision Events sink no longer drops assume-identity headers — the sink built its own request headers instead of going through build_roboflow_api_headers(), so deployments authenticating via ROBOFLOW_API_EXTRA_HEADERS had their event writes silently rejected upstream while the block reported success (@rvirani1, #2774).
• MemoryCache.acquire_lock actually serialises on one key now — a check-then-act race let two callers racing on a missing key each build their own lock and both enter the protected section, and the lock refresh reused Lock.acquire's -1 "block forever" sentinel as a cache expiry, storing the entry one second in the past and poisoning its own cache slot (@adhavan18, #2795).
• WebRTC callback shutdown deadlock removed — WebRTCSession.close() called from on_data/on_error callbacks blocked the session event loop waiting on cleanup scheduled onto that same loop; loop-initiated cleanup is now dispatched to a helper thread without blocking (@voropaevv, #2778).
• TrOCR works under transformers 5.15 — the new transformers force-routes vision-encoder-decoder checkpoints to its generic tokenizer backend, which cannot read the sentencepiece-only serialization Roboflow model packages ship; the TrOCR loader now builds its processor from the tokenizer class declared in the package instead (@PawelPeczek-Roboflow, #2798).
• pip install inference in Google Colab no longer crashes on import — Colab preinstalls a torchaudio whose CUDA build no longer matches the torch that dependency resolution installs, and transformers 5.x imports torchaudio whenever it is present; the Colab verification images drop the stale torchaudio, and Colab users hitting the crash can do the same with pip uninstall -y torchaudio (@PawelPeczek-Roboflow, #2798).
• GPU image dependency resolution fixed for torch 2.6+cu124 builds (@PawelPeczek-Roboflow, #2796).

🚧 Maintenance

🧵 transformers unpinned — one GPU build again

The dependency structure now allows transformers up to the 5.15 line across the repo and inference-models (@PawelPeczek-Roboflow, #2797, #2798). Since transformers 5.15.0 ships the NVIDIA Cosmos 3 Edge model code, the GPU image installs it straight from PyPI and the git-pin override step is gone — which retires the dual GPU build from v1.3.8/v1.3.9: there is no separate -cosmos3 tag anymore, the standard roboflow/roboflow-inference-server-gpu:1.4.1 image runs Cosmos 3 Edge out of the box.

• MQTT Writer sink gets a block namespace — roboflow_enterprise/mqtt_writer_sink@v1 is the primary identifier (it was the only block among 201 without a namespace), and the bare mqtt_writer_sink@v1 stays on as a legacy alias so existing workflows keep working (@shntu, #2783).
• Regression coverage for pipeline stream controls — pause_stream(), mute_stream() and resume_stream() are now exercised across all-sources, matching-source and unknown-source cases (@arubittu, #2790).
• PR-review skill suite updated after the tensor-native Workflows merge (@PawelPeczek-Roboflow, #2791).

🏅 New Contributors

• @arubittu made their first contribution in #2790
• @adhavan18 made their first contribution in #2795

Full Changelog: v1.4.0...v1.4.1

v1.4.0

📢 New chapter in Workflows begins now

This release introduces the biggest change to the Workflows execution stack since its inception: an experimental tensor-native execution mode that keeps video frames and model predictions on the GPU end to end, paired with hardware-accelerated video decoding. It ships disabled by default — nothing changes for existing deployments until you opt in.

⚡ Tensor-native Workflows execution (experimental) — #2357

Until now, every stage of a video Workflow — decoding, pre-processing, inference, post-processing, visualization — passed images and predictions through CPU memory as numpy arrays and sv.Detections, even when both the decoder and the model ran on the GPU. Tensor-native mode removes those round-trips: video frames are decoded straight into CUDA tensors, flow through Workflow blocks as torch tensors and native prediction objects, and only touch the CPU when something genuinely needs them there (a sink writing to disk, an HTTP response being serialized). This is an experimental mode, but it delivers promising speed-ups on GPU-equipped machines — most visibly in multi-camera InferencePipeline deployments on Jetson-class devices, where memory bandwidth is the scarcest resource.

To enable it, set one environment variable on the server or pipeline process:

docker run --rm --gpus all --network host \
  -e ENABLE_TENSOR_DATA_REPRESENTATION=True \
  roboflow/roboflow-inference-server-gpu:1.4.0

The mode is designed for and tested against InferencePipeline (video processing) — that is where the GPU-resident data path pays off. The HTTP server tolerates the flag, but video pipelines are the intended consumer:

from inference import InferencePipeline

pipeline = InferencePipeline.init_with_workflow(
    video_reference="rtsp://...",
    workspace_name="your-workspace",
    workflow_id="your-workflow",
    on_prediction=my_sink,
)
pipeline.start()

With the flag enabled, pipelines also default to a new self-tuning AUTO video-processing mode that matches the frame-collection rhythm to your cameras' actual frame rates. If you need the exact pre-1.4.0 collection behavior, pass video_processing_mode="legacy" — it is the explicit escape hatch and preserves the old semantics byte for byte.

Your Workflow definitions do not change: the same JSON runs in both modes, block names and kinds are identical, and with the flag off (the default) execution is unchanged from v1.3.x.

🎥 Hardware-accelerated video decoding

Video sources can now decode on the GPU instead of the CPU: NVDEC-backed producers land decoded frames directly in CUDA memory on both Jetson (a dedicated zero-copy tensor bridge) and x86 GPUs (GStreamer/CUDA and PyNvVideoCodec producers). Producer selection is automatic per platform and source type, with a transparent fallback to the classic OpenCV decoder whenever hardware decoding is unavailable — and the selection decision is now logged, so you can always tell which decoder served a source.

🤖 JetPack 7.2 (Thor) support

This release adds a server image for JetPack 7.2 (roboflow/roboflow-inference-server-jetson-7.2.0), bringing the full stack — including tensor-native mode and hardware decoding — to NVIDIA's newest Jetson generation.

⚠️ Breaking changes when tensor-native mode is enabled

If you do not set the flag, there are no breaking changes — flag-off behavior matches v1.3.x. When you do enable it, review the following before flipping it on a production deployment:

• In-process consumers receive tensors, not numpy. Custom on_prediction sinks and any code reading video_frame.image or prediction objects inside the pipeline process will receive torch tensors and native prediction objects instead of numpy arrays and sv.Detections. Built-in sinks were adapted; custom sinks that call OpenCV or numpy functions directly need to materialize first (helpers are provided in the codebase, e.g. for converting a tensor video frame back to numpy at the sink boundary).
• Some serialized payloads change shape. Most notably, instance segmentation masks may be serialized as compact run-length encodings (rle_mask) instead of polygon points for some block versions, and embedding/tensor outputs serialize as plain lists. If your downstream code parses serialized Workflow responses, verify it against a flag-on deployment before rolling out.
• Custom Python blocks keep working unchanged by default. Dynamic blocks declare a tensor_compatibility contract that defaults to legacy_compatibility: the engine converts native tensor objects to the documented sv.Detections/numpy representations at your block's input boundary and converts your returned values back, so existing user code runs as-is (locally and via remote execution). Blocks that want the GPU-resident data path can opt into tensor_compatibility: tensor_native and receive native objects directly (local execution only, for now).

🛠️ For contributors

Tensor-native mode changes how Workflow blocks are authored and tested, so please read this before opening a pull request that touches blocks:

• Blocks that consume or produce images/predictions now come in pairs: the classic numpy implementation (v1.py) and a tensor-native sibling (v1_tensor.py) registered in the block loader under the flag. Siblings re-implement the numpy behavior against native objects — they are deliberately standalone files, not wrappers around the numpy code. New blocks in these categories should ship both implementations; scalar/text/flow-control blocks typically need only one.
• Tests must pass in both flag directions. CI runs the Workflow suites with the flag on and off; use the established per-file _TENSOR_ONLY / _NUMPY_ONLY marker pattern for tests that only make sense in one mode, and mirror assertions across siblings.
• Manifest-level declarations (output kinds, dependent-resource discovery for model pre-loading) must be present on both siblings — a declaration only on the numpy side silently disappears under the flag.

This is a large and still-moving surface — if anything about the sibling pattern, the conversion boundary, or the testing conventions is unclear while you are contributing, please open a GitHub issue and ask. We would much rather answer questions early than review a pull request built on a wrong assumption.

📦 Side note — Cosmos 3 GPU image

This release ships a dedicated -cosmos3 variant of the GPU server image with preview support for NVIDIA Cosmos 3. The variant exists because Cosmos 3 required transformers and diffusers builds that were not officially released at the time the image was cut, so it pins pre-release snapshots of both. Official releases of those packages are now available downstream, so you can expect Cosmos support to arrive in the standard upstream GPU image in an upcoming release — at which point the dedicated variant will no longer be necessary.

Full Changelog: v1.3.10...v1.4.0

v1.3.10

Fixed

😨 Global MacOS bug with onnx models

We had problem causing all onnx models to fail on MacOS due to invalid use of utils for management of CUDA streams. Problem was patched by @voropaevv in #2771

🏅 New Contributors

• @voropaevv made their first contribution in #2771

Full Changelog: v1.3.9...v1.3.10

v1.3.9

🚀 Added

🔀 Detections Difference — a fusion block for before/after comparison

The new roboflow_core/detections_difference@v1 fusion block compares two sets of detections and reports what changed: which objects disappeared, which persisted, and which are new. Matching is configurable (IoU threshold plus optional class agreement), which makes the block a natural building brick for tamper detection, shelf-restock monitoring, or any workflow that needs to reason about the delta between a reference frame and the current one instead of raw per-frame predictions (@adawgwats, #2765).

📦 Write Vision Event Bundle sink

The new roboflow_core/vision_event_bundle@v1 sink block packages predictions, annotated frames and event metadata into a single portable bundle archive, with cooldown control so bundles are emitted at a sensible rate rather than per frame. This gives Workflows a first-class way to hand a complete, self-describing evidence package to downstream systems instead of loose files (@rvirani1, #2761).

🤖 VLM lineup: Qwen 3.8 Max and per-model detection prompts for open_ai@v5

The Qwen VLM block family gains Qwen 3.8 Max served via OpenRouter (@Erol444, #2755). Alongside it, open_ai@v5 learns per-model detection prompt styles — each OpenAI model family gets the prompt format it detects best with, including absolute-pixel box_2d output — and vlm_as_detector@v2 ships the matching parser so those responses turn into first-class detections, correctly rescaled from the downsized upload back to the original image (@SkalskiP, #2750).

🦴 rfdetr-keypoint-preview alias

The COCO-pretrained RF-DETR keypoint model is now reachable under the rfdetr-keypoint-preview alias, resolving like the other pretrained aliases (yolo26n-pose-640 and friends) in both inference and the SDK (@mkaic, #2758).

🔐 Hardened RTSPS streaming

Two more installments of the stream-security series land in this release. The GStreamer RTSPS path now supports a custom CA bundle and explicit TLS validation flags, so self-hosted deployments with private certificate authorities can validate camera connections properly instead of loosening security to connect (@NVergunst-ROBO, #2726). And camera source references are sanitized at the inference pipeline boundary — RTSP credentials embedded in stream URLs no longer leak into logs, status events or error messages, while error classification still runs on the raw text so redaction never changes what kind of error gets reported (@NVergunst-ROBO, #2683).

⚡ Faster semantic segmentation post-processing

Two performance PRs cut the cost of the semantic segmentation workflow path: model responses can now carry the class map as an in-process numpy array instead of a base64-encoded PNG when model and workflow run in the same process (@theo-roboflow, #2729), and post-processing uses a present_class_ids hint plus a single F-order conversion so per-class mask extraction stops re-scanning and re-converting the full-resolution class map for every class (@theo-roboflow, #2728).

🔧 Fixed

• Dataset Upload no longer cuts instance masks when downscaling images — annotation scaling now uses separate horizontal and vertical factors matched to the exact stored JPEG dimensions, and dense masks are resized directly instead of round-tripping through polygons, so boxes and masks stay on-canvas for non-uniform resizes (@SolomonLake, #2738).
• Usage tracking can no longer fail an inference call — telemetry errors are contained instead of propagating into the request path (@arthi-arumugam-git, #2745).
• Model usage attribution fixed for cached SAM2/SAM3 requests (@hansent, #2751) and fine-tuned SAM 3 Serverless failures now return actionable error messages (@hansent, #2756).
• OPC UA sessions are released instead of orphaned — the OPC UA writer sink now closes sessions on the server when done, preventing session exhaustion on industrial endpoints (@rvirani1, #2763).
• Weights proxy URL builder aligned with wrap_url — proxy-prefixed weight downloads now compose base paths correctly and idempotently (@rs-03, #2747).
• Removed a deadlock in the RTSPS TLS serialization test that hung the UNIT TESTS - inference job until its 15-minute timeout on every CI run (@PawelPeczek-Roboflow, #2749).

🚧 Maintenance

• supervision pinned <0.30.0 — supervision 0.30.0 changed detections-ingestion behavior (invalid polygons are no longer skipped) and added per-detection validation on the data container, both of which break current inference behavior and CI; the pin holds until the code is adapted deliberately (@Erol444, #2760).
• fastapi upper bound relaxed to <0.129, unblocking installs alongside newer FastAPI (@saikrishna01301, #2741).
• Security dependency patch, August 2026 edition (@PawelPeczek-Roboflow, #2766).
• Docs moved home: inference.roboflow.com now redirects to docs.roboflow.com (@Erol444, #2754).

---

📦 Side note: dual GPU build for Cosmos 3

There is still no released transformers version that ships the NVIDIA Cosmos 3 model code, so this release again publishes two GPU server builds:

• roboflow/roboflow-inference-server-gpu:1.3.9 — the standard build, with the regular dependency stack (no Cosmos 3).

• roboflow/roboflow-inference-server-gpu:1.3.9-cosmos3 — identical server, but with the custom transformers dependency set required by NVIDIA Cosmos 3 Edge:

  docker pull roboflow/roboflow-inference-server-gpu:1.3.9-cosmos3

Use the -cosmos3 tag only if you need the Cosmos 3 Edge preview self-hosted; all other images are single-build.

---

🏅 New Contributors

• @theo-roboflow made their first contribution in #2728
• @saikrishna01301 made their first contribution in #2741
• @rs-03 made their first contribution in #2747
• @adawgwats made their first contribution in #2765
• @arthi-arumugam-git made their first contribution in #2745

Full Changelog: v1.3.8...v1.3.9

v1.3.8

🚀 Added

⚡ Opt-in model pre-loading for Workflows in InferencePipeline

Video processing deserves predictable startup — until now, every model used by a Workflow was loaded lazily on the first frame, so the pipeline connected to the stream and then stalled while weights downloaded. InferencePipeline.init_with_workflow(...) gains an opt-in parameter that pre-loads all Roboflow models declared by the workflow's blocks at pipeline init, before a single frame is processed (@PawelPeczek-Roboflow, #2737):

pipeline = InferencePipeline.init_with_workflow(
    video_reference="rtsp://...",
    workflow_specification=workflow,
    workflows_dependencies_pre_init=["roboflow_platform_model"],
    on_prediction=my_sink,
)

• Concrete model ids (like yolov8n-640 in the spec) register in the model manager at init — weights are fetched upfront, first-frame latency stays flat.
• Input-fed model ids (model_id: "$inputs.model") resolve once, on the first frame, when runtime parameters are known — including ids that blocks synthesize from version fields (e.g. clip/<version>).
• Pre-loading honors the effective step execution mode — nothing is fetched when steps execute remotely — and if a size-bounded model manager evicts a pre-loaded model, you get a warning instead of a silent cold start.
• Everything defaults to off: without the parameter, behavior is exactly as before.

The same knob is available on ExecutionEngine.init(..., dependencies_pre_init=...) for anyone embedding the Workflows Execution Engine directly. Under the hood, Workflow blocks can now declare their dependent resources (Roboflow models, Roboflow projects, third-party hosted models) through a typed, serializable contract, and the compiler can deduce the full dependency set of a compiled workflow — groundwork that pre-loading is the first consumer of. Execution Engine version goes to v1.14.0; see the Execution Engine changelog and the block creation docs for the block-author contract.

🏷️ Rich Label visualization + Label v2 with adaptive text sizing

Detection labels finally look good: the new Rich Label visualization block renders sharp, anti-aliased text using TrueType fonts instead of OpenCV's dated bitmap fonts, with a dropdown of 20 approved fonts (downloaded from pinned URLs and verified against SHA-256 checksums). Alongside it, Label v2 adds an Automatic text-sizing mode that scales label text to the image resolution, keeping labels readable from thumbnails up to 4K (@SkalskiP, #2722).

🤖 Gemini v4 with native object detection coordinates

The new google_gemini@v4 Workflow block switches to Gemini-native box_2d object detection output and enforces a JSON output schema, eliminating the malformed-response parsing failures seen in v3. Across every tested Gemini model, v4 increased mAP@50 while reducing average token usage and inference time per image; the supported model catalog is expanded and v3 behavior is preserved for existing workflows (@SkalskiP, #2734).

🌍 Uniform region & environment selection

One switch selects the region, one the environment — and inference, the CLI, the SDK, and inference-models all resolve their default hosts from the same registry (@imbgar-roboflow, #2701):

ROBOFLOW_REGION=eu inference ...                              # api.roboflow.eu
ROBOFLOW_REGION=eu ROBOFLOW_ENVIRONMENT=staging inference ... # api.roboflow-eu.one

The scattered per-file host ternaries are gone; inference_sdk/regions.py is the single source of truth for the region × environment matrix.

🎛️ Execution & operations

• RF-DETR object detection now uses the full execution plan, aligning it with the rest of the inference_models execution stack (@dkosowski87, #2731; follow-up test fixes in #2733).
• RTSPS streams fall back to OpenCV/FFmpeg TLS when the primary path cannot negotiate the transport (@NVergunst-ROBO, #2727).
• Structured stream error codes for RTSPS and auth failures — stream connection problems now surface as typed, actionable errors instead of generic failures (@NVergunst-ROBO, #2725).

🔧 Fixed

• Workflow output serialization no longer 500s on string-declared kinds — output kinds declared as plain strings (e.g. "string") crashed serialization with TypeError: unhashable type: 'list'; such kinds are now resolved by name and the matching serializer applied (@rafel-roboflow, #2730).
• Profiling handles read-only filesystems — an OSError when dumping profiler traces on read-only deployments is caught instead of failing the run (@iamfaham, #2732).
• SAM3 usage tracking fixed (@grzegorz-roboflow, #2720).

🚧 Maintenance

• Security: setuptools>=83.0.0 dependency floor (@PawelPeczek-Roboflow, #2735).

---

📦 Side note: dual GPU build for Cosmos 3

There is still no released transformers version that ships the NVIDIA Cosmos 3 model code, so this release again publishes two GPU server builds:

• roboflow/roboflow-inference-server-gpu:1.3.8 — the standard build, with the regular dependency stack (no Cosmos 3).

• roboflow/roboflow-inference-server-gpu:1.3.8-cosmos3 — identical server, but with the custom transformers dependency set required by NVIDIA Cosmos 3 Edge:

  docker pull roboflow/roboflow-inference-server-gpu:1.3.8-cosmos3

Use the -cosmos3 tag only if you need the Cosmos 3 Edge preview self-hosted; all other images are single-build.

---

Full changelog: v1.3.7...v1.3.8

🏅 New Contributors

• @bczifra made their first contribution in #2698
• @iamfaham made their first contribution in #2732

Full Changelog: v1.3.7...v1.3.8

v1.3.7

🚀 Added

✈️ OFFLINE_MODE — air-gapped deployments

Inference servers can now run fully air-gapped (@alexnorell, #2263). The supported flow is deliberately simple:

1. Warm a mounted cache while network access and a Roboflow API key are available.
2. Restart the same deployment with the same cache, OFFLINE_MODE=True, and no API key.
3. Cached model metadata, weights, and Workflow specifications load with no built-in Roboflow API calls, no retries, and no cache-expiration failures.

📏 YOLO26 depth estimation

Public-pretrained YOLO26 depth estimation lands end-to-end as a drop-in alternative to Depth Anything (@leeclemnet,
#2691): inference_models backends for ONNX, TorchScript, and TensorRT (Ultralytics -depth pretrains, 768×768, metric log-depth head), public aliases yolo26{n,s,m,l,x}-depth-768, and the same five variants selectable in the roboflow_core/depth_estimation@v1 Workflow block. Outputs are normalized to Depth Anything's disparity-style convention (larger = closer), so downstream tooling is interchangeable; the block's default model is unchanged.

Alongside it, depth maps got dramatically cheaper on the wire (@leeclemnet, #2693): the SDK now requests the normalized depth map as a base64 PNG16 (uint16 quantization of the normalized map) instead of a JSON list of floats — roughly 17 MB → ~1 MB for a typical single-image response, with matching decode-time wins.
Raw HTTP callers are unaffected (the server-side default response format is still json), and the SDK decoder transparently accepts responses from older servers.

🔥 New workflow blocks

• Detections Nearest Neighbor (roboflow_core/detections_nearest_neighbor@v1) — a nearest-neighbor spatial join between two detection sets: for each query detection it finds the closest target detection(s) by 2D pixel distance between configurable anchor points (bbox corners/edges/center, or a named keypoint), with tie-awareness, self-match exclusion, and an optional max_distance cutoff (@bczifra, #2698). Emits enriched query predictions (nearest_target_distance) plus index-aligned matched-query/matched-target sets ready for standard downstream blocks.
• Grid Visualization now accepts multiple image inputs directly (@leeclemnet, #2694).

🎛️ Execution & operations

• Per-run workflow sink disabling — a disable_sinks API parameter, delivered to sink blocks through the existing dependency-injection system, turns off built-in sink side effects for a single run while the rest of the workflow executes normally; all 22 built-in side-effect sinks honor it with no manifest or spec changes (@joaomarcoscrs, #2697).
• TensorRT engine builds announce themselves — with the TRT execution provider and a cold engine cache, the first inference silently compiles an engine (measured ~9.5 minutes for a single COCO object-detection model on an Orin 16 GB) while requests time out. The server now logs an explicit warning at session-configuration time, pointing at the mitigation: persist MODEL_CACHE_DIR (@sberan, #2703).
• Billable and errored usage are aggregated separately — usage records now partition by billability, request outcome, structured error_type, and bounded HTTP error_status_code (400–599), so a failed request can no longer blend into a billable aggregate; billing intent is preserved and policy stays with the Roboflow backend. Payload schema is additive-only (@hansent, #2692).
• Roboflow API calls now default to a 120-second timeout (ROBOFLOW_API_REQUEST_TIMEOUT) instead of waiting indefinitely (@sberan, #2702).

🔧 Fixed

• Instance-segmentation dense-mask post-processing memory bounded — mask upscaling now runs in fixed-size chunks and RF-DETR segmentation applies the standard max_detections cap (default 300, overridable via INFERENCE_MODELS_RFDETR_DEFAULT_MAX_DETECTIONS) before the expensive full-resolution work. Peak CUDA memory on a dense-instance workload dropped 17.1 GiB → 4.2 GiB with no wall-time regression (@bigbitbus, #2682).
• Model-access failures keep their real HTTP statuses — Execution Engine v1.12.1 propagates upstream model-access errors (401/402/403/404…) through workflow runs instead of collapsing them into generic 500s (@hansent, #2690; @dkosowski87, #2709).
• PP-OCR — authorization fixed on Serverless (@Erol444, #2646), and the route now returns a clean 404 when the inference_models stack is disabled instead of an internal error (@dkosowski87, #2707).
• Phantom keypoints eliminated — padded keypoint slots are no longer emitted as real keypoints in predictions (@kounelisagis,
  #2677).
• detections_overlaps kind round-trips — serializer and deserializer registered, so the kind survives workflow JSON output/input (@kounelisagis, #2638).
• Grounding DINO accepts both canonical and legacy BERT cache layouts, so existing warmed caches keep working after the cache-layout changes (@dkosowski87, #2716).
• SAM3 package load no longer requires sam_configuration.json (@grzegorz-roboflow, #2678).
• Local package imports no longer write .pyc bytecode during module execution, keeping mounted/read-only caches byte-stable (@dkosowski87, #2713).
• Pre-release security patches — landing-page npm dependency chain moved to patched versions and Python dependency bumps rolled up ahead of the release (@PawelPeczek-Roboflow, #2715).

🚧 Maintenance

• inference-models 0.33.0 / 0.34.1 releases and pins (@PawelPeczek-Roboflow, #2706; @dkosowski87, #2711).
• Windows build fixes (@PawelPeczek-Roboflow, #2689).
• CI — PP-OCR T4 regression tests skipped when USE_INFERENCE_MODELS=False (@grzegorz-roboflow, #2712) and the workflows integration-test job timeout raised to 25 minutes (@grzegorz-roboflow, #2719).
• README contributing-guide link fixed (@bczifra, #2704).

---

📦 Side note: dual GPU build for Cosmos 3

There is still no released transformers version that ships the NVIDIA Cosmos 3 model code, so this release again publishes two GPU server builds:

• roboflow/roboflow-inference-server-gpu:1.3.7 — the standard build, with the regular dependency stack (no Cosmos 3).

• roboflow/roboflow-inference-server-gpu:1.3.7-cosmos3 — identical server, but with the custom transformers dependency set required by NVIDIA Cosmos 3 Edge:

  docker pull roboflow/roboflow-inference-server-gpu:1.3.7-cosmos3

Use the -cosmos3 tag only if you need the Cosmos 3 Edge preview self-hosted; all other images are single-build.

---

Full changelog: v1.3.6...v1.3.7

v1.3.6

🚀 Added

🧠 NVIDIA Cosmos 3 Edge — initial preview

This release ships an initial preview of the NVIDIA Cosmos 3 Edge model family (#2675):

• Reasoning stack in Workflows. The new roboflow_core/cosmos3_edge@v1 block exposes the Cosmos 3 Edge reasoner (VLM) in Workflows — single-image and multi-frame reasoning emitting language_model_output, so it chains directly into the existing VLM tooling (e.g. VLM as Detector). The block is GPU-gated.

• Available on the Roboflow platform.

• Standalone GPU build available as a trial:

  docker pull roboflow/roboflow-inference-server-gpu:1.3.6-cosmos3

The standard release images ship without the Cosmos 3 dependency stack — the trial build above is the way to run it self-hosted during the preview. Under the hood the preview also lands the Cosmos 3 Edge world-model surface (image-to-video generation, forward/inverse dynamics) in inference_models, ahead of an HTTP/Workflows surface for generative outputs in a future release.

⚡ RF-DETR object detection — TensorRT pre/post-processing speed-up

RF-DETR object detection on the TensorRT backend now runs GPU-accelerated (Triton-kernel) image pre-processing and fused post-processing, removing the CPU bottleneck around the TensorRT forward pass.

Measured end-to-end latency (Orin AGX, JetPack 6.2, TRT fp16 package):

Scenario	before (mean)	after (mean)	speed-up
3840×2160, batch 4	436.8 ms	36.0 ms	~12×
640×480, batch 1	17.3 ms	7.4 ms	~2.3×

Where it applies:

• GPU builds and Jetson JetPack 6+. CPU builds and older JetPacks (JetPack 5 and below) keep the existing pipeline, unchanged.
• Selected pre-processing pathways only. The accelerated path engages for model packages using plain stretch resize with standard 3-channel input and per-channel normalization. Models configured with other resize modes (letterbox / fit), dataset-version resize dimensions, or additional image transforms (static crop, contrast, grayscale, auto-orient) automatically and transparently fall back to the existing base implementation — no behavior change for those models.

Selection is automatic with conservative compatibility checks and produces identical predictions to the base pipeline. For explicit control (pinning or disabling per deployment) use the environment variables INFERENCE_MODELS_RFDETR_PREPROCESSOR and INFERENCE_MODELS_RFDETR_POSTPROCESSOR (values: base, threaded-exact-v1, triton-universal-v1 / triton-fused-v1).

🔥 New workflow blocks

• Auto Rotate on Edges (roboflow_core/auto_rotate_on_edges@v1) — rotates an image so its dominant straight lines become vertical, horizontal, or the nearest axis (@jeku46, #2655). Built for line-dominated inputs that arrive skewed — industrial X-rays, documents, labels, shelves — with a single-pass gradient-histogram estimate, sub-degree refinement, and identity-passthrough guards for flat or orientation-ambiguous images. Outputs the rotated image and the applied angle.
• Frame Delay (roboflow_core/frame_delay@v1) — returns any workflow value (detections, numbers, images, …) as it was |offset| frames ago on the same video stream, enabling cross-frame comparison and temporal alignment (@rafel-roboflow, #2668). Memory-bounded per-stream ring buffer; past-only by design; works in every execution context including the WebRTC video path.

🔭 Observability

• CUDA allocator memory breakdown on /model/registry — live tensor allocations vs. PyTorch-reserved vs. allocator cache vs. non-PyTorch device memory, so production incidents can tell allocator growth from real model memory (@hansent, #2657).
• Per-pipeline stream session id in usage tracking — each InferencePipeline now carries a stable stream session identity (callers may supply their own, e.g. DEVICE_ID:stream_name), so concurrent pipelines under one API key and workflow no longer merge in usage aggregation (@sberan, #2634).

🔧 Fixed

• TrOCR now served through the inference_models adapter by default — the original HuggingFace package path hit tokenizer-compatibility issues with newer transformers; the adapter implementation is compatible and is now the default (@PawelPeczek-Roboflow, #2681).
• SAM3 concept-path post-processing memory bounded — the upstream post-processor interpolated every kept mask to full resolution in one batch and applied the detection cap last, spiking multi-GiB host/GPU peaks on large images (a direct contributor to a serverless OOM crash loop). The pipeline is now chunked and cap-first, bounding peak memory regardless of instance count (@bigbitbus, #2670).
• OWLv2 embeddings cache fixes — re-signed image URLs no longer defeat the cache (#2659), and the cache is consulted before reference images are materialized, skipping redundant downloads entirely (@bigbitbus, #2660).
• Secure-gateway routing audit — every outbound HTTP call in inference/ and inference_models/ was audited for SECURE_GATEWAY compatibility; three gaps fixed, including the GitHub version check stalling server startup behind a gateway (now timeout-bounded and auto-disabled) and workflow remote step execution dead-ending against hosted endpoints (@alexnorell,
  #2658).
• Inner-workflow validation errors return HTTP 400, not 500 — compile-time child-workflow problems (stale parameter bindings, invalid nesting, cycles) are now correctly classified as client errors in both sync and async route handlers (@dkosowski87, #2645).
• Pre-release security patches rolled up ahead of the release (@PawelPeczek-Roboflow, #2688).

🚧 Maintenance

• JetPack 7.2 build workflow — groundwork for upcoming JetPack 7.2 server images (@alexnorell, #2654).
• RF-DETR server integration tests — all RF-DETR model aliases (detection + segmentation) now covered against both legacy and v1 server endpoints, in both USE_INFERENCE_MODELS modes (@PawelPeczek-Roboflow, #2673).
• New unit-test CI workflow for the model manager and inference server (@grzegorz-roboflow, #2669), and right-sized CI runners for the dev-test workflow (@iurisilvio, #2665).

---

Full changelog: v1.3.5...v1.3.6