v1.3.6

🚀 Added

🧠 NVIDIA Cosmos 3 Edge — initial preview

This release ships an initial preview of the NVIDIA Cosmos 3 Edge model family (#2675):

• Reasoning stack in Workflows. The new roboflow_core/cosmos3_edge@v1 block exposes the Cosmos 3 Edge reasoner (VLM) in Workflows — single-image and multi-frame reasoning emitting language_model_output, so it chains directly into the existing VLM tooling (e.g. VLM as Detector). The block is GPU-gated.

• Available on the Roboflow platform.

• Standalone GPU build available as a trial:

  docker pull roboflow/roboflow-inference-server-gpu:1.3.6-cosmos3

The standard release images ship without the Cosmos 3 dependency stack — the trial build above is the way to run it self-hosted during the preview. Under the hood the preview also lands the Cosmos 3 Edge world-model surface (image-to-video generation, forward/inverse dynamics) in inference_models, ahead of an HTTP/Workflows surface for generative outputs in a future release.

⚡ RF-DETR object detection — TensorRT pre/post-processing speed-up

RF-DETR object detection on the TensorRT backend now runs GPU-accelerated (Triton-kernel) image pre-processing and fused post-processing, removing the CPU bottleneck around the TensorRT forward pass.

Measured end-to-end latency (Orin AGX, JetPack 6.2, TRT fp16 package):

Scenario	before (mean)	after (mean)	speed-up
3840×2160, batch 4	436.8 ms	36.0 ms	~12×
640×480, batch 1	17.3 ms	7.4 ms	~2.3×

Where it applies:

• GPU builds and Jetson JetPack 6+. CPU builds and older JetPacks (JetPack 5 and below) keep the existing pipeline, unchanged.
• Selected pre-processing pathways only. The accelerated path engages for model packages using plain stretch resize with standard 3-channel input and per-channel normalization. Models configured with other resize modes (letterbox / fit), dataset-version resize dimensions, or additional image transforms (static crop, contrast, grayscale, auto-orient) automatically and transparently fall back to the existing base implementation — no behavior change for those models.

Selection is automatic with conservative compatibility checks and produces identical predictions to the base pipeline. For explicit control (pinning or disabling per deployment) use the environment variables INFERENCE_MODELS_RFDETR_PREPROCESSOR and INFERENCE_MODELS_RFDETR_POSTPROCESSOR (values: base, threaded-exact-v1, triton-universal-v1 / triton-fused-v1).

🔥 New workflow blocks

• Auto Rotate on Edges (roboflow_core/auto_rotate_on_edges@v1) — rotates an image so its dominant straight lines become vertical, horizontal, or the nearest axis (@jeku46, #2655). Built for line-dominated inputs that arrive skewed — industrial X-rays, documents, labels, shelves — with a single-pass gradient-histogram estimate, sub-degree refinement, and identity-passthrough guards for flat or orientation-ambiguous images. Outputs the rotated image and the applied angle.
• Frame Delay (roboflow_core/frame_delay@v1) — returns any workflow value (detections, numbers, images, …) as it was |offset| frames ago on the same video stream, enabling cross-frame comparison and temporal alignment (@rafel-roboflow, #2668). Memory-bounded per-stream ring buffer; past-only by design; works in every execution context including the WebRTC video path.

🔭 Observability

• CUDA allocator memory breakdown on /model/registry — live tensor allocations vs. PyTorch-reserved vs. allocator cache vs. non-PyTorch device memory, so production incidents can tell allocator growth from real model memory (@hansent, #2657).
• Per-pipeline stream session id in usage tracking — each InferencePipeline now carries a stable stream session identity (callers may supply their own, e.g. DEVICE_ID:stream_name), so concurrent pipelines under one API key and workflow no longer merge in usage aggregation (@sberan, #2634).

🔧 Fixed

• TrOCR now served through the inference_models adapter by default — the original HuggingFace package path hit tokenizer-compatibility issues with newer transformers; the adapter implementation is compatible and is now the default (@PawelPeczek-Roboflow, #2681).
• SAM3 concept-path post-processing memory bounded — the upstream post-processor interpolated every kept mask to full resolution in one batch and applied the detection cap last, spiking multi-GiB host/GPU peaks on large images (a direct contributor to a serverless OOM crash loop). The pipeline is now chunked and cap-first, bounding peak memory regardless of instance count (@bigbitbus, #2670).
• OWLv2 embeddings cache fixes — re-signed image URLs no longer defeat the cache (#2659), and the cache is consulted before reference images are materialized, skipping redundant downloads entirely (@bigbitbus, #2660).
• Secure-gateway routing audit — every outbound HTTP call in inference/ and inference_models/ was audited for SECURE_GATEWAY compatibility; three gaps fixed, including the GitHub version check stalling server startup behind a gateway (now timeout-bounded and auto-disabled) and workflow remote step execution dead-ending against hosted endpoints (@alexnorell,
  #2658).
• Inner-workflow validation errors return HTTP 400, not 500 — compile-time child-workflow problems (stale parameter bindings, invalid nesting, cycles) are now correctly classified as client errors in both sync and async route handlers (@dkosowski87, #2645).
• Pre-release security patches rolled up ahead of the release (@PawelPeczek-Roboflow, #2688).

🚧 Maintenance

• JetPack 7.2 build workflow — groundwork for upcoming JetPack 7.2 server images (@alexnorell, #2654).
• RF-DETR server integration tests — all RF-DETR model aliases (detection + segmentation) now covered against both legacy and v1 server endpoints, in both USE_INFERENCE_MODELS modes (@PawelPeczek-Roboflow, #2673).
• New unit-test CI workflow for the model manager and inference server (@grzegorz-roboflow, #2669), and right-sized CI runners for the dev-test workflow (@iurisilvio, #2665).

---

Full changelog: v1.3.5...v1.3.6