Skip to content

Release Notes: Intel® AI for Enterprise Agent Toolkit – v1.1.0

Latest
Latest

Choose a tag to compare

View all tags
@AhmedSeemalK AhmedSeemalK released this 22 Oct 17:16
· 1 commit to main since this release
release-1.1.0
0c726f5
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Intel® AI for Enterprise Agent Toolkit — v1.1.0

Highlights

  • Rack-scale deployment — a new deployment tier for multi-rack clusters with 100G fabric networking and NVIDIA GPU (B300 HGX) support
  • Smarter inference routing — a new Semantic Router and a Hybrid/Efficiency Router for content-aware, intelligent request routing
  • Content-safety Guardrails — a pluggable safety layer on the inference path
  • Broader hardware and runtime support — Intel Arc Battlemage (BMG) XPU as a compute platform support
  • Stronger workload isolation — Kata Containers (Firecracker/QEMU) microVM sandboxing and an NVIDIA OpenShell sandbox provider
  • More accurate resource sizing — CPU and tensor-parallel allocation for inference workloads is now derived per model and per node topology, instead of a generic cluster-wide estimate
  • Operational safety nets — preflight and post-deploy health validation, and a full container/IaC security scan in CI

✨ New Features

Deployment & Infrastructure

  • Rack-scale deployment tier: 100G networking with BGP/ECMP routing, Clos and hub-and-spoke topologies, multi-master control plane, and optional Ceph-backed storage, driven by a declarative rack profile.
  • Topology-aware CPU sizing: CPU and memory allocation for inference workloads is now computed per node and per model based on real NUMA topology, replacing a single generic allocation used previously.

Inference & Models

  • vLLM Semantic Router: optional, content-aware routing of inference requests to the most appropriate backend.
  • Hybrid/Efficiency Router: an optional intelligent routing strategy in the GenAI Gateway that selects between backends based on observed performance.
  • New CPU model support: Qwen/Qwen3.6-35B-A3B model supported by the toolkit.
  • vLLM upgraded to v0.28.0 across all CPU, GPU, and XPU deployment paths.
  • OpenVINO Model Server (OVMS) as a selectable serving engine for Docker deployments — vllm remains the default; switching requires no client-side changes.
  • Intel Arc Battlemage (BMG) XPU support: a third compute platform option alongside CPU and GPU, with six selectable models.

Safety & Security

  • Content-safety Guardrails: an optional, model-agnostic safety guardrail that can be registered into the gateway without redeploying it, and now receives the same topology-aware sizing as served models.

Agent Sandbox

  • Kata Containers microVM sandboxing (Firecracker or QEMU) for stronger, VM-level workload isolation.
  • NVIDIA OpenShell sandbox provider (optional): default-deny network egress, fine-grained policy enforcement, and secure credential handling for sandboxed agent workloads. Includes support for running OpenCode through the GenAI Gateway inside an OpenShell sandbox.

Operations & CI

  • Preflight and post-deploy health validation: automated checks of cluster and node health (kernel, kubelet, CPU/NUMA configuration, and more), with a readable health report and remediation guidance.

🔒 Security Hardening

  • Disabled third-party usage telemetry by default in the bundled Langfuse component.
  • OpenShell sandboxes: the gateway's master access key never leaves the gateway, and per-session credentials never enter the sandboxed workload.
  • Hardened health-check automation to avoid logging sensitive values.

⬆️ Behavior Changes

New optional configuration settings

Setting Purpose Default
rack_scale_mode Enable rack-scale deployment tier Off
deploy_vllm_sr Enable the vLLM Semantic Router Off
deploy_guardrails Enable content-safety Guardrails Off
deploy_ceph Enable Ceph-backed storage Off
air_gapped Enable air-gapped deployment mode Off
deploy_kata_containers / kata_runtime Enable Kata Containers sandboxing (kata-fc or kata-qemu) Off
deploy_openshell Enable the NVIDIA OpenShell sandbox provider Off
serving_engine (Docker only) Choose vllm or openvino as the model-serving engine vllm
compute_platform Now accepts a third value, xpu, alongside cpu and gpu —

⚠️ Known Limitations

Note

In this release: the vLLM Semantic Router, Kata Containers/Firecracker sandboxing, the OpenShell sandbox provider, the GenAI Gateway policy plane, NUMA-aware CPU sizing, Test coverage across topologies (multi-node clusters, proxy environments) is still being expanded for these features. Expect continued fixes and hardening in upcoming releases.

License

Licensed under the Apache License 2.0.

Assets 2
Loading