Repository navigation
Releases: intel/enterprise-agent-toolkit
Release list
Release Notes: Intel® AI for Enterprise Agent Toolkit – v1.1.0
Intel® AI for Enterprise Agent Toolkit — v1.1.0
Highlights
- Rack-scale deployment — a new deployment tier for multi-rack clusters with 100G fabric networking and NVIDIA GPU (B300 HGX) support
- Smarter inference routing — a new Semantic Router and a Hybrid/Efficiency Router for content-aware, intelligent request routing
- Content-safety Guardrails — a pluggable safety layer on the inference path
- Broader hardware and runtime support — Intel Arc Battlemage (BMG) XPU as a compute platform support
- Stronger workload isolation — Kata Containers (Firecracker/QEMU) microVM sandboxing and an NVIDIA OpenShell sandbox provider
- More accurate resource sizing — CPU and tensor-parallel allocation for inference workloads is now derived per model and per node topology, instead of a generic cluster-wide estimate
- Operational safety nets — preflight and post-deploy health validation, and a full container/IaC security scan in CI
✨ New Features
Deployment & Infrastructure
- Rack-scale deployment tier: 100G networking with BGP/ECMP routing, Clos and hub-and-spoke topologies, multi-master control plane, and optional Ceph-backed storage, driven by a declarative rack profile.
- Topology-aware CPU sizing: CPU and memory allocation for inference workloads is now computed per node and per model based on real NUMA topology, replacing a single generic allocation used previously.
Inference & Models
- vLLM Semantic Router: optional, content-aware routing of inference requests to the most appropriate backend.
- Hybrid/Efficiency Router: an optional intelligent routing strategy in the GenAI Gateway that selects between backends based on observed performance.
- New CPU model support: Qwen/Qwen3.6-35B-A3B model supported by the toolkit.
- vLLM upgraded to v0.28.0 across all CPU, GPU, and XPU deployment paths.
- OpenVINO Model Server (OVMS) as a selectable serving engine for Docker deployments —
vllmremains the default; switching requires no client-side changes. - Intel Arc Battlemage (BMG) XPU support: a third compute platform option alongside CPU and GPU, with six selectable models.
Safety & Security
- Content-safety Guardrails: an optional, model-agnostic safety guardrail that can be registered into the gateway without redeploying it, and now receives the same topology-aware sizing as served models.
Agent Sandbox
- Kata Containers microVM sandboxing (Firecracker or QEMU) for stronger, VM-level workload isolation.
- NVIDIA OpenShell sandbox provider (optional): default-deny network egress, fine-grained policy enforcement, and secure credential handling for sandboxed agent workloads. Includes support for running OpenCode through the GenAI Gateway inside an OpenShell sandbox.
Operations & CI
- Preflight and post-deploy health validation: automated checks of cluster and node health (kernel, kubelet, CPU/NUMA configuration, and more), with a readable health report and remediation guidance.
🔒 Security Hardening
- Disabled third-party usage telemetry by default in the bundled Langfuse component.
- OpenShell sandboxes: the gateway's master access key never leaves the gateway, and per-session credentials never enter the sandboxed workload.
- Hardened health-check automation to avoid logging sensitive values.
⬆️ Behavior Changes
New optional configuration settings
| Setting | Purpose | Default |
|---|---|---|
rack_scale_mode |
Enable rack-scale deployment tier | Off |
deploy_vllm_sr |
Enable the vLLM Semantic Router | Off |
deploy_guardrails |
Enable content-safety Guardrails | Off |
deploy_ceph |
Enable Ceph-backed storage | Off |
air_gapped |
Enable air-gapped deployment mode | Off |
deploy_kata_containers / kata_runtime |
Enable Kata Containers sandboxing (kata-fc or kata-qemu) |
Off |
deploy_openshell |
Enable the NVIDIA OpenShell sandbox provider | Off |
serving_engine (Docker only) |
Choose vllm or openvino as the model-serving engine |
vllm |
compute_platform |
Now accepts a third value, xpu, alongside cpu and gpu |
— |
⚠️ Known Limitations
Note
In this release: the vLLM Semantic Router, Kata Containers/Firecracker sandboxing, the OpenShell sandbox provider, the GenAI Gateway policy plane, NUMA-aware CPU sizing, Test coverage across topologies (multi-node clusters, proxy environments) is still being expanded for these features. Expect continued fixes and hardening in upcoming releases.
License
Licensed under the Apache License 2.0.
Release 1.0.3 Hotfix Latest
Summary
Hardens the first-run setup of the optional Agentic AI (Flowise) component on both deployment paths, and tightens a few related defaults. No change to how the platform is used: the site is published at the same address, and the administrator account is still created by hand on the Flowise Setup Account page.
Flowise 3.1.2 bootstraps its first organization owner through an unauthenticated POST /api/v1/account/register. Only the first caller to reach that endpoint can ever become the owner — every later attempt is refused with "You can only have one organization" — so that endpoint should not be served to a wider network while the database is still empty. These changes narrow who can reach it, and leave everything else about the deployment unchanged.
Changes
Kubernetes (plugins/agenticai/) — the register path is served from its own source-restricted Ingress (flowise-account-setup) controlled by agenticai_setup_source_ranges, defaulting to admitting nothing from the network (setup happens once over kubectl port-forward). The rule matches with a case-insensitive regex rather than pathType: Exact, because Express routes case-insensitively and non-strictly. The restricted rule is created before the root Ingress so there's no unguarded window. Optional agenticai_lock_registration drops the endpoint from the public whitelist once an owner exists. Sets SECURE_COOKIES/APP_URL, passed with --set-string so Helm doesn't type-infer them. Both knobs live in the plugin vars file — nothing was added to core/inventory/agentic-config.cfg.
Docker Compose (docker/) — Flowise publishes on loopback by default (FLOWISE_BIND_ADDR); deploy.sh detects an existing owner on later runs and sets DENYLIST_URLS automatically; NODE_ENV=development removed (Flowise reads it in exactly two places — stack traces at development, forced Secure cookies at production — so unset avoids both); unused FLOWISE_ADMIN_* generation dropped from the vault script; deploy_agenticai_plugin defaults to off.
Docs — quickstart and docker/README.md cover the first-run steps per path, opt-in browser setup, and 403/401 troubleshooting.
Verification
Exercised against a live single-node cluster and a separate Docker host. On Kubernetes: all 12 register path variants return 403 from a non-loopback source (trailing slash, mixed case, duplicate slashes, query string, /./ and /../, percent-encoded %72egister), while /, ping, version, logout, login, chatflows, forgot-password and sibling paths like …/registerfoo are unaffected; port-forward setup works; annotation semantics confirmed both ways on throwaway Ingresses; and an audit of the running image confirms the register route is the only unauthenticated path that creates an organization. On Docker: Flowise unreachable from another host while 127.0.0.1:3000 works locally, the owner-detection query returns the expected count, and an A/B run with and without NODE_ENV gives identical status codes with only the stack trace disappearing.
Release 1.0.2 Hotfix
Release 1.0.2 - Docker Compose, speculative decoding, Agent Sandbox v0.5.0 and bug fixes
Docker Compose deployment path (new, additive):
Add docker/ with deploy.sh, docker-compose.yml, model-configs.yaml (18 models), litellm routing config, and compose/secret generators
deploy-agentic-stack.sh gains --docker/--platform=docker/--down flags
Credentials generated at deploy time via generate-vault-secrets.sh
Speculative decoding:
vllm chart: speculative.enabled flag and --speculative_config wiring
xeon-values.yaml: draft-model definitions for Llama-3.1-8B, Qwen3-Coder-30B, Qwen2.5-Coder-14B (disabled by default)
Menu options 27-29 added in deploy-agentic-stack.sh
Agent Sandbox v0.4.6 -> v0.5.0:
Replace local nerdctl/BuildKit build with published upstream images
CRDs move to v1beta1; namespace is agent-sandbox-system
sandbox-router reads ROUTER_AUTH_TOKEN from a Kubernetes Secret
Redis Stack chart fixes:
Rename subchart values key redis -> redis-stack-server
Make storage_class optional
Removed Chart.lock and vendored redis-stack-server-0.4.21.tgz
Signed-off-by: vhpintel vijay.kumar.h.p@intel.com
Co-authored-by: jaswanth8888 jaswanth.karani@intel.com
Co-authored-by: sandeshk-intel sandesh.kumar.s@intel.com
Co-authored-by: sgurunat gurunath.s@intel.com
Release 1.0.1 Hotfix
This includes two security and architecture improvements to the Enterprise Agent Toolkit:
Secure standalone Redis Stack: require auth, drop NodePort, add NetworkPolicy
The standalone Redis Stack chart previously deployed with type: NodePort and no authentication. Since deploy_redis=on is the default, every deployment published Redis on all node IPs in the 30000–32767 range with full access, allowing anyone who could route to a node to read/modify agent session data.
Changes:
Add secret.yaml holding the password and full connection URL; inject via REDIS_ARGS=--requirepass
Default the Service type to ClusterIP (in-cluster only)
Add an opt-in NetworkPolicy for namespace-level access control
Generate redis_stack_password in vault.yml (independent of gateway's redis_password)
Document credentials, ClusterIP-only access model, and NodePort firewalling guidance
BREAKING CHANGE: Existing deployments lose the Redis NodePort and now require a password. Clients should read the URL from the redis-stack-server-credentials Secret.
Streamline authentication stack to use GenAI Gateway native auth
Remove legacy Keycloak identity provider integration that is no longer part of the active deployment path. The toolkit uses GenAI Gateway (LiteLLM) for API key management and access control.
Changes:
Simplify deployment flow by removing unused auth service orchestration
Clean up helm charts, playbooks, and scripts for leaner codebase
Retain OIDC configuration hooks in helm values for extensibility
Release Notes: Intel® AI for Enterprise Agent Toolkit – Version 1.0.0
Tag: release-1.0.0 | License: Apache 2.0
System Requirements
| Category | Minimum | Recommended |
|---|---|---|
| OS | Ubuntu 22.04, Ubuntu 24.04 | |
| CPU | Intel® Xeon® Scalable (x86_64), 96 cores | 96+ cores |
| RAM | 256 GB | 256+ GB |
| Disk | 150 GB (varies by model) | 200 GB |
| Architecture | x86_64 | x86_64 |
Additional requirements: internet access, SSH key pair, SSL/TLS certificates, Hugging Face API token with read access to gated models.
Deployment Modes
-
Single Node (all-in-one, development / demos)
-
Single Master + Multiple Workers (higher throughput, staging / Prod)
Key Features
Multi-Node Kubernetes Support
Full multi-node cluster deployment is enabled.
Memory
(Redis + RediSearch (short-term) · PostgreSQL + pgvector (long-term)
Full-lifecycle agent memory: short-term session context and long-term cross-session knowledge. Enables RAG, semantic search, and stateful multi-turn workflows.
Intelligent Routing
Routes LLM workloads to the optimal compute target by task intent — reasoning tasks to to GPU or cloud/MaaS endpoints; execution, encoding, and coordination tasks are routed to CPU on Xeon. Configurable per application, model-agnostic.
KubeRay Distributed Computing
KubeRay operator (v1.3.0) and RayCluster CR are deployed via Helm (core/helm-charts/kuberay/). Supports autoscaling head/worker group specs with security hardening. KubeRay configuration is managed through a dedicated kuberay-config.yaml file. Enables parallel tool calling for AI agent workflows. Controlled via the deploy_kuberay toggle in agentic-config.cfg.
Agent Sandbox
Isolated, ephemeral Kubernetes pod environments for safe code execution by AI agents (based on kubernetes-sigs/agent-sandbox v0.4.6). Each sandbox pod has its own filesystem, network namespace, and process tree. Controlled via deploy_agent_sandbox in agentic-config.cfg.
Flowise Integration
Flowise (low-code visual agent pipeline builder) is deployed as an optional plugin with PostgreSQL and Redis backends. A sample software-team multi-agent template is included. Controlled via the deploy_agenticai_plugin toggle.
New and Updated Model Support
The following models are available for CPU deployment via the interactive model selection menu:
| # | Model | Type |
|---|---|---|
| 21 | Qwen/Qwen3-Coder-30B-A3B-Instruct |
LLM (Coding) |
| 22 | Qwen/Qwen2.5-Coder-14B-Instruct |
LLM (Coding) |
| 23 | BAAI/bge-base-en-v1.5 |
Embedding |
| 24 | BAAI/bge-reranker-base |
Reranker |
| 25 | Qwen/Qwen3-30B-A3B-Instruct-2507 |
LLM |
| 26 | google/gemma-4-26B-A4B-it |
LLM |
vLLM Upgrade
vLLM CPU image upgraded to v0.23.0. Embedding and reranker model identifiers updated. Unused vault_pass_file variable removed.
Proxy Support
Corporate HTTP/HTTPS proxy support for fresh installs and multi-node cluster operations. All Ansible playbooks (cluster.yml, scale.yml) propagate proxy_env to package installs, kubeadm join, image pulls, and containerd systemd drop-in configuration.
Component Versions
| Component | Version |
|---|---|
| vLLM CPU Image | v0.23.0 |
| KubeRay Operator | 1.3.0 |
| Agent Sandbox | v0.4.6 |
| Kubespray | v2.27.0 |
| Observability Stack Chart | 72.5.1 |
| Langfuse Trace Chart | 1.5.30 |
| Ingress Controller | 4.12.2 |
Resources
License
- Licensed under the Apache License 2.0.
Thank you for using Intel® AI for Enterprise Agent Toolkit!
Release Notes: Intel® AI for Enterprise Agent Toolkit – Version 0.5.0
Tag: release-0.5.0 | License: Apache 2.0
Overview
Intel® AI for Enterprise Agent Toolkit an open source, production-grade, Kubernetes-based platform for deploying and operating enterprise AI agents on Intel® Xeon® processors.
The toolkit bundles all infrastructure components required to move from prototype to production: a policy-driven API gateway, intelligent LLM inference routing and full observability. All pre-integrated and deployable in a single automated workflow.
System Requirements
| Category | Minimum | Recommended |
|---|---|---|
| OS | Ubuntu 22.04, Ubuntu 24.04 | |
| CPU | Intel® Xeon® Scalable (x86_64), 96 cores | 96+ cores |
| RAM | 256 GB | 256+ GB |
| Disk | 150 GB (varies by model) | 200 GB |
| Architecture | x86_64 | x86_64 |
Additional requirements: internet access, SSH key pair, SSL/TLS certificates, Hugging Face API token with read access to gated models.
Deployment Modes
- Single Node Deployment
Key Features
Kubernetes Orchestration
Automates deployment, scaling, and management of AI inference clusters.
Model Management
Automated deployment and lifecycle management of LLM models; supports pre-validated models available on Hugging Face Hub.
Referred models will be automatically downloaded from Hugging Face.
Observability
Native Kubernetes monitoring (metrics, visualization, alerting) for apps and cluster health.
Automation Scripts
End-to-end scripts for cluster setup, deployment, and model onboarding.
Memory
(Redis + RediSearch (short-term) · PostgreSQL + pgvector (long-term)
Full-lifecycle agent memory: short-term session context and long-term cross-session knowledge. Enables RAG, semantic search, and stateful multi-turn workflows.
Intelligent Routing
Routes LLM workloads to the optimal compute target by task intent — reasoning tasks to to GPU or cloud/MaaS endpoints; execution, encoding, and coordination tasks are routed to CPU on Xeon. Configurable per application, model-agnostic.
Updated Model Support
The following models are available for CPU deployment via the interactive model selection menu:
| # | Model | Type |
|---|---|---|
| 21 | meta-llama/Llama-3.1-8B-Instruct |
LLM (General purpose)) |
| 22 | Qwen/Qwen3-Coder-30B-A3B-Instruct |
LLM (Coding) |
| 23 | Qwen/Qwen2.5-Coder-14B-Instruct |
LLM (Coding) |
| 24 | openai/whisper-small |
LLM (Coding) |
| 23 | BAAI/bge-small-en-v1.5 |
Embedding |
| 23 | BAAI/bge-reranker-base |
Reranker |
vLLM Version
vLLM CPU image version v0.21.0.
Proxy Support
Corporate HTTP/HTTPS proxy support for fresh installs . All Ansible playbooks (cluster.yml) propagate proxy_env to package installs, kubeadm join, image pulls, and containerd systemd drop-in configuration.
Component Versions
| Component | Version |
|---|---|
| vLLM CPU Image | v0.21.0 |
| Observability Stack Chart | 72.5.1 |
| Langfuse Trace Chart | 1.5.30 |
| Ingress Controller | 4.12.2 |
Resources
License
- Licensed under the Apache License 2.0.
Thank you for using Intel® AI for Enterprise Agent Toolkit!