Foundry Local v2.0.1

🚀 Foundry Local v2.0.1 Release Notes

We're excited to announce Foundry Local v2.0.1. This release introduces a unified, cross-language inference API, a shared native runtime, and packaging and reliability improvements for Linux and ARM64.

Important

In-process OpenAI-style SDK clients are replaced by the Session API. Migrate inference code to ChatSession, EmbeddingsSession, or AudioSession and use typed Request, Response, and Item objects.

The deprecated OpenAI-style clients remain temporarily available in C#, Python, JavaScript/TypeScript, and Rust to ease migration.

OpenAI-compatible request and response types remain supported through the local HTTP service.

🆕 What's New

🧰 Unified Session API

Foundry Local now provides a consistent inference model built around Model → Session → Request<Item...> → Response<Item...>. ChatSession, EmbeddingsSession, and AudioSession are available across C#, Python, JavaScript/TypeScript, Rust, C++, and the stable C ABI.

The SDKs now share the cross-platform foundry_local native runtime, a versioned C ABI, consistent native error propagation, and deterministic resource ownership. The C++ SDK includes a header-only RAII wrapper over the C ABI.

💬 Sessions, Items, and Streaming

• Stateful multi-turn chat with turn counting and history rollback.
• Typed text, message, image, audio, tensor, byte, tool-call, tool-result, and speech items.
• Multimodal messages with URI-based or in-memory image and audio inputs.
• Synchronous and streaming responses, request cancellation, streaming cancellation, finish reasons, and token usage.
• ItemQueue support for incremental input, including live PCM audio.
• Typed tensor outputs for batch embeddings and detailed speech results with segment and word timing.

🛠️ Tool Calling and Responses

• Session-level tool registration and removal, with auto, none, and required tool choice.
• Typed tool-call outputs and tool-result inputs for multi-turn tool-calling flows.
• Prompt-template application is now handled internally by ChatSession.
• The local HTTP service retains OpenAI-compatible chat completions, models, embeddings, audio transcription, Responses API operations, image/audio inputs, and Server-Sent Events streaming.

🔄 Migration Notes

Installation

Language	1.2.4 packages	2.0.1 package	Required migration
C#/.NET	Microsoft.AI.Foundry.Local or Microsoft.AI.Foundry.Local.WinML	Microsoft.AI.Foundry.Local	Remove .WinML; update the main package to 2.0.1
Python	foundry-local-sdk or foundry-local-sdk-winml	foundry-local-sdk	Remove -winml; pin the main package to 2.0.1
JavaScript	foundry-local-sdk plus optional foundry-local-sdk-winml	foundry-local-sdk	Remove the optional WinML package; require Node.js 20+
Rust	foundry-local-sdk with optional winml feature	foundry-local-sdk	Remove features = ["winml"]; published 2.0.1 availability remains a caveat
C++/C	Legacy Core/static SDK packages	Microsoft.AI.Foundry.Local.Runtime plus v2 headers	Replace the legacy Core dependency and migrate build/include/link configuration
REST	Any OpenAI-compatible client	Same OpenAI-compatible client	No Foundry Local language package is required

APIs

Manager, catalog, model discovery, download, load, unload, and web-service workflows remain largely familiar. Execution-provider handling is now automatic by default: the unified package detects available hardware and selects WinML, WebGPU, CPU, or CUDA as appropriate. Applications that explicitly selected or configured an execution provider in 1.2.4 should review that customization when migrating.

C++ applications require a broader migration to the new RAII wrapper and shared native runtime. Direct OpenAI-style in-process clients remain available but are deprecated and scheduled for removal at the end of 2026. New applications should use the Session API.

1.2.4 in-process API	Preferred 2.0.1 in-process API	OpenAI-compatible REST API
ChatClient / OpenAIChatClient	ChatSession	Existing /v1/chat/completions contract remains compatible
EmbeddingClient	EmbeddingsSession	Use /v1/embeddings; OpenAI-compatible request and response shapes
AudioClient	AudioSession	Use /v1/audio/transcriptions; OpenAI-compatible transcription contract
Live-audio client	AudioSession with ItemQueue	No equivalent persistent live-input REST session; use the native SDK session API
OpenAI request and message types	Request containing typed Item objects	Continue sending OpenAI JSON request and message objects
OpenAI response and streaming-chunk types	Response or streamed Item objects	Continue receiving OpenAI JSON responses and SSE streaming events
Per-request chat settings	RequestOptions and SearchOptions	Existing request fields such as temperature, max_tokens, and top_p remain in the JSON body
Tools embedded in an OpenAI chat request	Register tools on ChatSession; exchange ToolCallItem and ToolResultItem	Existing OpenAI tools, tool_choice, tool_calls, and tool-result messages remain compatible
Application-managed conversation history	Stateful ChatSession, including turn count and undo	Remains stateless at the HTTP layer; clients continue sending conversation history with each request
Prompt-template metadata	Deprecated; templates are applied internally by ChatSession	No migration required; prompt formatting remains internal to Foundry Local
Model catalog methods	Adds model-version enumeration	Existing /v1/models and /v1/models/{model_id} remain; model operations are expanded
In-process Responses client	Use typed sessions or the web service	/v1/responses remains available and gains response retrieval and input-item operations

🐞 Known Issues

• On RTX Spark (Windows Arm), NVIDIA TensorRT RTX models fail on first load when decoding is set to top_k
• Rust SDK package publishing is in progress

⚡ Improved

📦 Platform Packaging

• There is now one package per SDK; separate -winml packages are no longer needed.
• The unified package detects the available hardware and loads the appropriate WinML, WebGPU, CPU, or CUDA execution provider.
• Hardware acceleration is included in the primary packages.
• All language SDKs share the foundry_local native runtime.
• Linux and ARM64 packaging has been improved.
• Linux packages now target manylinux_2_28 compatibility.

🗂️ Models and Downloads

• Enumerate model versions and select or download a specific version.
• Configure catalog URLs, regions, filters, and version limits with region-aware fallback.
• More accurate cached-model enumeration and refreshed model information across language bindings.
• Resumable, chunked, transactional model downloads with cross-process locking, bounded-memory streaming, progress callbacks, and cancellation.

🧱 Reliability

• More reliable model downloads and concurrent cache access.
• Improved HTTP diagnostics and Linux certificate handling, including common CA bundle discovery and SSL_CERT_FILE support.
• Graceful manager shutdown that stops new operations, cancels active work, closes sessions, stops the web service, and unloads models.

🐛 Fixed in 2.0.1

Linux Packaging

• Linux Python wheels now use valid manylinux_2_28_x86_64 and manylinux_2_28_aarch64 tags.
• Restored Linux ARM64 wheel generation and added architecture-specific artifact validation.
• Builds and release validation now run in pinned manylinux containers with auditwheel repair, glibc compatibility checks, and bundled-library validation.
• Prevented duplicate ONNX Runtime and ONNX Runtime GenAI libraries in repaired Python wheels; they remain declared package dependencies.

Platform Reliability

• Fixed Linux HTTPS failures caused by unavailable CA bundle paths in statically linked curl or OpenSSL configurations.
• Restored explicit dependencies required for older glibc environments.
• Improved catalog-index synchronization and GCC/Clang compatibility for concurrent catalog access.

📚 Resources

📖 Documentation	https://aka.ms/foundry-local-docs
🐙 GitHub	https://github.com/microsoft/Foundry-Local
🧪 Samples	https://github.com/microsoft/Foundry-Local/tree/main/samples
💬 Discord	https://aka.ms/foundry-local-discord

💙 Feedback

File issues at https://github.com/microsoft/Foundry-Local/issues and include the SDK version, platform, and relevant diagnostics.