Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,10 @@
var catalog = await mgr.GetCatalogAsync();


// Get a model using an alias and select the CPU model variant
// Get a model using an alias and select the CPU variant
var model = await catalog.GetModelAsync("whisper-tiny") ?? throw new System.Exception("Model not found");
var modelVariant = model.Variants.First(v => v.Info.Runtime?.DeviceType == DeviceType.CPU);
model.SelectVariant(modelVariant);
var cpuVariant = model.Variants.First(v => v.Info.Runtime?.DeviceType == DeviceType.CPU);
model.SelectVariant(cpuVariant);


// Download the model (the method skips download if already cached)
Expand Down
20 changes: 10 additions & 10 deletions samples/cs/GettingStarted/src/ModelManagementExample/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -67,26 +67,26 @@


// OPTIONAL: `model` can be used directly and `model.SelectedVariant` will be used as the default.
// You can explicitly select or use a specific ModelVariant if you want more control
// You can explicitly select a specific variant if you want more control
// over the device and/or execution provider used.
// Model and ModelVariant can be used interchangeably in methods such as
// Model and its variants can be used interchangeably in methods such as
// DownloadAsync, LoadAsync, UnloadAsync and GetChatClientAsync.
//
// Choices:
// - Use a ModelVariant directly from the catalog if you know the variant Id
// - `var modelVariant = await catalog.GetModelVariantAsync("qwen2.5-0.5b-instruct-generic-gpu:3")`
// - Use a variant directly from the catalog if you know the variant Id
// - `var variant = await catalog.GetModelVariantAsync("qwen2.5-0.5b-instruct-generic-gpu:3")`
//
// - Get the ModelVariant from Model.Variants
// - `var modelVariant = model.Variants.First(v => v.Id == "qwen2.5-0.5b-instruct-generic-cpu:4")`
// - `var modelVariant = model.Variants.First(v => v.Info.Runtime?.DeviceType == DeviceType.GPU)`
// - optional: update selected variant in `model` using `model.SelectVariant(modelVariant);` if you wish to use
// - Get the variant from Model.Variants
// - `var variant = model.Variants.First(v => v.Id == "qwen2.5-0.5b-instruct-generic-cpu:4")`
// - `var variant = model.Variants.First(v => v.Info.Runtime?.DeviceType == DeviceType.GPU)`
// - optional: update selected variant in `model` using `model.SelectVariant(variant);` if you wish to use
// `model` in your code.

// For this example we explicitly select the CPU variant, and call SelectVariant so all the following example code
// uses the `model` instance.
Console.WriteLine("Selecting CPU variant of model");
var modelVariant = model.Variants.First(v => v.Info.Runtime?.DeviceType == DeviceType.CPU);
model.SelectVariant(modelVariant);
var cpuVariant = model.Variants.First(v => v.Info.Runtime?.DeviceType == DeviceType.CPU);
model.SelectVariant(cpuVariant);


// Download the model (the method skips download if already cached)
Expand Down
11 changes: 7 additions & 4 deletions sdk/cs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ The Foundry Local C# SDK provides a .NET interface for running AI models locally
- **Chat completions** — synchronous and `IAsyncEnumerable` streaming via OpenAI-compatible types
- **Audio transcription** — transcribe audio files with streaming support
- **Download progress** — wire up an `Action<float>` callback for real-time download percentage
- **Model variants** — select specific hardware/quantization variants per model alias
- **Model variants** — select specific hardware/quantization variants per model alias via the IModel interface
- **Optional web service** — start an OpenAI-compatible REST endpoint (`/v1/chat_completions`, `/v1/models`)
- **WinML acceleration** — opt-in Windows hardware acceleration with automatic EP download
- **Full async/await** — every operation supports `CancellationToken` and async patterns
Expand Down Expand Up @@ -138,11 +138,14 @@ var cached = await catalog.GetCachedModelsAsync();

// List models currently loaded in memory
var loaded = await catalog.GetLoadedModelsAsync();

// Check for a newer version of a model
var latest = await catalog.GetLatestVersionAsync(model);
```

### Model Lifecycle

Each `Model` wraps one or more `ModelVariant` entries (different quantizations, hardware targets). The SDK auto-selects the best variant, or you can pick one:
Each `Model` wraps one or more variant entries (different quantizations, hardware targets). The SDK auto-selects the best variant, or you can pick one:

```csharp
// Check and select variants
Expand Down Expand Up @@ -293,8 +296,8 @@ Key types:
| [`FoundryLocalManager`](./docs/api/microsoft.ai.foundry.local.foundrylocalmanager.md) | Singleton entry point — create, catalog, web service |
| [`Configuration`](./docs/api/microsoft.ai.foundry.local.configuration.md) | Initialization settings |
| [`ICatalog`](./docs/api/microsoft.ai.foundry.local.icatalog.md) | Model catalog interface |
| [`Model`](./docs/api/microsoft.ai.foundry.local.model.md) | Model with variant selection |
| [`ModelVariant`](./docs/api/microsoft.ai.foundry.local.modelvariant.md) | Specific model variant (hardware/quantization) |
| [`IModel`](./docs/api/microsoft.ai.foundry.local.imodel.md) | Model interface — shared by Model and ModelVariant |
| [`Model`](./docs/api/microsoft.ai.foundry.local.model.md) | Model with variant selection (implementation detail) |
| [`OpenAIChatClient`](./docs/api/microsoft.ai.foundry.local.openaichatclient.md) | Chat completions (sync + streaming) |
| [`OpenAIAudioClient`](./docs/api/microsoft.ai.foundry.local.openaiaudioclient.md) | Audio transcription (sync + streaming) |
| [`ModelInfo`](./docs/api/microsoft.ai.foundry.local.modelinfo.md) | Full model metadata record |
Expand Down
52 changes: 37 additions & 15 deletions sdk/cs/docs/api/microsoft.ai.foundry.local.icatalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ public abstract string Name { get; }
List the available models in the catalog.

```csharp
Task<List<Model>> ListModelsAsync(Nullable<CancellationToken> ct)
Task<List<IModel>> ListModelsAsync(Nullable<CancellationToken> ct)
```

#### Parameters
Expand All @@ -39,15 +39,15 @@ Optional CancellationToken.

#### Returns

[Task&lt;List&lt;Model&gt;&gt;](https://docs.microsoft.com/en-us/dotnet/api/system.threading.tasks.task-1)<br>
List of Model instances.
[Task&lt;List&lt;IModel&gt;&gt;](https://docs.microsoft.com/en-us/dotnet/api/system.threading.tasks.task-1)<br>
List of IModel instances.

### **GetModelAsync(String, Nullable&lt;CancellationToken&gt;)**

Lookup a model by its alias.

```csharp
Task<Model> GetModelAsync(string modelAlias, Nullable<CancellationToken> ct)
Task<IModel> GetModelAsync(string modelAlias, Nullable<CancellationToken> ct)
```

#### Parameters
Expand All @@ -60,15 +60,15 @@ Optional CancellationToken.

#### Returns

[Task&lt;Model&gt;](https://docs.microsoft.com/en-us/dotnet/api/system.threading.tasks.task-1)<br>
The matching Model, or null if no model with the given alias exists.
[Task&lt;IModel&gt;](https://docs.microsoft.com/en-us/dotnet/api/system.threading.tasks.task-1)<br>
The matching IModel, or null if no model with the given alias exists.

### **GetModelVariantAsync(String, Nullable&lt;CancellationToken&gt;)**

Lookup a model variant by its unique model id.

```csharp
Task<ModelVariant> GetModelVariantAsync(string modelId, Nullable<CancellationToken> ct)
Task<IModel> GetModelVariantAsync(string modelId, Nullable<CancellationToken> ct)
```

#### Parameters
Expand All @@ -81,15 +81,15 @@ Optional CancellationToken.

#### Returns

[Task&lt;ModelVariant&gt;](https://docs.microsoft.com/en-us/dotnet/api/system.threading.tasks.task-1)<br>
The matching ModelVariant, or null if no variant with the given id exists.
[Task&lt;IModel&gt;](https://docs.microsoft.com/en-us/dotnet/api/system.threading.tasks.task-1)<br>
The matching IModel, or null if no variant with the given id exists.

### **GetCachedModelsAsync(Nullable&lt;CancellationToken&gt;)**

Get a list of currently downloaded models from the model cache.

```csharp
Task<List<ModelVariant>> GetCachedModelsAsync(Nullable<CancellationToken> ct)
Task<List<IModel>> GetCachedModelsAsync(Nullable<CancellationToken> ct)
```

#### Parameters
Expand All @@ -99,15 +99,15 @@ Optional CancellationToken.

#### Returns

[Task&lt;List&lt;ModelVariant&gt;&gt;](https://docs.microsoft.com/en-us/dotnet/api/system.threading.tasks.task-1)<br>
List of ModelVariant instances.
[Task&lt;List&lt;IModel&gt;&gt;](https://docs.microsoft.com/en-us/dotnet/api/system.threading.tasks.task-1)<br>
List of IModel instances.

### **GetLoadedModelsAsync(Nullable&lt;CancellationToken&gt;)**

Get a list of the currently loaded models.

```csharp
Task<List<ModelVariant>> GetLoadedModelsAsync(Nullable<CancellationToken> ct)
Task<List<IModel>> GetLoadedModelsAsync(Nullable<CancellationToken> ct)
```

#### Parameters
Expand All @@ -117,5 +117,27 @@ Optional CancellationToken.

#### Returns

[Task&lt;List&lt;ModelVariant&gt;&gt;](https://docs.microsoft.com/en-us/dotnet/api/system.threading.tasks.task-1)<br>
List of ModelVariant instances.
[Task&lt;List&lt;IModel&gt;&gt;](https://docs.microsoft.com/en-us/dotnet/api/system.threading.tasks.task-1)<br>
List of IModel instances.

### **GetLatestVersionAsync(IModel, Nullable&lt;CancellationToken&gt;)**

Get the latest version of a model.
This is used to check if a newer version of a model is available in the catalog for download.

```csharp
Task<IModel> GetLatestVersionAsync(IModel model, Nullable<CancellationToken> ct)
```

#### Parameters

`model` [IModel](./microsoft.ai.foundry.local.imodel.md)<br>
The model to check for the latest version.

`ct` [Nullable&lt;CancellationToken&gt;](https://docs.microsoft.com/en-us/dotnet/api/system.nullable-1)<br>
Optional CancellationToken.

#### Returns

[Task&lt;IModel&gt;](https://docs.microsoft.com/en-us/dotnet/api/system.threading.tasks.task-1)<br>
The latest version of the model. Will match the input if it is the latest version.
49 changes: 49 additions & 0 deletions sdk/cs/docs/api/microsoft.ai.foundry.local.imodel.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,42 @@ public abstract string Alias { get; }

[String](https://docs.microsoft.com/en-us/dotnet/api/system.string)<br>

### **Info**

Full catalog metadata for this model or variant.

```csharp
public abstract ModelInfo Info { get; }
```

#### Property Value

[ModelInfo](./microsoft.ai.foundry.local.modelinfo.md)<br>

### **Variants**

Variants of the model that are available. Variants are optimized for different devices.

```csharp
public abstract IReadOnlyList<IModel> Variants { get; }
```

#### Property Value

[IReadOnlyList&lt;IModel&gt;](https://docs.microsoft.com/en-us/dotnet/api/system.collections.generic.ireadonlylist-1)<br>

### **SelectedVariant**

Currently selected model variant in use.

```csharp
public abstract IModel SelectedVariant { get; }
```

#### Property Value

[IModel](./microsoft.ai.foundry.local.imodel.md)<br>

## Methods

### **IsCachedAsync(Nullable&lt;CancellationToken&gt;)**
Expand Down Expand Up @@ -185,3 +221,16 @@ Optional cancellation token.

[Task&lt;OpenAIAudioClient&gt;](https://docs.microsoft.com/en-us/dotnet/api/system.threading.tasks.task-1)<br>
OpenAI.AudioClient

### **SelectVariant(IModel)**

Select a specific model variant from Variants to use for IModel operations.

```csharp
void SelectVariant(IModel variant)
```

#### Parameters

`variant` [IModel](./microsoft.ai.foundry.local.imodel.md)<br>
Model variant to select. Must be one of the variants in Variants.
37 changes: 7 additions & 30 deletions sdk/cs/docs/api/microsoft.ai.foundry.local.model.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,22 +15,22 @@ Attributes [NullableContextAttribute](https://docs.microsoft.com/en-us/dotnet/ap
### **Variants**

```csharp
public List<ModelVariant> Variants { get; internal set; }
public IReadOnlyList<IModel> Variants { get; }
```

#### Property Value

[List&lt;ModelVariant&gt;](https://docs.microsoft.com/en-us/dotnet/api/system.collections.generic.list-1)<br>
[IReadOnlyList&lt;IModel&gt;](https://docs.microsoft.com/en-us/dotnet/api/system.collections.generic.ireadonlylist-1)<br>

### **SelectedVariant**

```csharp
public ModelVariant SelectedVariant { get; internal set; }
public IModel SelectedVariant { get; internal set; }
```

#### Property Value

[ModelVariant](./microsoft.ai.foundry.local.modelvariant.md)<br>
[IModel](./microsoft.ai.foundry.local.imodel.md)<br>

### **Alias**

Expand Down Expand Up @@ -86,47 +86,24 @@ public Task<bool> IsLoadedAsync(Nullable<CancellationToken> ct)

[Task&lt;Boolean&gt;](https://docs.microsoft.com/en-us/dotnet/api/system.threading.tasks.task-1)<br>

### **SelectVariant(ModelVariant)**
### **SelectVariant(IModel)**

Select a specific model variant from [Model.Variants](./microsoft.ai.foundry.local.model.md#variants) to use for [IModel](./microsoft.ai.foundry.local.imodel.md) operations.

```csharp
public void SelectVariant(ModelVariant variant)
public void SelectVariant(IModel variant)
```

#### Parameters

`variant` [ModelVariant](./microsoft.ai.foundry.local.modelvariant.md)<br>
`variant` [IModel](./microsoft.ai.foundry.local.imodel.md)<br>
Model variant to select. Must be one of the variants in [Model.Variants](./microsoft.ai.foundry.local.model.md#variants).

#### Exceptions

[FoundryLocalException](./microsoft.ai.foundry.local.foundrylocalexception.md)<br>
If variant is not valid for this model.

### **GetLatestVersion(ModelVariant)**

Get the latest version of the specified model variant.

```csharp
public ModelVariant GetLatestVersion(ModelVariant variant)
```

#### Parameters

`variant` [ModelVariant](./microsoft.ai.foundry.local.modelvariant.md)<br>
Model variant.

#### Returns

[ModelVariant](./microsoft.ai.foundry.local.modelvariant.md)<br>
ModelVariant for latest version. Same as `variant` if that is the latest version.

#### Exceptions

[FoundryLocalException](./microsoft.ai.foundry.local.foundrylocalexception.md)<br>
If variant is not valid for this model.

### **GetPathAsync(Nullable&lt;CancellationToken&gt;)**

```csharp
Expand Down
4 changes: 3 additions & 1 deletion sdk/cs/docs/api/microsoft.ai.foundry.local.modelvariant.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,10 @@

Namespace: Microsoft.AI.Foundry.Local

> **Note:** `ModelVariant` is an internal implementation detail. Use [`IModel`](./microsoft.ai.foundry.local.imodel.md) in public API code.

```csharp
public class ModelVariant : IModel
internal class ModelVariant : IModel
```

Inheritance [Object](https://docs.microsoft.com/en-us/dotnet/api/system.object) → [ModelVariant](./microsoft.ai.foundry.local.modelvariant.md)<br>
Expand Down
4 changes: 2 additions & 2 deletions sdk/js/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -217,9 +217,9 @@ The SDK is configured via `FoundryLocalConfig` when creating the manager:
Auto-generated class documentation lives in [`docs/classes/`](docs/classes/):

- [FoundryLocalManager](docs/classes/FoundryLocalManager.md) — SDK entry point, web service management
- [Catalog](docs/classes/Catalog.md) — Model discovery and browsing
- [Catalog](docs/classes/Catalog.md) — Model discovery, browsing, and latest version checking
- [Model](docs/classes/Model.md) — High-level model with variant selection
- [ModelVariant](docs/classes/ModelVariant.md) — Specific model variant: download, load, inference
- [IModel](docs/interfaces/IModel.md) — Interface shared by Model and ModelVariant
- [ChatClient](docs/classes/ChatClient.md) — Chat completions (sync and streaming)
- [AudioClient](docs/classes/AudioClient.md) — Audio transcription (sync and streaming)
- [ModelLoadManager](docs/classes/ModelLoadManager.md) — Low-level model loading management
Expand Down
Loading
Loading