Skip to content

Model Package Support (Phase 2)#28271

Merged
jambayk merged 55 commits into
mainfrom
chi/model_package_2
Jun 4, 2026
Merged

Model Package Support (Phase 2)#28271
jambayk merged 55 commits into
mainfrom
chi/model_package_2

Conversation

@chilo-ms

@chilo-ms chilo-ms commented Apr 29, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR implements the model package feature for ONNX Runtime — a structured packaging format that bundles multiple model variants (targeting different EPs) with metadata for automatic variant selection at runtime. It builds on the foundational model package support introduced in #27786 by extracting the parsing logic into a standalone reusable library, simplifying the public API surface, adding new query/traversal APIs, and hardening security and correctness.

Key Components

  1. Standalone model_package/ library — A self-contained C library at the repo root that parses model packages (manifest.json, metadata.json, variant.json). No dependency on ONNX Runtime. Designed to be consumed independently by ORT, GenAI, and FL.

  2. ORT integration — ORT links the standalone library as a static lib and uses its internal C++ types directly (not the public C API) to avoid double-wrapping overhead. The public C API exists for external consumers.

  3. OrtModelPackageApi C API — A new vtable-based API surface (similar to OrtApi) exposing package parsing, variant enumeration, variant selection, and session creation.

  4. C++ wrappers and Python bindings — RAII C++ wrappers (ModelPackageContext, ModelPackageComponentContext, ModelPackageOptions) and full Python bindings.

API Surface — Standalone Library (model_package/include/model_package_api.h)

A dependency-free C API for parsing and inspecting model packages without linking ONNX Runtime.

Category Function
Status ModelPackage_ReleaseStatus
ModelPackage_GetErrorMessage
Context lifecycle ModelPackage_CreateContext
ModelPackage_ReleaseContext
Package queries ModelPackage_GetSchemaVersion
Component queries ModelPackage_GetComponentCount
ModelPackage_GetComponentName
Variant queries ModelPackage_GetVariantCount
ModelPackage_GetVariantName
ModelPackage_GetVariantFolderPath
EP compatibility ModelPackage_GetVariantEpName

API Surface — ORT (OrtModelPackageApi in onnxruntime_c_api.h)

Accessed via OrtApi::GetModelPackageApi(). Extends the standalone surface with EP-aware variant selection and session creation.

Category Function
Options CreateModelPackageOptionsFromSessionOptions
ReleaseModelPackageOptions
Context lifecycle CreateModelPackageContext
ReleaseModelPackageContext
Package queries ModelPackage_GetSchemaVersion
ModelPackage_GetComponentCount
ModelPackage_GetComponentNames
ModelPackage_GetVariantCount
ModelPackage_GetVariantNames
ModelPackage_GetVariantEpName
Selection SelectComponent
ReleaseModelPackageComponentContext
Selected variant ModelPackageComponent_GetSelectedVariantName
ModelPackageComponent_GetSelectedVariantFolderPath
Session CreateSession

Security & Correctness

  • Path traversal validation (ValidatePathSegment, ValidatePathConfinement)
  • Use-after-free fix: ModelPackageComponentContext owns all state
  • Deterministic tie-break in variant selection (first-declared wins)
  • CPU fallback loop when requested EP has no match

Session Creation

  • Clean-slate approach: default path starts from fresh OrtSessionOptions
  • Variant session/provider options merged via AddSessionConfigEntry
  • Advanced path: caller-supplied options used as-is (no metadata merge)

Files Added

  • model_package/ — Standalone library (header, parser, C API, CMakeLists, README)
  • onnxruntime/core/session/model_package_api.{h,cc} — ORT C API implementation + vtable
  • onnxruntime/core/session/model_package/model_package_options.{h,cc} — Options/EP resolution
  • onnxruntime/core/session/model_package/model_package_variant_selector.{h,cc} — Variant selector

Files Removed

  • onnxruntime/core/session/model_package/model_package_descriptor_parser.{h,cc} — Replaced by standalone lib

Testing

  • Extended test_model_package.cc with tests for all API functions, variant selection, tie-break determinism, session option dispatch, and error cases
  • Python bindings tested via e2e notebooks (phi4, qwen3-vl, whisper, nemotron)

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can commit the suggested changes from lintrunner.

Comment thread onnxruntime/core/session/onnxruntime_c_api.cc Outdated
@chilo-ms
chilo-ms marked this pull request as ready for review May 6, 2026 17:24
@chilo-ms
chilo-ms requested a review from skottmckay May 6, 2026 17:26
@chilo-ms

chilo-ms commented May 6, 2026

Copy link
Copy Markdown
Contributor Author

Please note that in variant selection, we need the new EP API SelectBestCompiledModelCompatibilityInfo from this PR.
Once that PR is merged, i can update the variant selection code accordingly.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can commit the suggested changes from lintrunner.

Comment thread include/onnxruntime/core/session/onnxruntime_c_api.h Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can commit the suggested changes from lintrunner.

Comment thread include/onnxruntime/core/session/onnxruntime_cxx_api.h Outdated
Comment thread onnxruntime/core/session/onnxruntime_c_api.cc Outdated
jambayk
jambayk previously approved these changes Jun 1, 2026
chilo-ms and others added 3 commits June 1, 2026 11:36
The submodule update in #28577 caused webgl-operators.md to be
out-of-date, failing the JS CI doc check. Revert to the prior
pointer until the doc is regenerated.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
jambayk
jambayk previously approved these changes Jun 1, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can commit the suggested changes from lintrunner.

Comment thread onnxruntime/core/session/model_package/model_package_context.cc Outdated
jambayk
jambayk previously approved these changes Jun 1, 2026
@jambayk
jambayk merged commit 27c6d31 into main Jun 4, 2026
88 checks passed
@jambayk
jambayk deleted the chi/model_package_2 branch June 4, 2026 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants