Skip to content

[EP ABI] Add support for creating EP Context models. #25124

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 103 commits into from
Jun 25, 2025
Merged

[EP ABI] Add support for creating EP Context models. #25124

merged 103 commits into from
Jun 25, 2025
+424 −82

Conversation

adrianlizarraga
Copy link
Contributor

@adrianlizarraga adrianlizarraga commented Jun 20, 2025 •
edited
Loading

Description

  • Updates OrtEp::Compile() to allow a plugin EP to create and return EPContext nodes.
  • Updates the example EP plugin to generate an example EPContext model:
image

Motivation and Context

Adds more of the functionality missing from the EP ABI used for plugin EPs.

adrianlizarraga and others added 30 commits May 28, 2025 11:38
@adrianlizarraga
Create IExecutionProvider wrapper for OrtEp
732f29a
@adrianlizarraga
Add OrtEp::GetCapability and stub implementation in dummy ep
9c39966
@adrianlizarraga
Simplify GetCapability for OrtEp (no allocator). Make OrtGraph compat…
db2cda3 
…ible with OrtEpApi.
@adrianlizarraga
Add API OrtGraph_GetNodes()
93900ba
@adrianlizarraga
Derive from OrtNode and OrtGraph instead of using variant (reduce str…
5676355 
…uct sizes for ep api)
@adrianlizarraga
Separate model-editor and ep-api graph types into different files
353fdb8
@adrianlizarraga
Revert variable name change to reduce diff
62eb132
@adrianlizarraga
Add some comments
38b5398
@adrianlizarraga
Add unit test that adds the example plugin ep to a session.
09cea7a
@adrianlizarraga
Merge branch 'main' into adrianl/ep-abi
a659a80
@adrianlizarraga
Add C APIs to get a node's name and op_type
9f45c18
@adrianlizarraga
Update error message when user passes in the wrong kind of OrtNode/Or…
7cac7d1 
…tGraph to the model editor api
@adrianlizarraga
Add ToInternal() methods to convert from OrtNode to its derived classes
5b38f4b
@adrianlizarraga
clean up
8abe39e
@adrianlizarraga
Rename AddSupportedNodes api func. Dont force eps to provide a subgra…
9abc3ff 
…ph/capability name
@adrianlizarraga
Add OrtEp::Compile()
df0636d
@adrianlizarraga
Fill out Compile() wrapper function more
177828a
@adrianlizarraga
Fix annotation
0206de9
@adrianlizarraga
Update NodeComputeInfo handling
a3ab415
@adrianlizarraga
Merge branch 'main' into adrianl/ep-abi
59b5367
@adrianlizarraga
Finish making Mul work in example plugin EP
77b3037
@adrianlizarraga
rename file
1b616d8
@adrianlizarraga @Copilot
Apply suggestions from code review
217aca1 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@adrianlizarraga @Copilot
Update onnxruntime/test/autoep/library/example_plugin_ep.cc
2adb618 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@adrianlizarraga
hide reinterpret_cast with a ToInternal call
e144c80
@adrianlizarraga
Use only one enum to differentiate between the ModelEditorApi and EpA…
9031bd5 
…pi versions of Graph structures
@adrianlizarraga
Add many more graph apis for traversing edges
37dd688
@adrianlizarraga
Move graph apis to OrtApi struct
ee354a4
@adrianlizarraga
ifdef out stuff for minimal build
3a99ff2
@adrianlizarraga
More ifdef for minimal builds
d969d11
adrianlizarraga and others added 21 commits June 17, 2025 14:08
@adrianlizarraga
Add unit test for a graph with 3 layers of nested subgraphs
93c6769
@adrianlizarraga
Start stubbing out IExecutionProvider::GetEpContextNodes()
cd785b9
@adrianlizarraga
Improve OrtArrayOfConstObjects to allow api user to create
e072a2d
@adrianlizarraga
Clean up
c40baba
@adrianlizarraga
Add ArrayOfConstObjects_GetConstData
74d3b4e
@adrianlizarraga @edgchen1
Apply suggestions from code review
bae384d 
Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
@adrianlizarraga
Adjust naming and description of OrtArrayOfConstObjects
981726f
@adrianlizarraga
wording: object vs element
910cd5e
@adrianlizarraga
Merge branch 'adrianl/ep-abi' into adrianl/ep-abi-ep-context-nodes
ba3f356
@adrianlizarraga
Add code to Example Plugin EP that creates EPContext nodes and return…
7950af3 
…s them from OrtEp::GetEpContextNodes()
@adrianlizarraga
Merge main and fix conflicts
8449070
@adrianlizarraga
Encounter block with current approach
b4f9f97
@adrianlizarraga
Get the plugin EP to generate an EPContext model and added unit test
d29285b
@adrianlizarraga
Update comments
4649264
@adrianlizarraga
Add missing documentation comment to the public API function
459c4b5
@adrianlizarraga
Clarify that EPContext nodes are constant
0aac40a
@adrianlizarraga
Merge branch 'main' into adrianl/ep-abi-ep-context-nodes
aa26dbd
@adrianlizarraga
More editing of the public documentation
cbd0a10
@adrianlizarraga
Example EP code was getting too long. Separate utility functions into…
af1a312 
… a separate compilation unit.
@adrianlizarraga
Version 2 of GetEpContextNodes
e3dff3e
@adrianlizarraga
Use correct SAL2 tag
9a127e7
@adrianlizarraga adrianlizarraga marked this pull request as ready for review June 20, 2025 21:57
@adrianlizarraga adrianlizarraga requested a review from chilo-ms June 20, 2025 21:58
@adrianlizarraga adrianlizarraga merged commit 7d22c09 into main Jun 25, 2025
89 checks passed
@adrianlizarraga adrianlizarraga deleted the adrianl/ep-abi-ep-context-nodes branch June 25, 2025 05:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HectorSVC HectorSVC HectorSVC approved these changes

@edgchen1 edgchen1 Awaiting requested review from edgchen1

@jywu-msft jywu-msft Awaiting requested review from jywu-msft

@chilo-ms chilo-ms Awaiting requested review from chilo-ms

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@adrianlizarraga @HectorSVC