Fix plugin EP profiling timestamp skew on macOS#27994
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes flaky AutoEP plugin-EP profiling tests on macOS by ensuring emitted EP child event timestamps are always strictly contained within their correlated ORT parent event interval (despite small cross-clock reconstruction skew).
Changes:
- Propagates correlated ORT parent event start timestamp and duration into each collected EP event.
- Updates the ORT event boundary helper (
PopOrtEvent) to carry parent timing metadata. - Clamps reconstructed EP event start/end timestamps to the correlated ORT parent interval before emitting profiling events.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| onnxruntime/test/autoep/library/example_plugin_ep_kernel_registry/ep_profiling.h | Extends stored EP event metadata with correlated ORT parent timing and updates helper function signatures. |
| onnxruntime/test/autoep/library/example_plugin_ep_kernel_registry/ep_profiling.cc | Captures ORT parent timing in StopEventImpl and clamps reconstructed EP event intervals within the ORT parent bounds in EndProfilingImpl. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
onnxruntime/test/autoep/library/example_plugin_ep_kernel_registry/ep_profiling.h
Outdated
Show resolved
Hide resolved
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This fixes a flaky failure in the plugin EP profiling tests on macOS, where reconstructed plugin event timestamps could land a few microseconds outside the correlated ORT parent event interval.
The current example plugin profiler reconstructs EP-relative timestamps by combining ORT's profiling-start offset with elapsed time from the EP clock. That reconstruction is close but not exact across clocks, and on macOS the skew was enough to fail the strict containment checks in
KernelPluginEp_SessionProfilingwith cases likeep_start < parent_startby a small margin.Instead of weakening the test, this change keeps the strict contract and fixes the profiler output so child EP events are always emitted within the correlated ORT parent event interval.
Key Changes
onnxruntime/test/autoep/library/example_plugin_ep_kernel_registry/ep_profiling.honnxruntime/test/autoep/library/example_plugin_ep_kernel_registry/ep_profiling.ccOrt::ConstProfilingEvent, attaches it to EP events duringStopEventImpl, and clamps the reconstructed EP start/end interval to the parent ORT interval before emitting the final profiling event.Why This Change Is Needed
KernelPluginEp_SessionProfilingflaky even though the logical event ordering was correct.Testing
ninja -C build/cuda/Debug onnxruntime_autoep_testcd build/cuda/Debug && ./onnxruntime_autoep_test --gtest_filter=OrtEpLibrary.KernelPluginEp_SessionProfilingcd build/cuda/Debug && ./onnxruntime_autoep_test --gtest_filter=OrtEpLibrary.KernelPluginEp_RunProfilingNotes For Reviewers
test_execution.cc; it fixes the emitted profiling data instead.