[WIP] [tracking] Improve Sequence operator handling #25066
Draft
+23
−6
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Analyzing sequence operator handling efficiency improvements in ONNX Runtime
Overview: The current sequence operator implementation relies heavily on tensor copies as noted in the TODO comment. This PR aims to improve efficiency by reducing unnecessary tensor copying through better use of move semantics and avoiding redundant allocations.
Plan:
Key optimizations implemented:
CreateTensorOrtValue()
function that creates OrtValue directly with move semantics instead of creating Tensor first then convertingNote: SequenceErase is already well-optimized as it avoids copying non-erased tensors. SequenceAt requires copying due to operational requirements but uses efficient DataTransferManager.
Fixes #18355.
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.