[Training/Python] Add option to enable symbolic shape inference#5107
[Training/Python] Add option to enable symbolic shape inference#5107
Conversation
ke1337
requested review from
a team,
BowenBao,
liqunfu,
spandantiwari and
thiagocrepaldi
as code owners
| # returns True if no more ready nodes | ||
| def _insert_ready_nodes(): | ||
| ready_nodes = [pn for pn in pending_nodes if all([i in defined for i in pn.input if i])] | ||
| ready_nodes = [pn for pn in pending_nodes if len(pn.input) == 0 or all([i in defined for i in pn.input if i])] |
There was a problem hiding this comment.
len(pn.input) == 0 [](start = 57, length = 19)
nit:
if not pn.input #Resolved
There was a problem hiding this comment.
thiagocrepaldi
left a comment
thiagocrepaldi
left a comment
There was a problem hiding this comment.
@spandantiwari FYI
|
|
||
| self._verify_fully_optimized_model(self.onnx_model_) | ||
|
|
||
| if self.save_model_dir or self.run_symbolic_shape_infer: |
There was a problem hiding this comment.
What scenarios are being unlocked by this change on ORTTrainer? Should this be done for both training and evaluation? InferenceSession is handled by https://github.com/microsoft/onnxruntime/blob/master/onnxruntime/python/session.py#L158
There was a problem hiding this comment.
BART needs this change, so most tensors could be statically planned. Inference does not rely on shape info for memory pattern though. (https://github.com/microsoft/onnxruntime/blob/master/onnxruntime/core/framework/session_state.cc#L313-L352)
In reply to: 488801960 [](ancestors = 488801960)
liqunfu
left a comment
liqunfu
left a comment
There was a problem hiding this comment.
Eventually we want merge shape inference function into ort c++ side.
|
Looks good to me, great to have this as an option! |
6 participants
and allow save model to directory
Description: Describe your changes.
Motivation and Context