Nano: fix keras onnx model output shape #7138
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rnwang04
approved these changes
Jan 3, 2023
python/nano/src/bigdl/nano/deps/onnxruntime/tensorflow/tensorflow_onnxruntime_model.py
Show resolved
Hide resolved
| @@ -78,7 +86,10 @@ def __call__(self, *args, **kwargs): | |||
| kwargs[name] = value | |||
| for param in self._call_fn_args_backup[len(inputs):len(self._forward_args)]: | |||
| inputs.append(kwargs[param]) | |||
| return self.call(*inputs) | |||
| outputs = self.call(*inputs) | |||
There was a problem hiding this comment.
If I understand right, here returns original onnx output(a list) ? so maybe it contains _nesting_level at first (not start from 0) ?
It's just a bit confusing whether this way of adding lists works in all cases
There was a problem hiding this comment.
No, here outputs is a single Tensor, not onnx output.
There was a problem hiding this comment.
self.call will call tf.py_function(func, ..., Tout=tf.float32), which will convert the output of func to a single Tensor
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fix the output shape of keras onnx model
1. Why the change?
tf.py_function(func, ..., Tout=tf.float32)will convert the output offuncto a single Tensor, that means[t]or[[t]](tis atf.Tensor) all will be converted tot.Besides, if the original model return a single tensor
t, then after tracing, onnxruntime model will return[t].These two rules will destroy the output shape when the original output is a single tensor, or a list of tensor with only one element, or a list of list of tensor with only one element, ...
Before this PR, we simply check whether the output is a list and has only one element, if is, then return its first element. We use this method to
However, this method cannot distinguish whether the original model return
tor[t](onnxruntime model will return[t]in both case). And we don't handle the influence oftf.py_function.This PR fix both, it save the right output shape first, and convert the final output to this right shape.
2. User API changes
N/A
3. Summary of the change
N/A
4. How to test?