Ensure shape in sample so that the shape is correct after TFLite conversion #902
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
chenmoneygithub
force-pushed
the
fix-shape
branch
3 times, most recently
from
773fb94 to
1cc280b
Compare
mattdangerw
reviewed
Mar 23, 2023
| next_token = tf.cast(next_token, prompt.dtype) | ||
| # Ensure shape is `[None]`, otherwise it causes issues after | ||
| # converting to TFLite. | ||
| next_token = tf.ensure_shape(next_token, [None]) |
There was a problem hiding this comment.
Do you have an example of where this is causing a failure? Very curious why this is needed.
Is there someway we can test the failure without needing to depend on TFLite you think?
There was a problem hiding this comment.
Very good question - I am going to talk to TFLite folks after I get back, I know there is some effort on bringing TFLite testing to TF/Keras before conversion.
For the issue I saw - without this ensure_shape, next_token becomes shape [] instead of [None], which causes dynamic_update_slice to have a shape mismatch between operand (1st arg) and update (2nd arg).
chenmoneygithub
force-pushed
the
fix-shape
branch
6 times, most recently
from
e1da248 to
76b9bbb
Compare
chenmoneygithub
force-pushed
the
fix-shape
branch
3 times, most recently
from
d549804 to
f1b5567
Compare
chenmoneygithub
force-pushed
the
fix-shape
branch
from
f1b5567 to
31e8cff
Compare
mattdangerw
approved these changes
Mar 29, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Magically the shape is lost after tflite conversion, and
ensure_shapeworks.