Use static shape where available in TensorFlow backend #32
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When running in graph mode, TensorFlow tensors have two types of shapes: - a static shape, known when the graph is built - a dynamic shape, only known when the graph is run The static shape is important especially for the channels dimension, since the number of variables needed for a convolution depends on this shape. In the current implementation, `einops` uses only the dynamic shape in graph execution mode, which means that static shape information is lost after calling one of the einops functions. I have updated the TensorFlow backend to only use dynamic shape information for dimensions where the static shape is unknown.
|
LGTM, thanks! Does it only help you with interpreting shapes while constructing graph or there are other important cases? |
|
Not only. Having unknown shapes for the channel dimension breaks TF Keras layers, since the number of variables becomes unknown and cannot be allocated (e.g. the kernel size in a convolutional layer depends on the number of input channels). |
|
@slimm some minimalistic example for tests would help |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When running in graph mode, TensorFlow tensors have two types of shapes:
The static shape is important especially for the channels dimension, since the number of variables needed for a convolution depends on this shape.
In the current implementation,
einopsuses only the dynamic shape in graph execution mode, which means that static shape information is lost after calling one of the einops functions. I have updated the TensorFlow backend to only use dynamic shape information for dimensions where the static shape is unknown.