Sketching out a "types and shapes" developer document. #7108
Conversation
ScottTodd
added
documentation
|
|
||
| ### Conversion process | ||
|
|
||
| IREE lowers programs from representations produced by high level frondends down |
There was a problem hiding this comment.
Its probably worth mentioning that we don't directly ingest Tensorflow and ingest TOSA instead. Our TOSA does not require fully defined shapes and we run shape inference after import. It will fully infer as far as it can, however not all cases on all ops are possible. E.g. transpose conv needs to know its shape at construction.
There was a problem hiding this comment.
Mentioned that a bit in the shapes section, thanks. Haven't yet gone into detail on each item down in this section of the doc.
There was a problem hiding this comment.
note: +1 on TF side. We will infer the shapes as far as it can after importing the model into MLIR.
This is the pass doing shape inference: https://github.com/google/iree/blob/94a2168c63ac5e075be132fe1b97dd8427c03733/integrations/tensorflow/iree_tf_compiler/TF/Passes.cpp#L46
ScottTodd
force-pushed
the
docs-types
branch
from
11dbb06
to
a2bdad5
Compare
| Backend references: | ||
|
|
||
| * Vulkan: [buffer and image formats](https://www.khronos.org/registry/vulkan/specs/1.0/html/vkspec.html#formats) | ||
| * SPIR-V: [types](https://www.khronos.org/registry/SPIR-V/specs/1.0/SPIRV.html#_types) and [capabilities](https://www.khronos.org/registry/SPIR-V/specs/1.0/SPIRV.html#_a_id_capability_a_capability) |
There was a problem hiding this comment.
Do we want to split this to codegen backends and drivers?
| Backend | Driver |
|---|---|
| llvm-aot | dylib |
| llvm-aot | dylib-sync |
| spir-v | vulkan |
| cuda | cuda |
There was a problem hiding this comment.
I'd prefer for the start of the document to give enough general context: we target different hardware devices and APIs, and some of those have very strict requirements, so the layers can be pretty opinionated.
The HAL section down below can go into specifics for each compiler target / device configuration.
|
|
||
| ### Conversion process | ||
|
|
||
| IREE lowers programs from representations produced by high level frondends down |
There was a problem hiding this comment.
note: +1 on TF side. We will infer the shapes as far as it can after importing the model into MLIR.
This is the pass doing shape inference: https://github.com/google/iree/blob/94a2168c63ac5e075be132fe1b97dd8427c03733/integrations/tensorflow/iree_tf_compiler/TF/Passes.cpp#L46
|
|
||
| ↓ | ||
|
|
||
| import dialects (`iree`, `tensor`, `linalg`, etc.) |
There was a problem hiding this comment.
IREE dialect was renamed to Util dialect. Maybe delete it or use standard or math dialect?
There was a problem hiding this comment.
Hm, the new architecture diagram still puts "iree" next to "tensor". I can just change it to standard here.
|
|
||
| #### Requirements for host code generation | ||
|
|
||
| #### Requirements for device code generation |
There was a problem hiding this comment.
IIRC, we don't want to generate (big?) memory allocation code on device side either.
|
|
||
| #### Requirements for device code generation | ||
|
|
||
| TODO: LLVM / SPIR-V emulation of types? |
There was a problem hiding this comment.
Do you aim to describe how codegen backends handle non native types? E.g., if i8 is not available on vulkan target env, how do we emulate i8 load/store?
There was a problem hiding this comment.
I have the "Strategies for converting between types" section up top for general information, then my hope was for you / others more familiar with the specifics of each layer to fill in the requirements or strategy preference for that layer down in this section of the doc (e.g. maybe LLVM has efficient support for emulating f64 but SPIR-V does not, so SPIR-V prefers to truncate f64 down to f32).
|
Closing as obsolete. Still would be good to have some documentation about this topic but it's probably worth doing this in the specific frontend documentation instead as frontend dialects have different restrictions themselves. |
Progress on #5223
This is light on details right now, but gives us a place to fill in more over time.