RFC: clarifying buffer declaration and access #63
Conversation
Co-authored-by: Eric Lunderberg <Lunderberg@users.noreply.github.com>
vinx13
force-pushed
the
clarify-buffer-access
branch
from
507c133
to
116cdc4
Compare
| convention of `PrimFunc` and are translated to assertions of buffer shapes, strides, etc. in | ||
| runtime. | ||
|
|
||
| During the lowering process, although buffer accesses inside `PrimFunc` are flattened to match |
There was a problem hiding this comment.
I have a proof-of-concept implementation of this removal at apache/tvm#10940. It is able to pass all tests in test_target_codegen_llvm.py, and I don't see any roadblocks for updating the other tests as well.
There was a problem hiding this comment.
Nice. We can then consider moving to explicit DeclBuffer (mentioned in this RFC) to define the alias.
There was a problem hiding this comment.
thanks @vinx13 @Lunderberg for this clarifying RFC! did a fine-grained pass and attempted to clarify some of the wording per my understanding.
Co-authored-by: Andrew Reusch <areusch@gmail.com>
| - Buffer map of `PrimFunc`. This specifies how opaque parameters of type `T.handle` should be | ||
| interpreted as a buffer inside the `PrimFunc`. `T.handle` represents an opaque pointer (`void *`). | ||
| - `T.alloc_buffer` is used `S-TIR` to create and allocate a buffer. | ||
| - `T.buffer_decl` can be used to create a buffer alias by specifying the underlying data variable to |
There was a problem hiding this comment.
Yes. In TVM script, it is T.buffer_decl and we have plan to rename it to T.decl_buffer
| that has a vectorized type, or by using `Ramp` as an index into a buffer that has a scalar type. | ||
| For N-D buffer indices, it is possible that `Ramp` being used in multiple dimensions | ||
| (e.g. `A[Ramp(...), ..., Ramp(...)]` ). In this case the number of lanes of the data type of such | ||
| value is the product of each `Ramp`. We may want to limit `Ramp` to only the last dimension as |
There was a problem hiding this comment.
@Lunderberg has already sent follow up PR to limit Ramp to the last dimension. I'll update this sentence
| for TIR transformations. Passes that rewrite buffers should clearly indicate how aliased buffers | ||
| are handled. For example, when changing the underlying layout of stored elements in a buffer, all | ||
| buffer aliases must also be updated. While buffer aliasing is typically free at runtime, this | ||
| imposes a cost for buffer aliasing both to compile times and development complexity. |
There was a problem hiding this comment.
we don't have any alias analysis right now
| - Buffer map of `PrimFunc`. This specifies how opaque parameters of type `T.handle` should be | ||
| interpreted as a buffer inside the `PrimFunc`. `T.handle` represents an opaque pointer (`void *`). | ||
| - `T.alloc_buffer` is used `S-TIR` to create and allocate a buffer. | ||
| - `T.buffer_decl` can be used to create a buffer alias by specifying the underlying data variable to |
There was a problem hiding this comment.
ok, if you're using T. notation here, you mean TVMScript, right? in which case, can you update all occurrences to match what you would grep for if using TVMScript? right now it's a dead-end.
|
|
||
| `T.buffer_decl` creates a buffer alias if the underlying data variable (`.data` field) overlaps with | ||
| another buffer. Buffer created via `alloc_buffer` always do not alias. Buffer aliases do not need | ||
| `Allocate` to create the data variable. Each data variable can be allocated via `Allocate` once. |
There was a problem hiding this comment.
ok, can you update this in the doc? you could simply state "Buffer aliases do not need Allocate to create the data variable--they may simply reuse the data variable from the Buffer being aliased."
| @@ -84,8 +92,9 @@ imposes a cost for buffer aliasing both to compile times and development complex | |||
| **Discussion: When it is safe to transform a buffer** | |||
|
|
|||
| We would like to discuss some examples of when it is safe to transform a buffer w.r.t. aliasing rules: | |||
|
|
|||
| (1) reshape, (2) layout transform (e.g. swap indices), (3) compact. | |||
| - (1) reshape | |||
There was a problem hiding this comment.
i think proper numbered list syntax is like
1. reshape
2. layout transform ...
3. compact
|
|
||
| (1) reshape, (2) layout transform (e.g. swap indices), (3) compact. | ||
|
|
||
| (1) is fine under aliasing as long as the low level memory is shared. (2) and (3) would need more |
There was a problem hiding this comment.
yeah that makes sense, can you update the RFC?
|
@vinx13 let me know when this is ready for another look! |
|
@areusch it is ready for another look |
| - Buffer map of `PrimFunc`. This specifies how opaque parameters of type `T.handle` should be | ||
| interpreted as a buffer inside the `PrimFunc`. `T.handle` represents an opaque pointer (`void *`). | ||
| - `T.alloc_buffer` is used `S-TIR` to create and allocate a buffer. | ||
| - `T.buffer_decl` can be used to create a buffer alias by specifying the underlying data variable to |
There was a problem hiding this comment.
could you add a note so someone knows what to grep for post-future rename?
|
|
||
| **Explicit `DeclBuffer` IR construct** | ||
|
|
||
| `T.buffer_decl` is not an explicit statement in TIR - There is no such node in TIR. |
| for TIR transformations. Passes that rewrite buffers should clearly indicate how aliased buffers | ||
| are handled. For example, when changing the underlying layout of stored elements in a buffer, all | ||
| buffer aliases must also be updated. While buffer aliasing is typically free at runtime, this | ||
| imposes a cost for buffer aliasing both to compile times and development complexity. |
There was a problem hiding this comment.
ok, perhaps we could state that in RFC i guess? it seems pretty useful to have that if we make this change.
|
@areusch added some minor updates based on the comments, see the last two commits |
In apache/tvm#9727 and RFC#39, we deprecated Load and Store to use BufferLoad and BufferStore instead in order to support generalized multi-dimensional physical buffer access. This is a follow-up that documents necessary clarifications, implications about the new buffer convention, as well as the post-hoc pass checklist.
Rendered version: https://github.com/vinx13/tvm-rfcs/blob/clarify-buffer-access/rfcs/0063-clarifying-buffer-declaration-and-access.md
cc @tqchen @Lunderberg @junrushao1994 @csullivan @mbs-octoml @jroesch @areusch @wrongtest @Hzfengsy