You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Do all the input dimensions must be greater than 16 for tl.dot() operation?
I'm trying to do tl.dot(a,b) between a 2D matrix (shape: [16,16]) and 1D vector (shape[16,1]). I'm getting a compilation error as AssertionError('All values in both first input shape ([constexpr[16], constexpr[16]]) and second input shape ([constexpr[16], constexpr[1]]) must be >= 16!')
I'm not sure whether I'm doing it wrong or Triton doesn't allow dot on 1D vectors.
Have experienced the same. Triton does put certain constraints on inputs to its functions. Large-ish blocks and power of 2 dimensions are the ones I regularly encounter. I think it's more efficient to tile up things like matrix multiplication into blocks rather than matrix-vector operations (see the L2 cache optimisations section).
It seems reasonable that in this case, Triton would not use mma instructions, but rather ordinary FMA instructions. This, however, appears to be unimplemented. The list of supported sizes is here.
To my understanding, Triton also does not support optimizing other "edge-cases" when it comes to dot perf, for instance tall-and-skinny matmuls.
Do all the input dimensions must be greater than 16 for tl.dot() operation?
I'm trying to do
tl.dot(a,b)
between a 2D matrix (shape: [16,16]) and 1D vector (shape[16,1]). I'm getting a compilation error asAssertionError('All values in both first input shape ([constexpr[16], constexpr[16]]) and second input shape ([constexpr[16], constexpr[1]]) must be >= 16!')
I'm not sure whether I'm doing it wrong or Triton doesn't allow dot on 1D vectors.
Below is the code for reference.
The text was updated successfully, but these errors were encountered: