Enable more vectorizations in LLVMCPUTileFuseAndVectorizePass #7652
Conversation
Vectorization will turn Linalg ops to vector ops and arith ops. Since we don't propagate information through arith ops during bufferization. We can't unconditionally vectorize all the ops. To prevent creating extra memref.alloc ops, we can't tile along reduction dims. The next step is to get unroll vector pass in, so we can vectorize more ops. This PR improves the performance of transformer-benchmark from 58 ms to 33 ms.
hanhanW
requested review from
antiagainst,
MaheshRavishankar and
ThomasRaoux
|
We are able to pass all the tests when turning the flag on by default. I'd like to do it after we're able to tile reduction loop in first level of tiling and enable unrolling vector passes. |
| @@ -171,10 +149,36 @@ void LLVMCPUTileFuseAndVectorizePass::runOnOperation() { | |||
|
|
|||
| // Tile and fuse for vector sizes, then tile reduction loops. We don't rely on | |||
| // unroll vector pass because it could introduce register pressure. | |||
| bool hasMatmulAndIsVectorizable = true; | |||
There was a problem hiding this comment.
Just an FYI, this should move into KernelDispatch.cpp so that we can enable this path for cases where this is true within a dispatch region.
Thats a good idea. At that time we probably have to also reduce the L1 tile size used here by default to reduce register pressure (of course search would also find that out probably, but good to have a decent default) |
Vectorization will turn Linalg ops to vector ops and arith ops. Since we
don't propagate information through arith ops during bufferization. We
can't unconditionally vectorize all the ops. To prevent creating extra
memref.alloc ops, we can't tile along reduction dims.
The next step is to get unroll vector pass in, so we can vectorize more
ops.
This PR improves the performance of transformer-benchmark from 58 ms to
33 ms.