Repository navigation
|
I noticed that FA3 utilizes persistent blocks and determines the block count based on the number of SMs (e.g., grid(SM, 1, 1)), whereas FA2 launches blocks according to the number of tasks (e.g., grid(num_m_block, params.b, params.h)). I'm curious if this persistent approach is a performance optimization specific to the Hopper architecture. Why doesn't FA2 adopt persistent blocks? Is it due to hardware limitations or other reasons? |
Replies: 1 comment 2 replies
|
Persistent is generally better, at the cost of some complexity in the implementation. Persistence becomes more important as tensor cores get faster (so the relative cost of prolog and epilog vs mainloop is higher), so persistence is mostly used for Hopper and Blackwell. |
Persistent is generally better, at the cost of some complexity in the implementation. Persistence becomes more important as tensor cores get faster (so the relative cost of prolog and epilog vs mainloop is higher), so persistence is mostly used for Hopper and Blackwell.
One could certainly implement it for Ampere as well (I think we have some version of that at some point) and get a bit of a boost.