Hey @Aquaticfuller @DiyouS,
found when running some custom kernels.
What the model does. The AMO shim keeps a "busy until cycle X" timestamp for the lane. When an AMO finishes, the model sets X forward by the total latency that this AMO was charged. The next request on the lane must wait until X.
Why that is wrong. The total latency of an AMO has two parts: the time the AMO unit actually worked on it (read + modify + write back, roughly 10 cycles on a cache hit, ~70 on a miss), and the time the request spent waiting in line before the unit took it. Only the first part keeps the unit busy. The model adds both parts to the busy timestamp. Concrete example, three AMOs arriving back-to-back, hit case (10 cycles of real work each):
| back-to-back AMOs (cache hit, 10 cycles real work each) |
real hardware: waits / charged |
model before fix: waits / charged |
busy-timestamp advance (model) |
| AMO 1 |
0 / 10 |
0 / 10 |
+10 |
| AMO 2 |
10 / 20 |
10 / 20 |
+20 |
| AMO 3 |
20 / 30 |
30 / 40 |
+40 |
| AMO 4 |
30 / 40 |
70 / 80 |
+80 |
| … |
linear: +10 per AMO |
doubling per AMO |
|
| AMO 10 |
90 / 100 |
~2500 / ~2550 |
|
Note: Found by AI, I have not verified the diagnosis unfortunately don't have time to verify it now. But I wanted to report before you run into this issue and have to debug it again :-)
Hey @Aquaticfuller @DiyouS,
found when running some custom kernels.
Note: Found by AI, I have not verified the diagnosis unfortunately don't have time to verify it now. But I wanted to report before you run into this issue and have to debug it again :-)