Fix a bug in CUDA scratch memory pool

[Rombur]

There is a bug when we try to acquire an object in the scratch memory pool. We basically do the opposite of what we want to do, if we can acquire a free object we mark it as busy and then, we try to acquire the next one. The loop exits when we find a busy object. The code is unique to CUDA and does not appear in HIP or SYCL.

[dalg24]

I see how this was wrong but it is not obvious to me how that was causing the issue reported in #4665

Would you please elaborate.

[dalg24]

Retest this please

[dalg24]

Would you please elaborate.

Never mind I got it. The first time around it marks all spots in the pool as in-use and take the 1st one. The second time around it marks the 1st one as in-use and use the 2nd.

[ ] 1  1  1
 1 [ ] 1  1
[ ] 0  1  1
 1  1 [ ] 1
[ ] 1  0  1
 1 [ ] 0  1
[ ] 0  0  1
 1  1  1 [ ]
[ ] 1  1  0
 1 [ ] 1  0
[ ] 0  1  0
 1  1 [ ] 0
etc.

Eventually it uses all 10 spots.

[dalg24]

All CUDA builds passed. Not bothering to wait for SYCL.