[PERFORMANCE] grads for bitwise ops + indexing

[sachinraja13]

Training of DeformableDETR, which uses gradients through indices is very slow (GPU utilisation ~60%). When I stop the gradients at sampling points, before calling grid_sample function, the GPU utilisation bumps up to ~99%

Reference Issue : Grid Sample

As fas as my understanding goes, the current implementation of grads for bitwise ops + indexing works on CPU only. Is it possible to extend this functionality to use GPU kernel also?

[sachinraja13]

My apologies, the performance bottleneck seems to be the random memory access during the sampling operation itself.