Add cuBLAS or cuDNN support

[coreylowman]

cuBLAS only has matmuls and vec mat muls, so no convs unless a im2col is also added. cuDNN definitely has convs and a couple other nice things, but unsure how matmuls work upon first look

[M1ngXU]

from what i've seen is that cudnn is great for RNN and CONVs, but i would have to dive deeper into the docs to understand their api

[M1ngXU]

but we could start with cublas, as it is most likely easier

Edit: I'll have a look into this

[M1ngXU]

I checked out cuDNN; there are so many functions! can you maybe give me a list of relevant functions? since cuDNN requires very "special" settings, cuDNN would most likely have to be a seperate device/have a different kind of tensor?

This is my cudnn branch: https://github.com/M1ngXU/cudarc/tree/cudnn

[coreylowman]

I really like the idea of cuDNN being a different device, that definitely aligns with the device refactor I'm planning for dfdx. Thanks for sharing your code that'll def be useful

[coreylowman]

cublas support was added. will add another issue for cudnn