Fix q to

[Qubitium]

@avtc Refractored code. May resolve your Q.to() crash.

[avtc]

Confirmed, this pr fixed issue with Q.to()

[Qubitium]

Confirmed, this pr fixed issue with Q.to()

Finally! A throne in my backside finally plucked. You can pull main for a gptq quantization quality fix. You can now properly increase batch_size to as high as your gpu can manage to increase throughput at the expensive of more vram usage.

Each gpu/model pair has a sweet spot toggle for batch_size. There is no one size fits all. More batch sometimes is slower but not always.