Params4bit.to does not move quant_state.code

[matthewdouglas]

System Info

• Transformers version: 4.49.0
• Accelerate version: 1.4.0
• 🤗 Diffusers version: 0.32.2
• Platform: Linux-6.13.2-arch1-1-x86_64-with-glibc2.41
• Python version: 3.12.8
• PyTorch version (GPU?): 2.6.0+cu124 (True)
• Bitsandbytes version: 0.45.3.dev0
• Safetensors version: 0.5.2
• Accelerator: NVIDIA GeForce RTX 4090, 24564 MiB

Reproduction

See huggingface/diffusers#10798

Expected behavior

Params4bit.to() should move all of the required quantization state.