You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
thanks for the request! it requires cuda files to compile like bitsandbytes optimizer, so it's complicated to move entire implementations into this project, instead, maybe we can support the optimizer like this example, that manually install or compile, and use it through pytorch-optimizer.
I'll figure it out how to smoothly support this in pytorch-optimizer :)
that's a great idea! it'd be much better that bnb supports 4-bit optimizers. then, for now, wait for the response and decide whether including both or only bnb.
Paper and Code
Paper: Memory Efficient Optimizers with 4-bit States
Code : https://github.com/thu-ml/low-bit-optimizers/blob/main/lpmm/optim/optimizer.py
The text was updated successfully, but these errors were encountered: