lion.mdx

Lion

Lion (Evolved Sign Momentum) is a unique optimizer that uses the sign of the gradient to determine the update direction of the momentum. This makes Lion more memory-efficient and faster than [AdamW] which tracks and store the first and second-order moments.

Lion[[api-class]]

[[autodoc]] bitsandbytes.optim.Lion - init

Lion8bit

[[autodoc]] bitsandbytes.optim.Lion8bit - init

Lion32bit

[[autodoc]] bitsandbytes.optim.Lion32bit - init

PagedLion

[[autodoc]] bitsandbytes.optim.PagedLion - init

PagedLion8bit

[[autodoc]] bitsandbytes.optim.PagedLion8bit - init

PagedLion32bit

[[autodoc]] bitsandbytes.optim.PagedLion32bit - init