OASIS: Online Activation Subspace Learning for Memory-Efficient Training
This repository will contain the official implementation of OASIS, a method for reducing memory footprint during large language model (LLM) training by learning a low-rank activation subspace.
🚧 Code release coming soon.