FloatLLM is a hardware-agnostic, memory-aware LLM inference engine built in C++ to execute massive models (up to 405B) on highly constrained, low-RAM devices using dynamic memory slicing. It supports GPU accelerations, making models run extremely fast!