Post-training LLM weight compression for low-RAM machines: Q4/Q8 quantization, green-format repair, AVX2 CPU inference, optional CUDA. ~45% less RAM at ~99.9% quality.
rust inference simd transformer neural-networks avx2 quantization int8 model-compression memory-optimization model-quantization post-training-quantization cpu-inference int4 llm gguf green-compression weight-compression low-memory-inference greencompress
-
Updated
Jul 13, 2026 - Rust