INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
-
Updated
May 29, 2024 - C++
INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
🤖 The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. It allows to generate Text, Audio, Video, Images. Also with voice cloning capabilities.
Add a description, image, and links to the rwkv topic page so that developers can more easily learn about it.
To associate your repository with the rwkv topic, visit your repo's landing page and select "manage topics."