A high-throughput and memory-efficient inference and serving engine for LLMs

Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.

LLM inference in C/C++

Build Conversational AI in minutes ⚡️

A full-stack Webui implementation of Large Language model, such as ChatGPT or LLaMA.

Large-scale LLM inference engine