A programming framework for agentic AI 🤖

A platform to crowdsource AI computation

SiLLM simplifies the process of training and running Large Language Models (LLMs) on Apple Silicon by leveraging the MLX framework.

Superduper: Build end-to-end AI applications and agent workflows on your existing data infrastructure and preferred tools - without migrating your data.

A library to communicate with ChatGPT, Claude, Copilot, Gemini, HuggingChat, and Pi

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

dstack is a lightweight, open-source alternative to Kubernetes & Slurm, simplifying AI container orchestration with multi-cloud & on-prem support. It natively supports NVIDIA, AMD, & TPU.

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

Monocle is a framework for tracing GenAI app code. This repo contains implementation of Monocle for GenAI apps written in Python.

Run any open-source LLMs, such as Llama, Gemma, as OpenAI compatible API endpoint in the cloud.

Design, conduct and analyze results of AI-powered surveys and experiments. Simulate social science and market research with large numbers of AI agents and LLMs.

Fast Inference of MoE Models with CPU-GPU Orchestration

Kurtis is a fine-tuning, inference and evaluation tool built for SLMs (Small Language Models), such as Huggingface's SmolLM2.

Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.

Official Implementation of EAGLE-1 (ICML'24) and EAGLE-2 (EMNLP'24)

AI-powered cybersecurity chatbot designed to provide helpful and accurate answers to your cybersecurity-related queries and also do code analysis and scan analysis.

Optimizing inference proxy for LLMs

Standardized Serverless ML Inference Platform on Kubernetes

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs