Multi-tenancy and policy-based framework for Kubernetes.

This project is designed to simulate GPU information, making it easier to test scenarios where a GPU is not available.

An open-source runtime for composable workflows. Great for AI agents and CI/CD.

aibrix static site

Drivers plans and notes

Reference implementations of MLPerf™ inference benchmarks

reflect api without runtime reflect.Value cost

GaussDB driver and toolkit for Go

Start the main process of a pod only if elected via kubernetes leader election. While this was developed for LINSTOR, it may proof useful for other use cases.

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting yo…

DeepEP: an efficient expert-parallel communication library

Hydrophone is a lightweight Kubernetes conformance tests runner

Cost-efficient and pluggable Infrastructure components for GenAI inference

Production-tested AI infrastructure tools for efficient AGI development and community-driven innovation

🤖💬 extra-small AI SDK for Browser, Node.js, Deno, Bun or Edge Runtime.

AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text.

An Envoy inspired, ultimate LLM-first gateway for LLM serving and downstream application developers and enterprises

A containerd snapshotter with data deduplication and lazy loading in P2P fashion

📍 A map of my life, where each week I've been alive is a little box.

GenAI inference performance benchmarking tool

InstructLab Core package. Use this to chat with a model and execute the InstructLab workflow to train a model using custom taxonomy data.

☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!