Repositories list

• llm-inference-handbook

  Public

  bentoml/llm-inference-handbook’s past year of commit activity
  Everything you need to know about LLM inference

  inference-optimizationllmllm-inference+ 2inference-handbookinference-infrastructure

  TypeScript

  •

  Apache License 2.0

  •22•241•3•0•Updated Nov 12, 2025Nov 12, 2025

• OpenLLM

  Public

  bentoml/OpenLLM’s past year of commit activity
  Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

  llamamistralfine-tuning+ 15mlopsbentomlvicunallmmodel-inferencellmopsllm-serving+ 8

  Python

  •

  Apache License 2.0

  •788•12k•3•3•Updated Nov 10, 2025Nov 10, 2025

• comfy-pack

  Public

  bentoml/comfy-pack’s past year of commit activity
  A comprehensive toolkit for reliably locking, packing and deploying environments for ComfyUI workflows.

  ai-artcomfyuigenai+ 4comfyui-managercomfyui-workflowcomfyui-nodescomfyui-api

  Python

  •

  Apache License 2.0

  •28•192•10•2•Updated Nov 10, 2025Nov 10, 2025

• sglang

  Public

  bentoml/sglang’s past year of commit activity
  SGLang is a fast serving framework for large language models and vision language models.

  Python

  •

  Apache License 2.0

  •3.4k•0•0•0•Updated Nov 7, 2025Nov 7, 2025

• BentoML

  Public
  The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

  pythonmachine-learningdeep-learning+ 12model-servingmultimodalmlopsml-engineeringai-inferencellmgenerative-ai+ 5

  Python

  •

  Apache License 2.0

  •887•8.2k•133•3•Updated Nov 4, 2025Nov 4, 2025

• BentoOCR

  Public
  Turn any OCR models into online inference API endpoint 🚀 🌖

  ocrmodel-deploymentmodel-serving+ 2ocr-pythonai-applications

  Python

  •4•57•1•0•Updated Oct 29, 2025Oct 29, 2025

• BentoVLLM

  Public
  Self-host LLMs with vLLM and BentoML

  Python

  •

  Apache License 2.0

  •20•156•2•3•Updated Oct 28, 2025Oct 28, 2025

• pipelines-workflows

  Public
  Gruntwork Pipelines for BentoML

  23•0•0•0•Updated Oct 23, 2025Oct 23, 2025

• lago-go-client

  Public
  Lago Go Client

  Go

  •

  MIT License

  •24•0•0•0•Updated Oct 15, 2025Oct 15, 2025

• lago

  Public
  Open Source Metering and Usage Based Billing API ⭐️ Consumption tracking, Subscription management, Pricing iterations, Payment orchestration & Revenue analytics

  Go

  •

  GNU Affero General Public License v3.0

  •478•0•0•0•Updated Oct 14, 2025Oct 14, 2025

• bentocloud-homepage-news

  Public
  2•3•0•0•Updated Sep 28, 2025Sep 28, 2025

• BentoSGLang

  Public
  Python

  •1•2•0•1•Updated Sep 24, 2025Sep 24, 2025

• yatai-common

  Public
  Go

  •6•2•0•2•Updated Sep 24, 2025Sep 24, 2025

• BentoYolo

  Public
  BentoML service of YOLO v8

  Python

  •

  Apache License 2.0

  •3•14•1•0•Updated Sep 17, 2025Sep 17, 2025

• llm-optimizer

  Public
  Benchmark and optimize LLM inference across frameworks with ease

  Python

  •

  Apache License 2.0

  •13•131•6•0•Updated Sep 12, 2025Sep 12, 2025

• helm-charts

  Public
  0•3•0•0•Updated Sep 3, 2025Sep 3, 2025

• yatai-image-builder

  Public
  🐳 Build OCI images for Bentos in k8s

  dockerkubernetesdocker-image+ 3containerk8scontainer-image

  Go

  •12•19•6•12•Updated Sep 3, 2025Sep 3, 2025

• kantoku

  Public
  A Process & Socket Manager built with zmq

  Python

  •

  Other

  •260•1•0•0•Updated Aug 28, 2025Aug 28, 2025

• byoc

  Public
  Shell

  •1•1•0•2•Updated Aug 28, 2025Aug 28, 2025

• BentoProphet

  Public
  BentoML with Facebook Prophet

  Python

  •0•1•0•0•Updated Aug 27, 2025Aug 27, 2025

• XGBoostDemo

  Public
  Python

  •0•3•0•0•Updated Aug 19, 2025Aug 19, 2025

• asynq

  Public
  Simple, reliable, and efficient distributed task queue in Go

  Go

  •

  MIT License

  •871•0•0•0•Updated Aug 5, 2025Aug 5, 2025

• BentoLlamaCpp

  Public
  BentoML + llama.cpp

  Python

  •0•0•0•1•Updated Jul 29, 2025Jul 29, 2025

• BentoSpacy

  Public
  Python

  •0•0•0•1•Updated Jul 29, 2025Jul 29, 2025

• BentoTriton

  Public
  Python

  •

  Apache License 2.0

  •1•3•0•1•Updated Jul 29, 2025Jul 29, 2025

• CLIP-API-service

  Public
  CLIP as a service - Embed image and sentences, object recognition, visual reasoning, image classification and reverse image search

  cloud-nativeclipmodel-serving+ 5mlopsai-applicationsopenai-clipmodel-inferencemodel-inference-service

  Jupyter Notebook

  •

  Apache License 2.0

  •4•65•0•1•Updated Jul 29, 2025Jul 29, 2025

• BentoLMDeploy

  Public
  Self-host LLMs with LMDeploy and BentoML

  Python

  •2•21•1•0•Updated Jul 7, 2025Jul 7, 2025

• sentence-embedding-bento

  Public archive
  Sentence Embedding as a Service

  Jupyter Notebook

  •

  Apache License 2.0

  •3•15•0•1•Updated Jun 30, 2025Jun 30, 2025

• Pneumonia-Detection-Demo

  Public
  Pneumonia Detection - Healthcare Imaging Application built with BentoML and fine-tuned Vision Transformer (ViT) model

  computer-visionhealthcaretransformer+ 2health-care-applicationhealthcare-imaging

  Python

  •2•8•1•1•Updated Jun 17, 2025Jun 17, 2025

• BentoGradio

  Public
  Python

  •0•0•0•0•Updated Jun 13, 2025Jun 13, 2025