Skip to content
@bentoml

BentoML

The easiest way to run AI Inference in the cloud

Welcome to BentoML 👋 Twitter Follow Slack

BentoML

What is BentoML? 👩‍🍳

BentoML is an open-source model serving library for building model inference APIs and multi-model serving systems with any open-source or custom AI models. It comes with everything you need for serving optimization, model packaging, and simplifies production deployment via ☁️ BentoCloud.

Get in touch 💬

👉 Join our Slack community!

👀 Follow us on X @bentomlai and LinkedIn

📖 Read our blog

Pinned Loading

  1. BentoML BentoML Public

    The easiest way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Multi-model Inference Graph/Pipelines, LLM/RAG apps, and more!

    Python 6.8k 767

  2. OpenLLM OpenLLM Public

    Run any open-source LLMs, such as Llama 3.1, Gemma, as OpenAI compatible API endpoint in the cloud.

    Python 9.4k 598

Repositories

Showing 10 of 81 repositories
  • BentoVLLM Public

    Self-host LLMs with vLLM and BentoML

    bentoml/BentoVLLM’s past year of commit activity
    Python 40 9 4 0 Updated Jul 24, 2024
  • llama_index Public Forked from run-llama/llama_index

    LlamaIndex (GPT Index) is a data framework for your LLM applications

    bentoml/llama_index’s past year of commit activity
    Python 1 MIT 4,858 0 0 Updated Jul 24, 2024
  • langchain Public Forked from langchain-ai/langchain

    ⚡ Building applications with LLMs through composability ⚡

    bentoml/langchain’s past year of commit activity
    Jupyter Notebook 4 MIT 14,553 0 0 Updated Jul 24, 2024
  • BentoML Public

    The easiest way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Multi-model Inference Graph/Pipelines, LLM/RAG apps, and more!

    bentoml/BentoML’s past year of commit activity
    Python 6,829 Apache-2.0 767 152 12 Updated Jul 24, 2024
  • bentoml/openllm-models’s past year of commit activity
    HTML 5 2 0 1 Updated Jul 24, 2024
  • OpenLLM Public

    Run any open-source LLMs, such as Llama 3.1, Gemma, as OpenAI compatible API endpoint in the cloud.

    bentoml/OpenLLM’s past year of commit activity
    Python 9,406 Apache-2.0 598 22 1 Updated Jul 24, 2024
  • BentoLMDeploy Public

    Self-host LLMs with LMDeploy and BentoML

    bentoml/BentoLMDeploy’s past year of commit activity
    Python 12 1 1 0 Updated Jul 23, 2024
  • yatai-image-builder Public

    🐳 Build OCI images for Bentos in k8s

    bentoml/yatai-image-builder’s past year of commit activity
    Go 14 9 4 5 Updated Jul 23, 2024
  • bentoml/BentoMoirai’s past year of commit activity
    Python 1 0 0 0 Updated Jul 18, 2024
  • bentoml/BentoResnet’s past year of commit activity
    Python 1 0 0 0 Updated Jul 18, 2024