The most flexible way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Inference Graph/Pipelines, Compound AI systems, Multi-Modal, RAG as a Service, and more!

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, FEDML Nexus AI (https://fedml.ai) is your generative AI platform at scale.

Boosting your Web Services of Deep Learning Applications.

Python + Inference - Model Deployment library in Python. Simplest model inference server ever.

Cognita by TrueFoundry - Framework for building modular, open source RAG applications for production.

A multi-functional library for full-stack Deep Learning. Simplifies Model Building, API development, and Model Deployment.

Starter app for fastai v3 model deployment on Render

A Beautiful Flask Web API for Yolov7 (and custom) models

Fast model deployment on any cloud 🚀

🤖 An automated machine learning framework for audio, text, image, video, or .CSV files (50+ featurizers and 15+ model trainers). Python 3.6 required.

BentoML Example Projects 🎨

Deploy DL/ ML inference pipelines with minimal extra code.

The official python package for NimbleBox. Exposes all APIs as CLIs and contains modules to make ML 🌸

gRPC server for hosting ML models trained on any framework in python

'Deploying machine learning models with a Flask API' tutorial, written for HyperionDev

Turn any OCR models into online inference API endpoint 🚀 🌖

Online Inference API for NLP Transformer models - summarization, text classification, sentiment analysis and more

Experiments with Model Training, Deployment & Monitoring

mlserve turns your python models into RESTful API, serves web page with form generated to match your input data.

ML container made simple