A fast, easy-to-use, production-ready inference server for computer vision supporting deployment of many popular model architectures and fine-tuned models.
-
Updated
Jun 28, 2024 - Python
A fast, easy-to-use, production-ready inference server for computer vision supporting deployment of many popular model architectures and fine-tuned models.
The simplest way to serve AI/ML models in production
Python + Inference - Model Deployment library in Python. Simplest model inference server ever.
This is a repository for an nocode object detection inference API using the Yolov3 and Yolov4 Darknet framework.
This is a repository for an nocode object detection inference API using the Yolov4 and Yolov3 Opencv.
This is a repository for an object detection inference API using the Tensorflow framework.
Deploy DL/ ML inference pipelines with minimal extra code.
Friendli: the fastest serving engine for generative AI
Advanced inference pipeline using NVIDIA Triton Inference Server for CRAFT Text detection (Pytorch), included converter from Pytorch -> ONNX -> TensorRT, Inference pipelines (TensorRT, Triton server - multi-format). Supported model format for Triton inference: TensorRT engine, Torchscript, ONNX
Benchmark for machine learning model online serving (LLM, embedding, Stable-Diffusion, Whisper)
Inference Server Implementation from Scratch for Machine Learning Models
Session Based Real-time Hotel Recommendation Web Application
A networked inference server for Whisper so you don't have to keep waiting for the audio model to reload for the x-hunderdth time.
Roboflow's inference server to analyze video streams. This project extracts insights from video frames at defined intervals and generates informative visualizations and CSV outputs.
Serve pytorch inference requests using batching with redis for faster performance.
Vision and vision-multi-modal components for geniusrise framework
Text components powering LLMs & SLMs for geniusrise framework
Run your own production inference code with Sagemaker
Audio components for geniusrise framework
Add a description, image, and links to the inference-server topic page so that developers can more easily learn about it.
To associate your repository with the inference-server topic, visit your repo's landing page and select "manage topics."