An endpoint server for efficiently serving quantized open-source LLMs for code.
-
Updated
Oct 15, 2023 - Python
An endpoint server for efficiently serving quantized open-source LLMs for code.
EchoSight is a tool that helps visually impaired individuals by audibly describing images taken with a Raspberry Pi Camera or inputted via image path or URL across different operating systems.
MLOps library for LLM deployment w/ the vLLM engine on RunPod's infra.
An simple implementation of Unet because all the implementations i've seen are wayy tooo complicated.
A discord bot which can call LLMs using either Hugging Face or vLLM on Windows platform. Combined with function calling.
A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
AI-Learning-Platform, a LLM-RAG pipeline which behaves like a guide and able to solve doubts. Deployed on-premise IBM ppc64le architecture. vLLM for model inference & Qdrant with Langchain for RAG Pipeline. Server written in django, postgres & cassandra as the sql & nosql databases.
This repository has a lot of LLM projects done. It is the best place to start learning LLM.
大模型推理框架加速,让 LLM 飞起来
Evaluate open-source language models on Agent, formatted output, command following, long text, multilingual, coding, and custom task capabilities. 开源语言模型在Agent,格式化输出,指令追随,长文本,多语言,代码,自定义任务的能力基准测试。
Add a description, image, and links to the vllm topic page so that developers can more easily learn about it.
To associate your repository with the vllm topic, visit your repo's landing page and select "manage topics."