A Python script designed to streamline the process of quantizing models to exllamav2 format

A constrained generation filter for local LLMs that makes them quote properly from a source document

This is a playground to explore the ExLlama project in a Windows environment.

Run gguf LLM models in Latest Version TextGen-webui

A.L.I.C.E (Artificial Labile Intelligence Cybernated Existence). A REST API of A.I companion for creating more complex system

A Simple webserver for generating text with exllamav2

Simple LLM inference server

A QT GUI for large language models

A Qt GUI for large language models

LLM telegram bot

An OpenAI-like LLaMA inference API

Booster - open accelerator for LLM models. Better inference and debugging for AI hackers