Llama-v2-Chat-App

This project leverages Llama v2 Chat models for a Chatbot Application

Description

This project was inspired on: The FASTEST way to build CHAT UI for LLAMA-v2 by Abhishek Thakur

This Chat Application include the following features:

• Llama.cpp.
• Memory.
• Inference API for Text Generation.

All in a containerized maner.

Results

Chatbot:

API:

Usage

Build APP and API Images

docker compose build

Get everything up and running

docker compose down && docker compose up -d

Have fun

Visit: http://localhost:7861/ to access the Gradio Chatbot UI.

To learn more about the inference API, please visit: http://localhost:5001/swagger

Contributing

Installing pre-commit

Pre-commit is already part of this project dependencies. If you would like to installed it as standalone run:

pip install pre-commit

To activate pre-commit run the following commands:

• Install Git hooks:

pre-commit install

• Update current hooks:

pre-commit autoupdate

To test your installation of pre-commit run:

pre-commit run --all-files