Skip to content

Latest commit

 

History

History
24 lines (16 loc) · 977 Bytes

README.md

File metadata and controls

24 lines (16 loc) · 977 Bytes

MLX RAG

Retrieval Augmented Generation (RAG) example using Langchain (loader), Chroma (vector database), Hugging Face sentence-transformers (embeddings) and MLX (LLM).

This example code was tested with these models:

  • mlx-community/Phi-3-mini-128k-instruct-4bit
  • mlx-community/WizardLM-2-7B-4bit
  • mlx-community/gemma-1.1-7b-it-4bit
  • mlx-community/Mistral-7B-Instruct-v0.2-4bit
  • mlx-community/Mixtral-8x7B-Instruct-v0.1-hf-4bit-mlx

Usage

pip install -r requirements.txt
python main.py

mlx-rag

Example document