This is a simple command line chat using MLX, mlx_lm to generate and display text from a local model.

Use HuggingFace.co to download a model. This implementation makes use of Safetensors, so be sure the model has .safetensors included.

e.g. to use Mistral 7B Instruct:

git lfs install

git clone https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2

---

Place your llm in the models folder. The folder should look like this for mistral:

-localchat

---->mlxfiles

---->models

\\--->Mistral-7B-Instruct-v0.2

\\\\\--->config.json

\\\\\--->generation_config.json

\\\\\---> etc...

codechat.py

requirements.txt

---

Using the terminal:

pip install mlx

cd localchat

pip install -r requirements.txt

python codechat.py