Local Microsoft Phi-2

Build & Run

• Clone the Repository:

git clone https://github.com/abughalib/phi2local

• Download the model, either the Dolphin 2.0 or Microsoft Phi2

• Add model path to environment Variable:

export QUANTIZED_MODEL_DIR=/your/model/path

• Build the code

cd phi2local; cargo build --release

• Run.

./target/release/phi2local

Build Dependencies

• Install Rust from Rustup.rs

Windows

• Microsoft Visual Studio Build Tools
• Clang Compiler

Debian based Linux

• Build Packages

sudo apt install build-essential clang