1. Installation

Installation

Aphrodite is primarily a python library, but also contains CUDA C/C++ code that needs to be compiled. We provide pre-compiled binaries in a wheel, so users don't have to deal with the compile times.

Requirements

1. Linux, Windows (WSL)
2. Python 3.8 - 3.11
3. NVIDIA (compute capability > 6.0) and AMD (MI200+)

Pre-compiled binaries via PyPi

For the pip package, all you need is NVIDIA drivers.

$ pip install aphrodite-engine

Build from source

The embedded runtime sets up micromamba and downloads CUDA 11.8 along with all the required dependencies.

$ git clone https://github.com/PygmalionAI/aphrodite-engine.git && cd aphrodite-engine
$ ./update-runtime.sh  # this can take a while

Alternatively, if you already have an environment with CUDA 11.8, you can simply run the following to build aphrodite-engine:

$ pip install -e .  # this can take a while