Repository navigation
AI Verify Evals Toolkit Installation Guide
This guide walks you through installation and starting AI Verify Evals Toolkit.
- The first section is for users using Docker Desktop.
- The second section is for developers to install and run the Toolkit from source code using Poetry.
| Requirement | Details |
|---|---|
| Docker | Docker Desktop (macOS/Windows). The Docker engine must be running to run Moonshot. |
| OS | macOS or Linux (Ubuntu 22.04 LTS+ recommended). Windows: use WSL2 with Docker Desktop. |
| Disk space | At least 3 GB of free disk space for Docker images and data storage. |
| RAM | 8 GB or more recommended. |
| Network | Required for downloading Moonshot and connecting to LLM inference providers or apps. |
| Ports | Moonshot uses port 8000 by default. You can configure Docker to map to another port if 8000 is not available. |
If your laptop/PC doesn't meet the requirements (e.g. no admin rights, no virtualisation support) to install Docker, or cannot install due to a restriction placed by your organisation, you can install and run Moonshot from source code instead.
Go to Docker Desktop for downloading and installation. After installation, open the Docker Desktop application.
Docker Desktop may prompt a welcome panel for login/signup — click the "Skip" button (top-right), as Moonshot doesn't require an account to run.
Click on the lower-right terminal icon to open a terminal panel with a new terminal session.
Paste the following command in the terminal and press enter to run. This command will download and run the latest Moonshot Docker image and make it accessible via localhost:8000.
docker run --detach --name moonshot-web --platform linux/amd64 -p 8000:8000 -v moonshot-cicd-data-volume:/var/lib/moonshot -e MOONSHOT_DB_PATH=/var/lib/moonshot/moonshot.db -e MOONSHOT_BENCHMARK_RESULTS_DIR=/var/lib/moonshot/results ghcr.io/aiverify-foundation/moonshot:latest moonshot-web
Once the command is done, it will print the text Status: Downloaded newer image for ghcr.io/aiverify-foundation/moonshot:latest in the terminal, and Docker will create a container named moonshot-web, listed under the Containers tab.
Ensure the container is running (green status icon) and then click on 8000:8000 to open a new browser tab and load the Moonshot homepage (via localhost:8000).
Refer to Releases or Moonshot Docker Images to check if there is a new Moonshot version. Open a new terminal session in Docker Desktop and run the following command:
docker pull ghcr.io/aiverify-foundation/moonshot:latest --platform linux/amd64This command will download the latest version of the image. If there is no newer version, it will display "Status: Image is up to date for…".
Ensure all tests on Moonshot are completed before proceeding.
If there is a newer version of the Moonshot Docker image and you intend to upgrade, proceed to delete the existing moonshot-web container by clicking on the trash can icon.
Deleting the moonshot-web container will not lose data, as the data is stored separately in the moonshot-cicd-data-volume Docker volume (available under the Volumes tab in Docker Desktop). The newer version of the Moonshot app will also connect to moonshot-cicd-data-volume to retrieve and store data.
Why is it required to delete the existing moonshot-web container?
-
The
docker runcommand reuses the same container name, which is not allowed while an existing container with that name exists. -
While a different container name can be assigned, it is not recommended to keep and reuse an older version of Moonshot after upgrading. This is because the new version of Moonshot may make changes to the data store that are not backward compatible.
Run the command below (same as the installation step) to create a new moonshot-web container based on the latest Moonshot Docker image.
docker run --detach --name moonshot-web --platform linux/amd64 -p 8000:8000 -v moonshot-cicd-data-volume:/var/lib/moonshot -e MOONSHOT_DB_PATH=/var/lib/moonshot/moonshot.db -e MOONSHOT_BENCHMARK_RESULTS_DIR=/var/lib/moonshot/results ghcr.io/aiverify-foundation/moonshot:latest moonshot-webNote: The following installation guide is for installing and running Moonshot from source code.
Before installing Project Moonshot, ensure your environment meets the following requirements:
- Python: Version 3.12 is required
- Operating System: macOS or Linux (Ubuntu 22.04 LTS or later recommended). Windows users should leverage WSL2.
- Package Manager: pip (updated to the latest version).
- Poetry
- Git
- NPM
- Node: Version 22
First, clone the Project Moonshot repository from GitHub to your local machine or build server:
git clone https://github.com/aiverify-foundation/moonshot-cicd.git
cd moonshot-cicdWe highly recommend using a virtual environment to avoid dependency conflicts with other Python packages:
# Create a virtual environment named 'moonshot-env'
python3 -m venv moonshot-env
# Activate the virtual environment
# On macOS/Linux:
source moonshot-env/bin/activateInstall the required packages and the Moonshot framework in editable mode:
# Ensure your current folder is in moonshot-cicd
cd moonshot_core
poetry installInstall for NodeJS to work:
cd ..
cd moonshot_portal_app
npm installTo run the server, you need to start the Python application and the NodeJS web server.
- Locate the
moonshot-cicddirectory. - Make sure your virtual environment is activated.
- Start the servers with the below commands:
cd moonshot_core
python run_api.py
cd ..
cd moonshot_portal_app
npm run devThis will start the server at port 3000 on your local machine, and you can access the Moonshot web UI at http://localhost:3000/.
👍 Success: If the installation was successful, you will see the landing page.
- AI Verify Evals Toolkit Installation Guide
- AI Verify Evals Toolkit User Guide
- AI Verify Evals Toolkit Developer Guide
- Onboarding Guide for CI/CD Deployment
- Basic User Guide - Moonshot Configurations
- Basic User Guide - Running Moonshot Tests
- Advanced User Guide - Create Custom Connectors for your Application Endpoint
- Advanced User Guide - Create Custom Evaluation Metrics for your Tests
- Expert User Guide - Create Custom Moonshot Tests