Skip to content

AI Verify Evals Toolkit Installation Guide

Harry Zhao edited this page Sep 22, 2026 · 3 revisions

AI Verify Evals Toolkit Installation Guide

This guide walks you through installation and starting AI Verify Evals Toolkit.

  • The first section is for users using Docker Desktop.
  • The second section is for developers to install and run the Toolkit from source code using Poetry.

AI Verify Evals Toolkit Installation Guide (Docker)

Prerequisites

Requirement Details
Docker Docker Desktop (macOS/Windows). The Docker engine must be running to run Moonshot.
OS macOS or Linux (Ubuntu 22.04 LTS+ recommended). Windows: use WSL2 with Docker Desktop.
Disk space At least 3 GB of free disk space for Docker images and data storage.
RAM 8 GB or more recommended.
Network Required for downloading Moonshot and connecting to LLM inference providers or apps.
Ports Moonshot uses port 8000 by default. You can configure Docker to map to another port if 8000 is not available.

Step-by-Step Installation

Step 1 – Install Docker Desktop

If your laptop/PC doesn't meet the requirements (e.g. no admin rights, no virtualisation support) to install Docker, or cannot install due to a restriction placed by your organisation, you can install and run Moonshot from source code instead.

Go to Docker Desktop for downloading and installation. After installation, open the Docker Desktop application.

image

Docker Desktop may prompt a welcome panel for login/signup — click the "Skip" button (top-right), as Moonshot doesn't require an account to run.

image

Step 2 – Run Moonshot in terminal

Click on the lower-right terminal icon to open a terminal panel with a new terminal session.

image image

Paste the following command in the terminal and press enter to run. This command will download and run the latest Moonshot Docker image and make it accessible via localhost:8000.

docker run --detach --name moonshot-web --platform linux/amd64 -p 8000:8000 -v moonshot-cicd-data-volume:/var/lib/moonshot -e MOONSHOT_DB_PATH=/var/lib/moonshot/moonshot.db -e MOONSHOT_BENCHMARK_RESULTS_DIR=/var/lib/moonshot/results ghcr.io/aiverify-foundation/moonshot:latest moonshot-web
image

Once the command is done, it will print the text Status: Downloaded newer image for ghcr.io/aiverify-foundation/moonshot:latest in the terminal, and Docker will create a container named moonshot-web, listed under the Containers tab.

image

Ensure the container is running (green status icon) and then click on 8000:8000 to open a new browser tab and load the Moonshot homepage (via localhost:8000).

image image

Upgrade to the Latest Version

Step 1: Download the latest Moonshot Docker image

Refer to Releases or Moonshot Docker Images to check if there is a new Moonshot version. Open a new terminal session in Docker Desktop and run the following command:

docker pull ghcr.io/aiverify-foundation/moonshot:latest --platform linux/amd64

This command will download the latest version of the image. If there is no newer version, it will display "Status: Image is up to date for…".

image

Step 2: Stop and remove the existing moonshot-web container

Ensure all tests on Moonshot are completed before proceeding.

If there is a newer version of the Moonshot Docker image and you intend to upgrade, proceed to delete the existing moonshot-web container by clicking on the trash can icon.

image

Deleting the moonshot-web container will not lose data, as the data is stored separately in the moonshot-cicd-data-volume Docker volume (available under the Volumes tab in Docker Desktop). The newer version of the Moonshot app will also connect to moonshot-cicd-data-volume to retrieve and store data.

Why is it required to delete the existing moonshot-web container?

  1. The docker run command reuses the same container name, which is not allowed while an existing container with that name exists.

  2. While a different container name can be assigned, it is not recommended to keep and reuse an older version of Moonshot after upgrading. This is because the new version of Moonshot may make changes to the data store that are not backward compatible.

Step 3: Run Moonshot with the latest version

Run the command below (same as the installation step) to create a new moonshot-web container based on the latest Moonshot Docker image.

docker run --detach --name moonshot-web --platform linux/amd64 -p 8000:8000 -v moonshot-cicd-data-volume:/var/lib/moonshot -e MOONSHOT_DB_PATH=/var/lib/moonshot/moonshot.db -e MOONSHOT_BENCHMARK_RESULTS_DIR=/var/lib/moonshot/results ghcr.io/aiverify-foundation/moonshot:latest moonshot-web

AI Verify Evals Toolkit Installation Guide (Poetry)

Note: The following installation guide is for installing and running Moonshot from source code.

📋 Prerequisites

Before installing Project Moonshot, ensure your environment meets the following requirements:

🚧 System Requirements

  • Python: Version 3.12 is required
  • Operating System: macOS or Linux (Ubuntu 22.04 LTS or later recommended). Windows users should leverage WSL2.
  • Package Manager: pip (updated to the latest version).
  • Poetry
  • Git
  • NPM
  • Node: Version 22

🛠️ Step-by-Step Installation

Step 1: Clone the Repository

First, clone the Project Moonshot repository from GitHub to your local machine or build server:

git clone https://github.com/aiverify-foundation/moonshot-cicd.git
cd moonshot-cicd

Step 2: Set Up a Virtual Environment

We highly recommend using a virtual environment to avoid dependency conflicts with other Python packages:

# Create a virtual environment named 'moonshot-env'
python3 -m venv moonshot-env

# Activate the virtual environment
# On macOS/Linux:
source moonshot-env/bin/activate

Step 3: Install Dependencies

Install the required packages and the Moonshot framework in editable mode:

# Ensure your current folder is in moonshot-cicd
cd moonshot_core
poetry install

Install for NodeJS to work:

cd ..
cd moonshot_portal_app
npm install

🚀 Start Server

To run the server, you need to start the Python application and the NodeJS web server.

  1. Locate the moonshot-cicd directory.
  2. Make sure your virtual environment is activated.
  3. Start the servers with the below commands:
cd moonshot_core
python run_api.py

cd ..
cd moonshot_portal_app
npm run dev

✅ Verifying the Installation

This will start the server at port 3000 on your local machine, and you can access the Moonshot web UI at http://localhost:3000/.

👍 Success: If the installation was successful, you will see the landing page.

Clone this wiki locally