Skip to content

AI Verify Evals Toolkit User Guide

Harry Zhao edited this page Sep 22, 2026 · 5 revisions

Run Benchmark Testing via AI Verify Evals Toolkit


About This Guide

This guide provides a step-by-step walkthrough of how to run a safety benchmark test using AI Verify Evals Toolkit. Each section corresponds to a screen or step in the application and includes annotated screenshots for reference.

By the end of this guide, you will be able to:

  • Understand and select the appropriate benchmark bundles and individual tests for your use case
  • Configure the LLM model or application you want to evaluate
  • Connect the required evaluator endpoints for LLM-as-judge evaluation
  • Configure the statistical sample size for your test
  • Run a benchmark and monitor its progress
  • Review the benchmark report, including model responses and evaluator verdicts
  • Manually review and override evaluator verdicts where necessary
  • Download the test report for further analysis or sharing

Singapore Benchmarks used as Example

AI Verify Evals Toolkit includes two benchmark test bundles designed specifically for Singapore's context:

  • Singapore Factuality
  • Singapore Undesirable Content

To support product discovery and evaluation, 10% of the Singapore benchmark dataset is currently available on GitHub.

This guide walks you through how to run these benchmarks against your LLM application or model. We welcome your feedback as you explore the benchmarks and results.


1. The Home Page

When you first open the Toolkit in your browser at http://localhost:8000, you will see the Home Page.

AI Verify Evals Toolkit Home Page

The Home Page provides three main entry points:

Card What it does Use it when
Run a benchmark test Launches the benchmark test workflow Starting a new evaluation
How-to guide Provides a step-by-step product walkthrough Learning how Moonshot works
IMDA's Starter Kit Opens IMDA's official safety testing guidance Reviewing recommended safety testing practices

Action: Click Run a benchmark test to begin.


2. Selecting Tests — Bundle Selection Screen

After clicking Run a benchmark test, you will arrive at the Bundle Selection screen. This is where you choose the safety tests you want to run against your LLM or application.

2.1 Understanding the Layout

The screen has two main areas:

  • Main area: Displays test bundles grouped by category, such as Adversarial Prompts and Undesirable Content. You can select or deselect bundles here.
  • Selected Tests panel: Displays a live summary of your selections, organised by bundle. The counter at the top—for example, 13/13—shows how many individual tests are selected out of the total available.
Bundle Selection screen with right-hand summary panel

2.2 What Is a Bundle?

A bundle is a collection of related safety tests grouped together for a particular evaluation purpose.

For the Singapore benchmarks, there are two bundles:

  • Facts in Singapore
  • Singapore Undesirable Content

Each bundle card displays:

  • The number of tests and total prompts included
  • A list of the individual tests included in the bundle
  • A Select / Selected button
  • A Learn more link

2.3 Selecting a Bundle

To select a bundle:

  1. Find the bundle you want to evaluate.
  2. Click the white Select button.
  3. The button changes to Selected.
  4. The Selected Tests panel updates automatically.

💡 Tip: You can select multiple bundles. The panel on the right keeps track of all your selections.


2.4 Viewing Bundle Details

Bundle detail modal for Singapore Undesirable Content

Click Learn more on a bundle card to open its detail modal.

The modal shows:

  • A description of what the bundle measures
  • The Required Endpoint Connectors, such as TogetherAI, used to evaluate model responses
  • All individual tests included in the bundle
  • A description and prompt count for each test
  • A Select button for each individual test

💡 Tip: You can select individual tests from the bundle detail modal instead of selecting the entire bundle. This is useful when you only need a subset of tests.


2.5 Understanding Individual Tests

Individual test detail page for Singapore - Violent Content

Click Learn More on an individual test to open its detail page.

The page shows:

  • A plain-language description of what the test measures
  • Dataset Information: The number of prompts in the test dataset
  • Evaluator Information: The evaluation logic used to assess responses, such as llamaguardannotator_adapter
  • Sample Results: A sample of test inputs, expected targets, model responses, and evaluator verdicts

The sample results are intended to help you understand what the model will be asked and how responses are evaluated before running the test.


2.6 Managing the Selected Tests Panel

Expanded right panel showing all individual tests under Undesirable Content

In the Selected Tests panel, click the expand arrow (▼) next to a bundle to view its individual tests.

Each test includes:

  • A checkbox to select or deselect the test
  • An expand arrow (▼) to view its description and prompt count

You can deselect individual tests without removing the entire bundle.

📝 Note: The bundle counter updates automatically as you select or deselect tests. For example, Undesirable Content [8/12] means that 8 of the 12 tests in the bundle are currently selected.


2.7 Proceeding to Configuration

Once you are satisfied with your test selections, click Configure and Run Benchmark Tests in the bottom-right corner.

This takes you to the test configuration screen.


3. Configuring and Running Tests

The Configure and Run Tests screen is a single scrollable page divided into four collapsible sections.

A green tick (✓) indicates that a section has been completed. A red warning icon (⚠) indicates that the section still requires attention.


3.1 Section A — Enter Test Name

Enter Test Name

The first section asks you to provide:

  • Test Name (required): A descriptive name for your benchmark run

💡 Tip: Use a descriptive test name so you can easily identify the run later on the History page.


3.2 Section B — Select the App or Model

This section defines the LLM or application you want to evaluate.

3.2.1 Choose a Provider

LLM Provider dropdown showing available providers

Click the LLM Provider/Application Connector dropdown to view the available options.

You can choose from:

  • Model Providers: Pre-integrated model providers such as OpenAI or OpenRouter
  • Application Connections: Applications that you have already configured, or can configure as a new connector

3.2.2 Choose a Model

image

After selecting a provider, click the Model dropdown.

The list displays models already configured for that provider.

You can also:

  • Search for a model using the search box
  • Click + Add New Model Configuration to configure a model that is not yet available

3.2.3 Add a New Model Configuration

image

When adding a model, provide the following information:

  • Model Configuration Name: A human-readable name for this configuration
  • Model Provider: Automatically populated based on your provider selection
  • Token (optional): Your API key for the provider. If a key has already been saved, leave this field blank to keep the existing key
  • Model: The provider-specific model ID, such as meta-llama/Meta-Llama-3-70B-Instruct
  • Advanced Parameters: Additional inference settings, such as temperature

Click Test Connection to verify the connection, then click Save.

📝 Note: The Token field is masked for security. If a token has already been saved, Moonshot displays a message indicating that no further action is required unless you want to replace it.


3.2.4 Add a New Application Configuration

To evaluate an application instead of a directly accessible model:

  1. Select API Connector from the LLM Provider/Application Connector dropdown.
  2. Click the Application Configuration dropdown.
  3. Select an existing application or choose Add New App Configuration.
image

3.2.5 Configure a New Application

image

Provide the following information:

  • Configuration Name: A human-readable name for the application configuration
  • API Type: Select POST, PUT, or FETCH
  • URL: The URL endpoint of your application
  • Requested Body: The request body sent to your application. Include {{prompt}} where the benchmark prompt should be inserted.
image

Click Test Connection to verify the connection.

A successful connection will display Connection succeeded (HTTP 200).

Next, scroll down in the Path/Value panel and select the response field that contains the model's answer.

For example:

choices[0].message.content

Click Save to confirm the configuration.


3.3 Section C — Connect Required Evaluator Endpoints

image

Some tests use LLM-as-judge, where a second LLM evaluates the response generated by your primary model.

This section shows the endpoint connectors required to run the selected tests.

For each required connector:

  1. Click Connect on the connector card.
  2. Enter the API token for the provider.
  3. Click Test to verify the connection.
  4. If the test succeeds, a green Test Passed badge will appear.
  5. Click Save.
  6. The connector card will show a green Connected status.

3.4 Section D — Configure Sample Size

image

This section determines how many prompts from each test dataset will be sent to your model.

Running fewer prompts reduces the amount of information available for estimating the model's score. Moonshot uses three parameters to calculate the required sample size.

Expected Test Score

The expected proportion of prompts that will receive a safe verdict.

Available options are:

50%, 60%, 70%, 80%, 90%, 95%

The default is 90%, which assumes that the model is expected to handle most prompts safely.

Confidence Level

The confidence level used to construct the confidence interval around the estimated score.

Available options range from 85% to 99%.

The default is 95%.

Margin of Error

The desired precision of the estimated score.

Available options are:

1%, 2%, 3%, 4%, 5%

The default is 3%.

For example, a measured score of 90% with a ±3 percentage-point margin of error corresponds to an interval of approximately 87%–93%, subject to the statistical assumptions used by the calculation.

Parameter Recommended Increasing the value Decreasing the value
Expected Test Score 90% Generally fewer samples needed Generally more samples needed
Confidence Level 95% More samples needed Fewer samples needed
Margin of Error 3% Fewer samples needed More samples needed

Choosing the Number of Prompts

Below the statistical parameters, you can choose between:

  • Calculated: Moonshot calculates a sample size based on the parameters above
  • All prompts: Runs every prompt available in the selected tests

💡 Recommendation for the Singapore benchmark: The currently released dataset represents only 10% of the full benchmark and is relatively small. We therefore recommend selecting All prompts to maximise the amount of available evaluation data.


3.5 Running the Tests

Once all four sections show a green tick (✓), click Run Benchmark Tests.

Moonshot will begin sending the benchmark prompts to your configured model or application.

You will then be taken to the History page, where you can monitor the progress of your test run.


4. Monitoring Test Runs — History Page

image

The History page is accessible through the clock icon in the left navigation bar.

It displays your benchmark runs as cards. Each card includes:

  • Test name and start date/time
  • Number of bundles and tests included
  • Status: In Progress or Complete
  • A status message, such as Run in progress or Completed in X min

Click a test run to open its progress tracker.

image

The progress tracker shows the status of the individual tests included in the run.


5. Reading the Results Report

The Report page provides the detailed results of your benchmark run, including the evaluator's assessment of the model's responses.

5.1 Report Header

At the top of the report, you will see:

  • Test name and current status
  • Run number, such as Run #6
  • Endpoint type and name
  • Total number of prompts evaluated
  • Download button for exporting the report

Each new run of the same configuration receives a new run number.


5.2 Overview Tab

image

The Overview tab provides a high-level summary of the selected bundles.

Each bundle displays its overall score—for example, 83% for Singapore Undesirable Content.

Below the bundle-level score, each individual test is displayed as a horizontal bar showing:

  • The test score
  • The 95% confidence interval

The score represents the percentage of evaluated prompts that received a safe verdict.


5.3 Bundle-Specific Results

image

Click a bundle tab, such as Singapore Undesirable Content, to view its detailed results.

The summary metrics include:

Metric Meaning
Evaluator Score Percentage of prompts classified as safe by the automated evaluator
Reviewed Score Score after incorporating manual reviewer overrides
Evaluation reviewed Number of prompts manually reviewed
Evaluation adjusted Number of evaluator verdicts changed by a reviewer
Safe → Unsafe Prompts originally classified as safe that were changed to unsafe
Unsafe → Safe Prompts originally classified as unsafe that were changed to safe

Per-Test Bar Chart

The chart displays two bars for each test:

  • Evaluator Score: The original score produced by the automated evaluator
  • Reviewed Score: The score after applying any manual overrides

Hover over a bar to view:

  • The test score
  • The 95% confidence interval
  • The reviewed score, where applicable

The error bars represent the statistical uncertainty associated with the estimated score.


5.4 Reviewing Individual Prompt Results

Below the chart, a table displays the evaluated prompts.

The table includes:

  • Test: The test the prompt belongs to
  • Prompt: The input sent to the model
  • Target: The expected target or ground truth, where applicable
  • Response: The model's response; click to expand
  • Evaluation: The evaluator's verdict — Safe or Unsafe
  • Your Agreement with Evaluation: Indicate whether you agree with the evaluator using 👍 or 👎
  • Note: Add a free-text annotation

Filtering Results

Use the filters at the top of the table to focus on specific results:

  • Bundle: Filter by bundle
  • Evaluation: Show only safe or unsafe verdicts
  • Your Agreement with Evaluation: Filter by reviewed/unreviewed or agree/disagree
  • Adjusted / Not Adjusted: Show prompts where the evaluator verdict was changed
  • Search: Search across the available table fields

The counts displayed next to filter options update based on the current selection.

Reviewing Prompt Details

Click any row in the results table to open the Prompt Detail panel on the right.

image

The panel provides an expanded view of the selected prompt, including:

  • Bundle and test name
  • Full prompt text
  • Target or ground truth
  • Full model response
  • Evaluator verdict
  • Your agreement with the evaluator
  • Notes

Use the Previous and Next buttons at the bottom of the panel to move between prompts without closing the panel.


5.5 Downloading the Report

Click Download in the report header to export the full test results.

The downloaded report can be used for further analysis, shared with your team, or retained as part of your evaluation records.

image

Quick Reference — End-to-End Workflow

# Step What to do
1 Open Moonshot Navigate to http://localhost:8000
2 Start a test Click Run a benchmark test
3 Select tests Select the required bundles and individual tests
4 Review test details Click Learn more to understand what each test measures
5 Configure the test Click Configure and Run Benchmark Tests
6 Name your test Enter a descriptive test name
7 Choose model/app Select an LLM provider and model, or configure an application
8 Connect evaluators Configure the required LLM-as-judge endpoints
9 Set sample size Use the recommended defaults or select All prompts
10 Run benchmark Click Run Benchmark Tests
11 Monitor progress Track the run from the History page
12 Review results Examine scores, confidence intervals, responses, and evaluator verdicts
13 Review verdicts Indicate your agreement and override evaluator verdicts where appropriate
14 Export results Click Download to export the report

Glossary

Term Definition
Bundle A collection of related safety tests grouped together for an evaluation purpose
LLM-as-judge A language model used to automatically evaluate another model's responses
Evaluator Score The percentage of prompts classified as safe by the automated evaluator
Reviewed Score The score after incorporating manual reviewer overrides
Confidence Interval A statistical interval used to express uncertainty around an estimated population score
Margin of Error The desired level of precision around the estimated score
Expected Test Score The assumed proportion of safe responses used in the sample-size calculation
Prompt Injection An adversarial technique in which instructions embedded in user input attempt to alter or hijack a model's behaviour
MLCommons AILuminate An open safety evaluation framework from MLCommons that provides standardised safety evaluation datasets
IMDA Singapore's Infocomm Media Development Authority, which provides guidance including the IMDA Starter Kit

Clone this wiki locally