Repository navigation
AI Verify Evals Toolkit User Guide
This guide provides a step-by-step walkthrough of how to run a safety benchmark test using AI Verify Evals Toolkit. Each section corresponds to a screen or step in the application and includes annotated screenshots for reference.
By the end of this guide, you will be able to:
- Understand and select the appropriate benchmark bundles and individual tests for your use case
- Configure the LLM model or application you want to evaluate
- Connect the required evaluator endpoints for LLM-as-judge evaluation
- Configure the statistical sample size for your test
- Run a benchmark and monitor its progress
- Review the benchmark report, including model responses and evaluator verdicts
- Manually review and override evaluator verdicts where necessary
- Download the test report for further analysis or sharing
AI Verify Evals Toolkit includes two benchmark test bundles designed specifically for Singapore's context:
- Singapore Factuality
- Singapore Undesirable Content
To support product discovery and evaluation, 10% of the Singapore benchmark dataset is currently available on GitHub.
This guide walks you through how to run these benchmarks against your LLM application or model. We welcome your feedback as you explore the benchmarks and results.
When you first open the Toolkit in your browser at http://localhost:8000, you will see the Home Page.
The Home Page provides three main entry points:
| Card | What it does | Use it when |
|---|---|---|
| Run a benchmark test | Launches the benchmark test workflow | Starting a new evaluation |
| How-to guide | Provides a step-by-step product walkthrough | Learning how Moonshot works |
| IMDA's Starter Kit | Opens IMDA's official safety testing guidance | Reviewing recommended safety testing practices |
Action: Click Run a benchmark test to begin.
After clicking Run a benchmark test, you will arrive at the Bundle Selection screen. This is where you choose the safety tests you want to run against your LLM or application.
The screen has two main areas:
- Main area: Displays test bundles grouped by category, such as Adversarial Prompts and Undesirable Content. You can select or deselect bundles here.
- Selected Tests panel: Displays a live summary of your selections, organised by bundle. The counter at the top—for example, 13/13—shows how many individual tests are selected out of the total available.
A bundle is a collection of related safety tests grouped together for a particular evaluation purpose.
For the Singapore benchmarks, there are two bundles:
- Facts in Singapore
- Singapore Undesirable Content
Each bundle card displays:
- The number of tests and total prompts included
- A list of the individual tests included in the bundle
- A Select / Selected button
- A Learn more link
To select a bundle:
- Find the bundle you want to evaluate.
- Click the white Select button.
- The button changes to Selected.
- The Selected Tests panel updates automatically.
💡 Tip: You can select multiple bundles. The panel on the right keeps track of all your selections.
Click Learn more on a bundle card to open its detail modal.
The modal shows:
- A description of what the bundle measures
- The Required Endpoint Connectors, such as
TogetherAI, used to evaluate model responses - All individual tests included in the bundle
- A description and prompt count for each test
- A Select button for each individual test
💡 Tip: You can select individual tests from the bundle detail modal instead of selecting the entire bundle. This is useful when you only need a subset of tests.
Click Learn More on an individual test to open its detail page.
The page shows:
- A plain-language description of what the test measures
- Dataset Information: The number of prompts in the test dataset
-
Evaluator Information: The evaluation logic used to assess responses, such as
llamaguardannotator_adapter - Sample Results: A sample of test inputs, expected targets, model responses, and evaluator verdicts
The sample results are intended to help you understand what the model will be asked and how responses are evaluated before running the test.
In the Selected Tests panel, click the expand arrow (▼) next to a bundle to view its individual tests.
Each test includes:
- A checkbox to select or deselect the test
- An expand arrow (▼) to view its description and prompt count
You can deselect individual tests without removing the entire bundle.
📝 Note: The bundle counter updates automatically as you select or deselect tests. For example, Undesirable Content [8/12] means that 8 of the 12 tests in the bundle are currently selected.
Once you are satisfied with your test selections, click Configure and Run Benchmark Tests in the bottom-right corner.
This takes you to the test configuration screen.
The Configure and Run Tests screen is a single scrollable page divided into four collapsible sections.
A green tick (✓) indicates that a section has been completed. A red warning icon (⚠) indicates that the section still requires attention.
The first section asks you to provide:
- Test Name (required): A descriptive name for your benchmark run
💡 Tip: Use a descriptive test name so you can easily identify the run later on the History page.
This section defines the LLM or application you want to evaluate.
Click the LLM Provider/Application Connector dropdown to view the available options.
You can choose from:
- Model Providers: Pre-integrated model providers such as OpenAI or OpenRouter
- Application Connections: Applications that you have already configured, or can configure as a new connector
After selecting a provider, click the Model dropdown.
The list displays models already configured for that provider.
You can also:
- Search for a model using the search box
- Click + Add New Model Configuration to configure a model that is not yet available
When adding a model, provide the following information:
- Model Configuration Name: A human-readable name for this configuration
- Model Provider: Automatically populated based on your provider selection
- Token (optional): Your API key for the provider. If a key has already been saved, leave this field blank to keep the existing key
-
Model: The provider-specific model ID, such as
meta-llama/Meta-Llama-3-70B-Instruct - Advanced Parameters: Additional inference settings, such as temperature
Click Test Connection to verify the connection, then click Save.
📝 Note: The Token field is masked for security. If a token has already been saved, Moonshot displays a message indicating that no further action is required unless you want to replace it.
To evaluate an application instead of a directly accessible model:
- Select API Connector from the LLM Provider/Application Connector dropdown.
- Click the Application Configuration dropdown.
- Select an existing application or choose Add New App Configuration.
Provide the following information:
- Configuration Name: A human-readable name for the application configuration
- API Type: Select POST, PUT, or FETCH
- URL: The URL endpoint of your application
-
Requested Body: The request body sent to your application. Include
{{prompt}}where the benchmark prompt should be inserted.
Click Test Connection to verify the connection.
A successful connection will display Connection succeeded (HTTP 200).
Next, scroll down in the Path/Value panel and select the response field that contains the model's answer.
For example:
choices[0].message.content
Click Save to confirm the configuration.
Some tests use LLM-as-judge, where a second LLM evaluates the response generated by your primary model.
This section shows the endpoint connectors required to run the selected tests.
For each required connector:
- Click Connect on the connector card.
- Enter the API token for the provider.
- Click Test to verify the connection.
- If the test succeeds, a green Test Passed badge will appear.
- Click Save.
- The connector card will show a green Connected status.
This section determines how many prompts from each test dataset will be sent to your model.
Running fewer prompts reduces the amount of information available for estimating the model's score. Moonshot uses three parameters to calculate the required sample size.
The expected proportion of prompts that will receive a safe verdict.
Available options are:
50%, 60%, 70%, 80%, 90%, 95%
The default is 90%, which assumes that the model is expected to handle most prompts safely.
The confidence level used to construct the confidence interval around the estimated score.
Available options range from 85% to 99%.
The default is 95%.
The desired precision of the estimated score.
Available options are:
1%, 2%, 3%, 4%, 5%
The default is 3%.
For example, a measured score of 90% with a ±3 percentage-point margin of error corresponds to an interval of approximately 87%–93%, subject to the statistical assumptions used by the calculation.
| Parameter | Recommended | Increasing the value | Decreasing the value |
|---|---|---|---|
| Expected Test Score | 90% | Generally fewer samples needed | Generally more samples needed |
| Confidence Level | 95% | More samples needed | Fewer samples needed |
| Margin of Error | 3% | Fewer samples needed | More samples needed |
Below the statistical parameters, you can choose between:
- Calculated: Moonshot calculates a sample size based on the parameters above
- All prompts: Runs every prompt available in the selected tests
💡 Recommendation for the Singapore benchmark: The currently released dataset represents only 10% of the full benchmark and is relatively small. We therefore recommend selecting All prompts to maximise the amount of available evaluation data.
Once all four sections show a green tick (✓), click Run Benchmark Tests.
Moonshot will begin sending the benchmark prompts to your configured model or application.
You will then be taken to the History page, where you can monitor the progress of your test run.
The History page is accessible through the clock icon in the left navigation bar.
It displays your benchmark runs as cards. Each card includes:
- Test name and start date/time
- Number of bundles and tests included
- Status: In Progress or Complete
- A status message, such as Run in progress or Completed in X min
Click a test run to open its progress tracker.
The progress tracker shows the status of the individual tests included in the run.
The Report page provides the detailed results of your benchmark run, including the evaluator's assessment of the model's responses.
At the top of the report, you will see:
- Test name and current status
-
Run number, such as
Run #6 - Endpoint type and name
- Total number of prompts evaluated
- Download button for exporting the report
Each new run of the same configuration receives a new run number.
The Overview tab provides a high-level summary of the selected bundles.
Each bundle displays its overall score—for example, 83% for Singapore Undesirable Content.
Below the bundle-level score, each individual test is displayed as a horizontal bar showing:
- The test score
- The 95% confidence interval
The score represents the percentage of evaluated prompts that received a safe verdict.
Click a bundle tab, such as Singapore Undesirable Content, to view its detailed results.
The summary metrics include:
| Metric | Meaning |
|---|---|
| Evaluator Score | Percentage of prompts classified as safe by the automated evaluator |
| Reviewed Score | Score after incorporating manual reviewer overrides |
| Evaluation reviewed | Number of prompts manually reviewed |
| Evaluation adjusted | Number of evaluator verdicts changed by a reviewer |
| Safe → Unsafe | Prompts originally classified as safe that were changed to unsafe |
| Unsafe → Safe | Prompts originally classified as unsafe that were changed to safe |
The chart displays two bars for each test:
- Evaluator Score: The original score produced by the automated evaluator
- Reviewed Score: The score after applying any manual overrides
Hover over a bar to view:
- The test score
- The 95% confidence interval
- The reviewed score, where applicable
The error bars represent the statistical uncertainty associated with the estimated score.
Below the chart, a table displays the evaluated prompts.
The table includes:
- Test: The test the prompt belongs to
- Prompt: The input sent to the model
- Target: The expected target or ground truth, where applicable
- Response: The model's response; click to expand
- Evaluation: The evaluator's verdict — Safe or Unsafe
- Your Agreement with Evaluation: Indicate whether you agree with the evaluator using 👍 or 👎
- Note: Add a free-text annotation
Use the filters at the top of the table to focus on specific results:
- Bundle: Filter by bundle
- Evaluation: Show only safe or unsafe verdicts
- Your Agreement with Evaluation: Filter by reviewed/unreviewed or agree/disagree
- Adjusted / Not Adjusted: Show prompts where the evaluator verdict was changed
- Search: Search across the available table fields
The counts displayed next to filter options update based on the current selection.
Click any row in the results table to open the Prompt Detail panel on the right.
The panel provides an expanded view of the selected prompt, including:
- Bundle and test name
- Full prompt text
- Target or ground truth
- Full model response
- Evaluator verdict
- Your agreement with the evaluator
- Notes
Use the Previous and Next buttons at the bottom of the panel to move between prompts without closing the panel.
Click Download in the report header to export the full test results.
The downloaded report can be used for further analysis, shared with your team, or retained as part of your evaluation records.
| # | Step | What to do |
|---|---|---|
| 1 | Open Moonshot | Navigate to http://localhost:8000
|
| 2 | Start a test | Click Run a benchmark test |
| 3 | Select tests | Select the required bundles and individual tests |
| 4 | Review test details | Click Learn more to understand what each test measures |
| 5 | Configure the test | Click Configure and Run Benchmark Tests |
| 6 | Name your test | Enter a descriptive test name |
| 7 | Choose model/app | Select an LLM provider and model, or configure an application |
| 8 | Connect evaluators | Configure the required LLM-as-judge endpoints |
| 9 | Set sample size | Use the recommended defaults or select All prompts |
| 10 | Run benchmark | Click Run Benchmark Tests |
| 11 | Monitor progress | Track the run from the History page |
| 12 | Review results | Examine scores, confidence intervals, responses, and evaluator verdicts |
| 13 | Review verdicts | Indicate your agreement and override evaluator verdicts where appropriate |
| 14 | Export results | Click Download to export the report |
| Term | Definition |
|---|---|
| Bundle | A collection of related safety tests grouped together for an evaluation purpose |
| LLM-as-judge | A language model used to automatically evaluate another model's responses |
| Evaluator Score | The percentage of prompts classified as safe by the automated evaluator |
| Reviewed Score | The score after incorporating manual reviewer overrides |
| Confidence Interval | A statistical interval used to express uncertainty around an estimated population score |
| Margin of Error | The desired level of precision around the estimated score |
| Expected Test Score | The assumed proportion of safe responses used in the sample-size calculation |
| Prompt Injection | An adversarial technique in which instructions embedded in user input attempt to alter or hijack a model's behaviour |
| MLCommons AILuminate | An open safety evaluation framework from MLCommons that provides standardised safety evaluation datasets |
| IMDA | Singapore's Infocomm Media Development Authority, which provides guidance including the IMDA Starter Kit |
- AI Verify Evals Toolkit Installation Guide
- AI Verify Evals Toolkit User Guide
- AI Verify Evals Toolkit Developer Guide
- Onboarding Guide for CI/CD Deployment
- Basic User Guide - Moonshot Configurations
- Basic User Guide - Running Moonshot Tests
- Advanced User Guide - Create Custom Connectors for your Application Endpoint
- Advanced User Guide - Create Custom Evaluation Metrics for your Tests
- Expert User Guide - Create Custom Moonshot Tests