diff --git a/public/images/docs/evaluation/explore-playground/test-an-eval/custom-test.png b/public/images/docs/evaluation/explore-playground/test-an-eval/custom-test.png new file mode 100644 index 00000000..60a92fb5 Binary files /dev/null and b/public/images/docs/evaluation/explore-playground/test-an-eval/custom-test.png differ diff --git a/public/images/docs/evaluation/explore-playground/test-an-eval/tracing-result.png b/public/images/docs/evaluation/explore-playground/test-an-eval/tracing-result.png new file mode 100644 index 00000000..a627ee79 Binary files /dev/null and b/public/images/docs/evaluation/explore-playground/test-an-eval/tracing-result.png differ diff --git a/src/pages/docs/evaluation/guides/explore-playground/test-an-eval.mdx b/src/pages/docs/evaluation/guides/explore-playground/test-an-eval.mdx new file mode 100644 index 00000000..7243b428 --- /dev/null +++ b/src/pages/docs/evaluation/guides/explore-playground/test-an-eval.mdx @@ -0,0 +1,40 @@ +--- +title: "Test an eval" +description: "Try an eval on sample or real data and read its verdict" +--- + +Every eval has a **Test Evaluation** panel on its page, so you can run it on a bit of data and read the result and the reason before you attach it to anything. Open an eval from the playground, and the panel sits on the right of its **Eval Details** tab. + +## Four ways to feed it data + +The panel gives the eval its inputs one of four ways, each its own tab: + +- **[Dataset](/docs/dataset):** pull a row from one of your datasets +- **[Tracing](/docs/observe):** pull a span or trace from a project's traces, so you test on real production data +- **[Simulation](/docs/simulation):** pull a call from a simulation +- **Custom:** paste your own test data as JSON, one value per variable the eval needs + +The eval's Test Evaluation panel in Custom mode, with a JSON test-data editor and the input, output, and context field mapping +*The Test Evaluation panel in Custom mode* + +## Map the eval's inputs + +An eval declares the variables it needs, and this hallucination eval needs `input`, `output`, and `context`. In **Custom** those variables are the JSON keys, so they are already mapped and you fill in the values directly. For a real data source like a **trace**, you map each variable to a field in your data, so `context` might map to a span attribute like `input.value`, because the attribute names depend on how your app is instrumented. + +## Read the result + +Click **Test Evaluation** and the panel returns the eval's **Result**, a pass or fail here, and an **Explanation** of why. That is your check that the eval behaves the way you meant before you wire it in anywhere. + +Testing the eval on a trace, showing the mapped span attributes, a Pass result, and the explanation of the verdict +*Testing on a trace: the eval returns a pass or fail and explains its reasoning* + +## Dive deeper + + + + Attach an eval to your data and run it for real + + + Write your own eval to test and use + +