Regex Assertions

Regex Assertions

Use a regex assertion when generated output must match a pattern rather than a fixed substring or complete string.

Availability

Interface	Support
Dashboard	Author in the visual or JSON editor and inspect normalized results
Mix CLI	Evaluate stored or file-based suites with mix aludel.eval
Elixir API	Evaluate inline or as part of a suite
ExUnit	Assert inline output or gate a persisted suite
CSV and JSON imports	Validate patterns before importing test cases

All interfaces use the same matcher and result contract.

Assertion Format

{"type": "regex", "value": "(?i)confidence: \\d+%"}

Inline PCRE options such as (?i) are supported. A successful match returns a score of 100.0; a valid non-match returns 0.0 with Output does not match regular expression.

Dashboard

Open a suite test case, add an assertion, choose regex, and enter the pattern. The visual and raw JSON editors save the same assertion map. Invalid or oversized patterns are rejected before the test case is stored.

Elixir API

Evaluate already-generated output without creating a suite:

result =
  Aludel.Evals.AssertionEvaluator.evaluate(
    "Confidence: 92%",
    %{"type" => "regex", "value" => "(?i)^confidence: \\d+%$"}
  )

result["passed"]
#=> true

Persisted suites use the same assertion map in each test case.

ExUnit

defmodule ResponseTest do
  use ExUnit.Case
  use Aludel.ExUnit

  test "includes a confidence percentage" do
    assert_evaluation(
      "Confidence: 92%",
      %{"type" => "regex", "value" => "(?i)^confidence: \\d+%$"}
    )
  end
end

Limit failures follow the normal ExUnit.AssertionError path and keep generated output out of evaluator error details.

File-Based Suites and Mix CLI

File-based manifests select a persisted suite whose test cases contain the regex assertion. They do not duplicate test-case definitions:

schema_version: 1
suite_id: 9a756a58-eaec-43ca-99e6-f5c016d85d0c
prompt_version_id: e74cf2e1-94b6-4bcb-9ed9-b259661be906
provider_id: e1c60ec0-6d55-419b-b958-7d088055254f

Run it with mix aludel.eval --file evals/support-answer.yaml. Regex assertions need no separate CLI command or configuration.

Resource Limits

Regex assertions are treated as untrusted input at both save time and execution time:

• patterns are limited to 4 KiB
• evaluated output is limited to 1 MiB
• backtracking work and depth are bounded
• each match has a 250 ms wall-clock ceiling
• assertion-provided PCRE limit directives cannot raise Aludel's limits

Aludel rejects rather than truncates oversized values so a shortened subject cannot produce a misleading pass. A size, complexity, depth, or timeout limit returns a normalized failed assertion with evaluator error type regex_resource_limit. Suite execution can continue with later assertions and test cases.

Choosing Another Assertion

• Use contains for a literal substring without regex syntax.
• Use exact_match when the entire output must equal one value.
• Use json_field or json_deep_compare for structured JSON.
• Use rubric_judge when correctness depends on meaning rather than format.

See Evaluation Suites, ExUnit Evaluations, and File-Based Suites for complete workflows.