An invalid pattern in a JSON schema throws and kills the whole run

[royalpinto007]

Desired outcome

An invalid pattern in a user's JSON schema produces a scorer failure, not a crash.

Why it matters

validate() in src/scorers/json-schema.ts compiles a user-supplied regex with no guard:

if (schema.pattern && !new RegExp(schema.pattern).test(value))
  errors.push(`${path}: does not match pattern ${schema.pattern}`);

schema comes straight from the eval file, so pattern is arbitrary user input. A typo such as "[a-z" makes new RegExp throw a SyntaxError, which propagates out of validate, out of the scorer, and up through the runner, taking the whole eval run with it. The user sees a raw stack trace mentioning RegExp and nothing that points at which case or which schema key is at fault.

The sibling scorer already handles this properly. src/scorers/regex.ts wraps compilation in try/catch and returns:

reason: `invalid regex: ${(err as Error).message}`,

so the same class of mistake is a clean, attributable failure in one scorer and a crash in the other.

The regex is also recompiled on every call, once per case per run, which is wasteful but secondary.

Steps

1. Wrap the new RegExp(schema.pattern) in try/catch inside validate().
2. On a compile error, push a descriptive entry into errors, for example `${path}: invalid pattern ${schema.pattern}: ${message}`, so the failure carries the JSON path and is reported like any other validation error.
3. Add a test in tests/scorers.test.ts with a schema containing an unclosed character class, asserting the scorer returns a failing result rather than throwing.

Small and self-contained, one function.

Claiming this

Comment below to claim it. A reply usually comes within a day.