MaxModel groundedness gate (GitHub Action)

Fail a pull request when your RAG/answer pipeline's deterministic groundedness drops below a threshold. It runs the MaxModel verified eval over a dataset of cases and gates on coverage_mean — scored by a verbatim string match, no LLM judge, so the gate is reproducible (same input → same result).

Usage

# .github/workflows/groundedness.yml
name: Groundedness
on: [pull_request]
jobs:
  gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: maxmodel-docs/groundedness-gate@v1   # or ./action when vendored
        with:
          api-key: ${{ secrets.MAXMODEL_KEY }}
          dataset: eval/dataset.json
          model: gpt-5
          min-coverage: '0.8'

Inputs

input	required	default	description
api-key	yes	—	MaxModel key (a maxmodel.com gateway token). Use a repo secret.
dataset	yes	—	Path to a JSON file: { "cases": [ { "messages": [...], "sources": [...] }, ... ] } (or a bare array of cases).
model	no	gpt-5	Model to run the eval with.
min-coverage	no	0.8	Fail if aggregate coverage_mean is below this.
mode	no	strict	strict | lenient.
base-url	no	https://api.maxmodel.com	Override for self-host.

The job writes a summary table (coverage_mean, unsupported_rate, scored/errored counts) to the GitHub step summary, and fails with ::error:: when below threshold.

Dataset format: see example/dataset.json. The same eval is available programmatically via mx.verified.eval(...) — see https://docs.maxmodel.com/eval/.

Example workflow

A ready-to-copy workflow is in examples/groundedness.yml — drop it into your repo's .github/workflows/ and add a MAXMODEL_KEY repo secret.

License

MIT — see LICENSE.