Repository navigation
CI Integration
ctxray belongs in CI: every merge request then shows whether the agents still load the right rules, whether the context grew, and whether a policy broke. This page covers GitHub Actions, GitLab CI, and any other CI that runs a container or a shell script.
Before you start, commit two files: your config and ctxray.lock. The lockfile pins the harness builds CI runs, and without it every test from L4 up fails in CI (Harnesses explains why).
The action MairwunNx/ctxray-action installs ctxray, caches the harnesses ctxray.lock pins, and runs ctxray test:
- uses: actions/checkout@v7
- uses: MairwunNx/ctxray-action@v0@v0 follows the newest 0.x release. Pin an exact tag, like @v0.1.0, when you want CI to change only on your say.
| Input | Default | What it does |
|---|---|---|
version |
the action's own version | The ctxray version to install, like 0.1.0
|
args |
test |
What to run, as arguments to ctxray: test --layer L1-L3, inspect, diff --base origin/master
|
working-directory |
. |
The project's folder, where ctxray.lock lies |
cache |
true |
Cache the harnesses ctxray.lock pins between runs |
download-base |
GitHub Releases | Where the releases live, for a mirror. Same as CTXRAY_DOWNLOAD_BASE in install.sh
|
The action splits args on spaces, so a single argument can't hold a space. Pass such values through the config or an environment variable instead.
The action runs on Linux, macOS, and Windows runners, on x64 and arm64. It installs through install.sh (or install.ps1 on Windows) into the runner's temporary folder and adds it to PATH, so later steps can call ctxray too.
name: ctxray
on:
push:
pull_request:
jobs:
ctxray:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: MairwunNx/ctxray-action@v0
- uses: actions/upload-artifact@v7
if: always()
with:
name: ctxray-report
path: .ctxray/reportIn CI, ctxray writes the console output plus JSON, JUnit, Markdown, and HTML reports into .ctxray/report/. On GitHub it also appends the Markdown summary to the job's summary page, so you see the result without opening the log. The artifact step keeps the HTML report for anyone who wants the details.
ctxray diff compares what each harness loads on the pull request with what it loaded on the base branch. It needs the base branch's history, so check out with fetch-depth: 0:
name: ctxray diff
on: pull_request
jobs:
diff:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- uses: MairwunNx/ctxray-action@v0
with:
args: diff --base origin/${{ github.base_ref }}ctxray diff only reports: it exits with 0 whatever changed. To put the same comparison into the job summary and the HTML report next to the test results, run the tests with --base instead:
- uses: MairwunNx/ctxray-action@v0
with:
args: test --base origin/${{ github.base_ref }}L6 Activate sends prompts to a real model many times, which costs money. Run it on a schedule with a key from your repository secrets, and cap the spend with --budget in US dollars:
name: ctxray activation
on:
schedule:
- cron: "0 4 * * 1"
workflow_dispatch:
jobs:
activation:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: MairwunNx/ctxray-action@v0
with:
args: test --layer L6 --backend real --budget 5
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}Once the budget is spent, the tests that haven't started yet are skipped with the reason “budget spent”, and the run doesn't fail on that alone. The harness itself never sees the key: ctxray's proxy adds it on the way to the model. Enterprise lists which variable serves which provider, and how to go through a gateway instead.
Record the model's answers once with --record, commit the cassettes, and every pull request replays them for free. Writing Tests explains recording and where the cassettes live.
ctxray test --layer L6-L7 --backend real --recordname: ctxray
on: pull_request
jobs:
ctxray:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: MairwunNx/ctxray-action@v0
with:
args: test --backend replayReplay needs no key and no network: L1–L5 run as always, and L6–L7 play back the recorded answers.
Each release carries a template, ctxray.gitlab-ci.yml. Include it and extend its hidden job .ctxray:
include:
- remote: https://github.com/MairwunNx/ctxray/releases/latest/download/ctxray.gitlab-ci.yml
ctxray:
extends: .ctxraylatest follows every release. To pin one, name its tag: https://github.com/MairwunNx/ctxray/releases/download/v0.1.0/ctxray.gitlab-ci.yml. In a closed network, keep a copy of the template in a project of your own and include it from there:
include:
- project: platform/ci-templates
ref: main
file: ctxray.gitlab-ci.yml
ctxray:
extends: .ctxrayThe .ctxray job runs ctxray test $CTXRAY_ARGS in the image ghcr.io/mairwunnx/ctxray:<version>-runtime, which holds ctxray, git, and Node. It keeps .ctxray/report/ as an artifact, even when the job fails, and hands junit.xml to GitLab as a JUnit report. GitLab then shows each failed test in the merge request's test summary.
| Variable | Default | What it does |
|---|---|---|
CTXRAY_IMAGE |
ghcr.io/mairwunnx/ctxray:<version>-runtime |
The image the job runs in. Point it at your registry's mirror, or at an image with the harnesses inside |
CTXRAY_ARGS |
empty | Extra arguments to ctxray test, like --layer L1-L3
|
CTXRAY_CACHE_DIR |
$CI_PROJECT_DIR/.ctxray-cache |
The harness cache. GitLab caches only inside the project's folder |
The job caches .ctxray-cache/ with a key built from ctxray.lock, so a new lockfile starts a fresh cache. ctxray never counts .ctxray-cache/ and .ctxray/ as project files, so the cached harness builds stay out of L1 lint and out of the project copy L4 launches on. Add .ctxray-cache/ to your .gitignore anyway if you don't want git status to list it.
To compare with the merge request's target branch, fetch the whole history and pass the merge base:
ctxray:
extends: .ctxray
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
variables:
GIT_DEPTH: "0"
CTXRAY_ARGS: --base $CI_MERGE_REQUEST_DIFF_BASE_SHAStore the key as a masked CI/CD variable, and let the job run only in scheduled pipelines:
ctxray-activation:
extends: .ctxray
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
variables:
CTXRAY_ARGS: --layer L6 --backend real --budget 5Two ways work everywhere: the Docker images, or install.sh.
The image. Run the job in ghcr.io/mairwunnx/ctxray:<version>-runtime for L4 and up, or in ghcr.io/mairwunnx/ctxray:<version> for L1–L3 alone. The image's entrypoint is ctxray, so reset it where your CI runs a script inside the container. Getting Started lists the tags.
The script. Pin the version, so every run installs the same build:
curl -fsSL https://github.com/MairwunNx/ctxray/releases/download/v0.1.0/install.sh | CTXRAY_VERSION=0.1.0 sh
export PATH="$HOME/.local/bin:$PATH"
ctxray testThen do what the action and the template do for you:
- Cache the folder
ctxray cache dirprints, with a key built from the hash ofctxray.lock. - Keep
.ctxray/report/as an artifact, and hand.ctxray/report/junit.xmlto your CI's test view. - Make sure your CI sets
CI, or pass--ci(below).
ctxray takes a run for CI when CI, GITLAB_CI, or GITHUB_ACTIONS is set to anything other than empty, 0, or false. CI=false switches CI mode off by hand.
| Locally | In CI | |
|---|---|---|
| Output | live view in a terminal | plain lines a log can keep |
| Reporters | console |
console, json, junit, markdown, html
|
ctxray.lock |
a mismatch with the config is only mentioned |
--frozen: a mismatch fails the run |
| GitHub job summary | – | the Markdown summary is appended when GITHUB_STEP_SUMMARY is set |
The CI reporters apply only while your config sets no reporters, and --reporter overrides both. Reporters and Reports describes each file.
The --ci flag does all of this on any machine, --frozen included. Pass it when your CI sets none of the variables.
The harnesses are the slow part of a cold run: Claude Code, Codex, and the others weigh hundreds of megabytes together. ctxray installs the builds ctxray.lock pins into its cache on first use and takes them from there afterwards. ctxray cache dir prints the folder, which is exactly what a CI cache step needs:
- id: harness-cache
run: echo "dir=$(ctxray cache dir)" >> "$GITHUB_OUTPUT"
- uses: actions/cache@v6
with:
path: ${{ steps.harness-cache.outputs.dir }}
key: ctxray-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('ctxray.lock') }}
- run: ctxray harness installThe action does exactly this when cache is true. ctxray harness install as its own step is optional: ctxray test installs on first use too, but a separate step shows the download apart from the tests. Harnesses covers the cache folders and CTXRAY_CACHE_DIRS.
A runner that may not reach npm, or a pipeline that shouldn't wait for downloads, needs the harnesses inside its image. Each release carries harnesses.Dockerfile for that. Put it into your project and build it there:
docker build -f harnesses.Dockerfile -t registry.example.com/ci/ctxray .The file starts from ghcr.io/mairwunnx/ctxray:<version>-runtime, copies ctxray.config.ts and ctxray.lock, and runs ctxray harness install --frozen into /opt/ctxray/harnesses, which it names in CTXRAY_CACHE_DIRS. If your config is a .jsonc file or extends other files, change the COPY line to match. The harnesses come from your registry under your licences: ctxray's own images never carry them.
Rebuild the image whenever ctxray.lock changes. Then point CI at it: CTXRAY_IMAGE on GitLab, container: on GitHub, the job image elsewhere.
Scripts and pipelines can branch on these numbers, and they stay the same across releases:
| Code | Meaning |
|---|---|
0 |
Everything passed |
1 |
A test or a check failed |
2 |
An unknown command, flag, or argument, or an unknown --base revision |
3 |
The config is missing or invalid |
4 |
A harness, the network, or the file system let the run down: ctxray doctor found an error, ctxray harness install couldn't install, or a command that needs the network runs offline |
5 |
A bug in ctxray. The message links to the issue tracker |
130 |
Ctrl-C or SIGINT |
From L4 up, ctxray launches the real harness on a copy of your project. In headless mode a harness starts the project's MCP servers and hooks without asking, so the project's code runs. On macOS ctxray confines the run's writes with sandbox-exec. Linux and Windows have no such sandbox: there the CI container, or a runner that's thrown away after the job, is the sandbox.
Hosted runners on GitHub and GitLab are throwaway machines, so they're fine. On a self-hosted runner, run the job in a container. ctxray doctor says which case your machine is: “No sandbox on this system: run L4 and up in a container, as CI does”.
Start
Use
Teams
Develop
По-русски
- Начало работы · Понятия · Loadout Stack · Режимы захвата
- Тесты · Конфигурация · Политики · Харнессы · Команды · Отчёты · Плагин Claude Code
- CI · Enterprise
- Архитектура · Локализация · Участие