Repository navigation
Writing Tests
A ctxray test states what your agent should have in context and checks it on every harness you use. You write it once, and ctxray runs it against Claude Code, Codex, and the rest, one cell per harness. This page starts with a single rule check and grows it into a test that asks a real model a hundred prompts. The reference at the end lists every function, option, and matcher.
Each kind of test checks one layer of the Loadout Stack: test.resolve checks L3, test.assemble L4, test.wire L5, test.activate L6, and test.scenario L7. L1 Lint and L2 Policy need no tests: ctxray runs them on every ctxray test, as Policies explains.
ctxray picks up every file named *.ctxray.ts in the repository. .js, .mjs, .cjs, .mts, and .cts work too, and *.ctxray.jsonc (or .json) holds declarative tests. Files your .gitignore hides, node_modules included, stay out. Run ctxray test from a subfolder, and only the tests under it run. ctxray reads the config from the folder it runs in, so point a subfolder run at the root one with --config ../ctxray.config.ts.
A test file imports its API from @mairwunnx/ctxray:
import { corpus, discover, expect, suite, test } from "@mairwunnx/ctxray";ctxray serves this module to the file itself, so a test runs with no node_modules at all. The npm package only gives your editor the types:
npm install -D @mairwunnx/ctxrayWhen the installed package and the ctxray executable differ in version, the run says so: “node_modules holds @mairwunnx/ctxray 0.1.0, and the tests run on this ctxray, 0.2.0”. Your editor then checks the code against an API that ctxray may not run.
ctxray init writes a first test to ctxray/loadout.ctxray.ts.
L3 predicts what a harness loads from the project's files alone. It launches nothing and answers in milliseconds. The test below comes from ctxray's own fixtures: payment rules must load when the agent works on payment code.
import { expect, suite, test } from "@mairwunnx/ctxray";
suite("payments", { fixture: "../payments" }, () => {
test.resolve("payments rules attach to payment sources", { requires: ["pathRules"] }, async ({ resolve }) => {
const l = await resolve({ cwd: "app", files: ["app/src/main/kotlin/payments/Charge.kt"] });
expect(l.rules).toInclude([".claude/rules/kotlin.md", ".claude/rules/payments.md"]);
expect(l.rules).not.toInclude(".claude/rules/web.md");
expect(l.rules.get(".claude/rules/payments.md").reason).toBe("glob:**/payments/**/*.kt");
});
});suite groups tests under a name. Its fixture names the project the tests look at, relative to the test file. Without it, the tests look at the repository itself, and nested suites inherit their parent's fixture.
resolve() takes the folder the agent starts in (cwd) and the files it works with (files), both relative to the project root. It returns the loadout the harness would build: rules, skills.listed, skills.loaded, mcp.servers, commands, and tokens. Every list also finds an entry by path or name: l.rules.get(".claude/rules/payments.md") returns the rule with its activation, reason, and tokens. An id the list doesn't hold fails the test and names what the list does hold.
Codex has no rules that load by path, so requires: ["pathRules"] skips it instead of failing it. Run the layer:
ctxray test --layer L3 ctxray 0.1.0 · 2 harnesses · backend: mock
payments payments
L3 resolve payments rules attach to payment sources
claude-code ✓ codex ○ needs pathRules
When an expectation fails, ctxray shows what it expected and what it found:
✗ claude-code › demo › web rule stays out
expect(received).toInclude(expected)
Missing .claude/rules/web.md
Received CLAUDE.md, AGENTS.md, .claude/rules/style.md, app/CLAUDE.md, app/AGENTS.md, …
Harnesses read different files. Claude Code takes CLAUDE.md, Codex takes AGENTS.md. expect.on holds expectations for some harnesses only:
test.resolve("instructions of the app load in app", async ({ resolve }) => {
const l = await resolve({ cwd: "app" });
expect(l.rules).toInclude("app/AGENTS.md");
expect.on("codex", () => expect(l.rules).toInclude("AGENTS.md"));
expect.on("claude-code", () => expect(l.rules).toInclude(["CLAUDE.md", "app/CLAUDE.md"]));
});The body also gets harness, the id of the harness it runs on. ctxray init uses it to pick each harness's root file:
const rootRules: Record<string, string> = { "claude-code": "CLAUDE.md", codex: "AGENTS.md" };
suite("loadout", () => {
test.resolve("the root rules file loads for every file", async ({ harness, resolve }) => {
const loadout = await resolve({ files: ["README.md"] });
expect(loadout.rules).toInclude(rootRules[harness]);
});
});Harnesses lists which files each harness reads and which capabilities it has.
A snapshot holds a whole loadout, so any change to it shows up in review. toMatchLoadoutSnapshot() compares the loadout with the stored one:
const l = await resolve({ cwd: "app" });
expect(l).toMatchLoadoutSnapshot();The first run fails with “Snapshot … isn't written yet”, because a test run never writes your files by itself. Write the snapshots on purpose and commit them:
ctxray test -uThey land next to the test file: payments.ctxray.ts keeps its snapshots in __snapshots__/payments.ctxray.snap.json. A key holds the suite, the test, the harness, and its version: payments › instructions of the app load in app › claude-code@static. L3 reads files only, so its version is static. An L4 or L5 snapshot carries the harness version and the model, so a harness upgrade changes the key by itself.
An L3 snapshot keeps each rule with its activation, reason, and tokens, each skill, the MCP servers, the commands, and the tokens per component. When the loadout changes, ctxray names each value that moved:
✗ claude-code › payments › instructions of the app load in app
The loadout differs from snapshot “payments › instructions of the app load in app › claude-code@static”
Changed rules › app/CLAUDE.md › tokens 23 → 40
Changed tokens › rules 115 → 132
An L4 or L5 snapshot keeps names only: system prompt sections, rules, skills, MCP servers with their tools, and commands. Captured token counts move with the date and the paths in the request, the names don't.
L4 launches the real harness against ctxray's mock model and takes its first request. You see what L3 can't: the system prompt, the built-in tools, and the MCP tool schemas. The mock model costs nothing and answers the same way every time. The harness has to be on your PATH or pinned in ctxray.lock, as Harnesses describes.
test.assemble("context budget", async ({ assemble }) => {
const l = await assemble({ cwd: "app" });
expect(l.tokens.total).toBeLessThan(25_000);
expect(l.tokens.byMcpServer("jira")).toBeLessThan(3_000);
expect(l.mcp.servers).toEqual(["github", "jira"]);
expect(l).toMatchLoadoutSnapshot();
});assemble() returns the captured loadout. On top of what L3 shows, it has system (the sections of the system prompt), mcp.tools, duplicates, the harness version and model, and tokens.total. tokens.byMcpServer(name) gives what one server costs once its tools load. tokens.exact is true only when a provider counted the tokens. Otherwise they're a local estimate, and the report marks them ≈.
toEqual compares an entry list with a list of ids in order, so ["github", "jira"] reads as the server names.
You don't need a test to hold the starting context to a limit. Each ctxray test with L4 in its layers launches every harness once and checks the starting context against the maxTokens of your policies.
In L5 your test plays the model. It tells the harness to read a file, waits for the next request, and checks what the harness added to the context:
test.wire("nested instructions load on read", { requires: ["pathRules"] }, async ({ session }) => {
await session.model.calls("Read", { path: "app/src/main/kotlin/payments/Charge.kt" });
const next = await session.nextRequest();
expect(next.loadout.rules).toInclude("app/src/main/kotlin/payments/AGENTS.md");
expect(next).toAgreeWithResolver();
});session.model.calls(tool, input) answers the harness's latest request with a tool call. Read with a path and Skill work on every harness: ctxray turns them into the harness's own tool. session.model.says(text) answers with text. session.nextRequest() waits for the request that follows, and session.firstRequest() returns the first one.
A request holds loadout (the captured loadout at that point), added (the rules and skills this request brought in), and predicted (what L3 expects for the same folder and the files read so far). Two matchers work on it:
-
toLoadRule(rule)checks that the rule is in context at this request. With.not, it checks that this request didn't bring the rule in, so a rule an earlier read loaded doesn't fail a later read. -
toAgreeWithResolver()checks that the harness loaded the rules and project skills L3 predicts. When they differ, ctxray reports resolver drift: the harness changed its behaviour, and ctxray's adapter for it needs an update.
session.events() returns the harness's own event stream, such as Claude Code's stream-json. startSession({ cwd, prompt }) opens one more session in the same test. ctxray ends every session when the test body returns. A harness that exits with an error, or that tries to reach the internet past ctxray, fails the test.
Every test kind has .each. Over a list of objects, it registers one test per item, and {field} in the name takes the item's value:
test.resolve.each([
{ file: "app/src/main/kotlin/payments/Charge.kt", rule: ".claude/rules/payments.md" },
{ file: "app/src/main/kotlin/web/Page.kt", rule: ".claude/rules/kotlin.md" },
])("{rule} loads for {file}", { requires: ["pathRules"] }, async ({ file, rule, resolve }) => {
expect((await resolve({ files: [file] })).rules).toInclude(rule);
});A list of rule paths goes stale the day someone adds a rule. discover.rules() selects rules and skills by what they are instead:
| Selector | Picks |
|---|---|
owner |
Rules whose owner in CODEOWNERS matches, like "@acme/payments", or a list of owners. ctxray looks in .github/CODEOWNERS, .gitlab/CODEOWNERS, CODEOWNERS, and docs/CODEOWNERS
|
under |
Rules inside a folder or matching a glob, relative to the project root |
activation |
always, path, intent, or manual, or a list of them |
.where() narrows a selection further. The selection resolves inside each test, because which rules exist depends on the harness. A selection that matches nothing fails the test: a typo in a selector would otherwise pass as a test that checks nothing.
const payments = discover.rules({ owner: "@acme/payments" });
suite("payments rules", { fixture: "../payments" }, () => {
test.resolve.each(payments.where({ activation: "path" }))(
"{rule.path} attaches to matching files",
{ requires: ["pathRules"] },
async ({ rule, resolve }) => {
for (const file of rule.matchingFiles({ sample: 3 })) {
expect((await resolve({ files: [file] })).rules).toInclude(rule);
}
const other = rule.nonMatchingFile();
if (other) expect((await resolve({ files: [other] })).rules).not.toInclude(rule);
},
);
});.each over a rule set gives each test a rule. rule.matchingFiles({ sample: 3 }) returns three project files the rule applies to, spread evenly over the list, and rule.nonMatchingFile() returns one it doesn't. toInclude takes the rule, a rule set, a path, or a list of paths.
The same selection works in L4 and L5. A rule set's root is the folder its rules live in, a good place to start the agent:
test.assemble("always rules present in payments zone", async ({ assemble }) => {
const always = payments.where({ activation: "always" });
const l = await assemble({ cwd: always.root });
expect(l.rules).toInclude(always);
});A skill loads when the model decides it fits the prompt. Only a model that answers by itself can tell whether it does, so L6 runs the prompt several times and counts:
test.activate(
"refund prompt → refunds skill",
{ prompt: "Make a partial refund of 40 EUR for order 123.", samples: 5, requires: ["skills"] },
({ runs }) => {
expect(runs).skill("refunds").toActivate({ min: 0.5 });
expect(runs).skill("payouts").not.toActivate({ max: 0.5 });
},
);Each sample is a whole session: the harness gets the prompt and works until it stops, or until it reaches the turns limit and ctxray answers in the model's place. Then the body sees every run. expect(runs).skill(name), .rule(path), and .anyOf(set) count the runs that loaded the skill, the rule, or any of a set.
L6 needs a model. With the default mock backend, the test shows “needs --backend replay or real” and skips. Backends and cassettes explains both.
Ten runs out of ten don't prove a 90% rate. ctxray puts a 95% Wilson interval around the rate and judges the interval, not the bare rate:
-
toActivate({ min })passes when the interval's lower bound reachesmin. -
.not.toActivate({ max })passes when its upper bound stays at or undermax. - The test fails when the rate itself is on the wrong side of the threshold.
- Otherwise the samples can't tell yet, and the test is inconclusive. ctxray names how many samples in all would settle it if the rate held.
skill(refunds) loaded at a rate of 1.00 [0.72–1.00, 95% CI]: too few samples to tell
Samples in all to tell skill(refunds): 35
Without a threshold, toActivate() means { min: 0.8 } and .not.toActivate() means { max: 0.2 }. Without samples, the test makes enough runs to prove them: 16 for each group of prompts, which is the single prompt, the positive list, and the negative list. ctxray splits a group's runs between its prompts and rounds up, so two positive prompts run 8 times each. An explicit samples wins and counts per prompt.
Count the runs before you pick a stricter threshold. Even when every run loads the skill, the lower bound reaches 0.5 at 4 runs, 0.8 at 16, 0.9 at 35, and 0.95 at 73. The activate section of the config sets the thresholds and the run count for the whole project. Raise its min to 0.9, and the default count grows to 35 by itself. A threshold you pass to toActivate({ … }) still beats the config.
An inconclusive test doesn't fail the run. Sessions that broke (the harness crashed, a cassette ran out) say nothing about the model, so ctxray leaves them out of the count. When every one of them broke, the first failure becomes the test's.
One prompt tells you about one wording. A corpus gives the test many: prompts that should pick the skill (positive) and borderline ones that shouldn't (negative). Put them in a module whose default export holds both lists, or in a JSON file:
// tests/prompts/payments.ts
export default {
positive: ["Refund order 123 in part", "Pay the seller out for last week"],
negative: ["Rename the Android build flavour"],
};corpus() loads the file, relative to the test file, before any test runs. runs.positive and runs.negative hold the runs of each list. Each list gets its own 16 runs, so below the two positive prompts run 8 times each and the negative one 16 times:
import { corpus, discover, expect, suite, test } from "@mairwunnx/ctxray";
const payments = discover.rules({ owner: "@acme/payments" });
const prompts = corpus("./prompts/payments.ts");
suite("payments rules", { fixture: "../payments" }, () => {
test.activate(
"payments intents activate payments skills",
{ positive: prompts.positive, negative: prompts.negative },
({ runs }) => {
expect(runs.positive)
.anyOf(payments.where({ activation: "intent" }))
.toActivate();
expect(runs.negative)
.anyOf(payments.where({ activation: "intent" }))
.not.toActivate();
},
);
});Writing a hundred prompts by hand is slow. ctxray gen-prompts asks a model to write them from the description of each skill and each rule the model picks by description:
ctxray gen-prompts --from-descriptions --out tests/prompts/intents.jsonIt writes --count positive and as many negative prompts per rule or skill (5 by default) with claude-opus-5-5, through your upstream and key. Choose another model with --model, but not the one under test: a model that writes its own exam passes it too easily, and ctxray warns when it does. Read the file before a test uses it. The corpus stays as it is until you write it again, and an existing file needs --force.
{
"model": "claude-opus-5-5",
"generated": "2026-11-16",
"positive": ["…"],
"negative": ["…"],
"by": { ".claude/skills/refunds/SKILL.md": { "positive": ["…"], "negative": ["…"] } }
}corpus() reads the top-level positive and negative lists. by shows which prompts came from which file.
A scenario gives the agent a task and checks what it did. The body runs once per run, and gets that run:
test.scenario(
"a partial refund goes through the refunds skill",
{ prompt: "Make a partial refund of 40 EUR for order 123.", samples: 3, min: 0.6, requires: ["skills"] },
({ run }) => {
expect(run.skills).toInclude("refunds");
expect(run.changed).not.toInclude("app/build.gradle.kts");
// Tool names are the harness's own: Claude Code reads with Read, Codex with exec_command.
expect.on("claude-code", () => expect(run.tools).toInclude("Read"));
},
);A run holds the prompt and its sample number, the skills and rules that reached the model, the tools it called in order, the files it changed, its turns, whether ctxray cut the last turn, and the loadout of its last request. turns defaults to 30 here. Without min, every run must pass. With min, the share of passing runs goes through the same interval as L6.
A backend decides who answers the harness:
| Backend | Who answers | Runs |
|---|---|---|
mock |
Your test, or nobody. The default | L3, L4, L5. L6 and L7 skip |
real |
The model provider, or your gateway | L3–L7 |
replay |
Answers recorded earlier with real
|
L3–L7 |
L4 and L5 always run on the mock model, whatever backend you pick: it's free and deterministic.
A real run costs money, so record it once and replay it on every merge request:
ctxray test --layer L6-L7 --backend real --record
ctxray test --backend replay--record writes the model's answers next to the test file, one file per harness: __cassettes__/activate.ctxray.ts.claude-code.json. Commit them. ctxray redacts secrets before a cassette reaches disk. Each tape hangs on the test's name, the prompt's position, and the sample number. Rename a test or reorder its prompts, and record again.
A replay runs the harness for real and answers each turn from the tape. When the harness goes another way than it did while recording, the run fails with a cassette miss, which usually means the harness or your files changed. Record again with --backend real --record.
A real run reads the provider key from CTXRAY_UPSTREAM_KEY, ANTHROPIC_API_KEY, OPENAI_API_KEY, or GEMINI_API_KEY, and stops spending at the budget. Configuration covers upstream, budget, prices, and named backends, and Enterprise covers gateways.
Simple checks don't need TypeScript. A *.ctxray.jsonc file holds suites of tests as data, and ctxray turns each into the same test the DSL registers:
Each test names its layer. Lists take include, exclude, and equal. Numbers take lessThan, atMost, moreThan, and atLeast. L3 and L4 expect rules, skills.listed, skills.loaded, mcp.servers, commands, tokens, and snapshot. An L5 step does one read, call, or say and may expect what the next request holds. An L6 expectation names a skill, rule, or anyOf with min or max, and runs picks all, positive, or negative. L7 expects changed, tools, skills, and rules. The tests.schema.json release asset holds the full schema, so your editor completes it.
ctxray test # every layer the backend allows, every configured harness
ctxray test --layer L1-L3 # files only: no harness launches
ctxray test --harness claude-code # one harness
ctxray test --filter "payments rules" # tests whose name holds this text
ctxray test --watch # again after each change--filter matches the full name, suites included, ignoring case. ctxray's own checks of L1, L2, and the L4 starting context run whatever the filter says, so limit them with --layer. --watch (-w) runs again a moment after you save a file, Enter runs right away, and q stops. Changes in .git, node_modules, .ctxray, __snapshots__, __cassettes__, and the report folder start no run.
Every cell ends with a status:
| Status | Means | Fails the run |
|---|---|---|
| passed | Every expectation held | No |
| failed | An expectation failed, the body threw, or the time ran out | Yes |
| flaky | It failed, then passed on a retry | No |
| inconclusive | L6 or L7 had too few samples to tell | No |
| skipped |
filtered by --layer or --filter, unsupported by the harness (“needs pathRules”), unavailable here, waiting for a backend (“needs --backend replay or real”), or out of budget
|
No |
Commands lists every flag of ctxray test, and Reporters and Reports covers the HTML, JUnit, and Markdown output.
| Export | What it is |
|---|---|
suite(name, options?, body) |
Groups tests. options.fixture: the project folder, relative to the test file. The body must register its tests right away: an async body fails the file |
test.resolve, test.assemble, test.wire, test.activate, test.scenario
|
Register a test of L3, L4, L5, L6, or L7: (name, options?, body)
|
test.<kind>.each(items) |
(name, options?, body) once per object, or once per rule of a rule set |
expect(value) |
Matchers, below |
expect.on(harnesses, body) |
Runs body only on these harnesses: one id or a list. Returns what body returns, so await it when it's async |
discover.rules(selector?) |
A rule set, selected inside each test |
corpus(path) |
Prompts from a module or a JSON file, relative to the test file |
defineConfig(config) |
Types for ctxray.config.ts, see Configuration
|
Types: TestOptions, SuiteOptions, Rule, RuleSet, RuleSelector, ResolvedLoadout, ResolveRequest.
| Kind | Layer | The body gets | Default timeout |
|---|---|---|---|
test.resolve |
L3 | resolve({ cwd?, files? }) |
30 s |
test.assemble |
L4 | assemble({ cwd?, prompt? }) |
120 s |
test.wire |
L5 |
session, startSession({ cwd?, prompt? })
|
120 s |
test.activate |
L6 | runs |
300 s per sample |
test.scenario |
L7 | run |
300 s per sample |
Every body also gets harness and, under .each, the item's fields or rule.
| Option | Type | Default | Layers | Means |
|---|---|---|---|---|
requires |
Capability[] |
[] |
all | A harness without one of these skips the test as unsupported: skills, pathRules, intentRules, mcp, hooks, headless, proxyable
|
retries |
number |
0 |
all | Runs again after a failure. A later pass makes the test flaky |
timeout |
number (ms) |
see above | all | For L6 and L7 it limits each sample, not the test |
cwd |
string |
project root | L4–L7 | Where the harness starts, relative to the project root |
prompt |
string |
– | L6, L7 | The prompt |
positive |
string[] |
[] |
L6, L7 | Prompts that should load the subject, in runs.positive
|
negative |
string[] |
[] |
L6, L7 | Prompts that shouldn't, in runs.negative
|
samples |
number |
16 per group of prompts (L6), 1 (L7) | L6, L7 | Runs per prompt. Without it, L6 splits the config's activate.runs between the prompts of each group |
turns |
number |
2 (L6), 30 (L7) | L6, L7 | Model turns before ctxray answers in the model's place |
min |
number |
every run | L7 | The share of runs that must pass |
L6 and L7 need at least one of prompt, positive, and negative.
| Field | L3 resolve()
|
L4 assemble(), L5 request.loadout
|
|---|---|---|
harness |
✓ | ✓ |
cwd, files
|
✓ | |
version, model
|
✓ | |
system |
✓ | |
rules, commands
|
✓ | ✓ |
skills.listed, skills.loaded
|
✓ | ✓ |
mcp.servers |
✓ | ✓ |
mcp.tools (id server/tool) |
✓ | |
duplicates |
✓ | |
tokens.known |
✓ | |
tokens.total, tokens.byMcpServer(name), tokens.framing
|
✓ | |
tokens.byComponent, tokens.exact, tokens.method
|
✓ | ✓ |
Every list is an array with .get(id), which finds an entry by path or name. A rule has path, activation, reason, and tokens. A skill has name, path, builtin, and tokens. L3 doesn't know the system prompt, the tools, or the MCP schemas, so tokens.known counts only rules, skills, and commands. ctxray's own plugin stays out of every captured list.
| Member | Does |
|---|---|
firstRequest() |
The first request of the session |
nextRequest() |
The request that follows the latest answer |
model.calls(tool, input?) |
Answers with a tool call. Read with path and Skill work on every harness |
model.says(text) |
Answers with text |
events() |
The harness's own events, like Claude Code's stream-json and Codex's --json
|
end() |
Ends the session. ctxray calls it after the body anyway |
A request has index, loadout, added.rules, added.skills, predicted, raw, and snapshot.
runs is an array of runs with runs.positive and runs.negative. A run has prompt, sample, skills, rules, tools ({ name, input }, oldest first), changed, turns, cut, loadout, added (what joined after the first request), and error when the session itself failed.
| Member | Means |
|---|---|
RuleSelector.owner |
A CODEOWNERS owner or a list of them |
RuleSelector.under |
A folder or a glob, relative to the project root |
RuleSelector.activation |
always, path, intent, manual, or a list |
RuleSet.where(selector) |
A narrower set |
RuleSet.rules |
The selected rules, inside a test |
RuleSet.root |
The under folder, or the common folder of the rules |
Rule.path, Rule.dir
|
The file, and the folder it applies to |
Rule.activation, Rule.globs
|
How it loads, and its globs |
Rule.matchingFiles({ sample? }) |
Project files it applies to, spread evenly |
Rule.nonMatchingFile() |
One project file it doesn't apply to, or undefined
|
Instructions without globs count as always, with globs as path. Skills the harness lists count as intent, the others as manual.
| Matcher | Takes | Checks |
|---|---|---|
toBe(value) |
anything | Object.is |
toEqual(value) |
anything | Deep equality. An entry list equals a list of ids in the same order |
toBeDefined(), toBeUndefined()
|
anything | |
toBeLessThan(n), toBeLessThanOrEqual(n)
|
a number | |
toBeGreaterThan(n), toBeGreaterThanOrEqual(n)
|
a number | |
toHaveLength(n) |
anything with length
|
|
toMatch(text or RegExp) |
a string | A substring or a match |
toInclude(ids) |
a list | Holds every id: a path, a name, a rule, a rule set, or a list. .not: holds none of them |
toMatchLoadoutSnapshot() |
a loadout | Equals the stored snapshot. No .not
|
toLoadRule(rule) |
an L5 request | The rule is in context. .not: this request didn't bring it in |
toAgreeWithResolver() |
an L5 request | The harness loaded what L3 predicts. No .not
|
skill(name).toActivate({ min }) |
runs | The interval's lower bound reaches min, by default 0.8 or the config's activate.min
|
rule(path).toActivate({ min }) |
runs | The same for a rule |
anyOf(set).toActivate({ min }) |
runs | The same for any of a rule set or a list of paths and names |
….not.toActivate({ max }) |
runs | The upper bound stays at or under max, by default 0.2 or the config's activate.max
|
Matchers have a .not form unless the table says otherwise. The activation matchers put it before toActivate.
| Path | Written by | Commit it |
|---|---|---|
__snapshots__/<name>.ctxray.snap.json |
ctxray test -u |
Yes |
__cassettes__/<name>.ctxray.ts.<harness>.json |
ctxray test --backend real --record |
Yes |
a corpus, like prompts/intents.json
|
ctxray gen-prompts or you |
Yes |
Start
Use
Teams
Develop
По-русски
- Начало работы · Понятия · Loadout Stack · Режимы захвата
- Тесты · Конфигурация · Политики · Харнессы · Команды · Отчёты · Плагин Claude Code
- CI · Enterprise
- Архитектура · Локализация · Участие
{ "$schema": "https://github.com/MairwunNx/ctxray/releases/latest/download/tests.schema.json", "suites": [ { "name": "payments", "fixture": "../payments", "tests": [ { "name": "kotlin rule on Kotlin files", "layer": "L3", "requires": ["pathRules"], "files": ["app/src/main/kotlin/payments/Charge.kt"], "expect": { "rules": { "include": [".claude/rules/kotlin.md"], "exclude": [".claude/rules/web.md"] }, "tokens": { "total": { "lessThan": 25000 } } } }, { "name": "nested instructions load on read", "layer": "L5", "requires": ["pathRules"], "steps": [ { "read": "app/src/main/kotlin/payments/Charge.kt", "expect": { "rules": { "include": ["app/src/main/kotlin/payments/AGENTS.md"] }, "agreeWithResolver": true } } ] }, { "name": "refund prompt → refunds skill", "layer": "L6", "corpus": "./prompts/intents.json", "expect": [{ "skill": "refunds", "min": 0.8, "runs": "positive" }] } ] } ] }