Skip to content

Querying Reports

aryehcitron@gmail.com edited this page Aug 26, 2026 · 10 revisions

Querying Reports

kronikol query answers questions about a TestRunReport.json without anyone — human or agent — having to open it.

A real report measured in August 2026 was 10.7 MB, with a single embedded PlantUML diagram of 663 KB. For an AI agent those are roughly 2.7 million and 166,000 tokens: one diagram larger than most context windows. Reading the file to debug a test run is not slow, it is impossible. kronikol query exists so the file never has to be read.

dotnet tool install -g Kronikol.Tool

kronikol query summary .logs/kronikol/TestRunReport.json
kronikol query failures .logs/kronikol
kronikol query http .logs/kronikol s3/i47 --keys

<report> may be the file or a directory holding one. Given a directory with several reports, the tool lists them and stops rather than guessing which you meant.

Using it from an AI agent? See Using it from an AI agent at the bottom — there is a skill that teaches the whole workflow, and updating AI-Integration-Prompt matters too.


The idea

A report is four layers, and only one of them is large:

Layer Share of the file
Narrative — features, scenarios, steps, assertions 0.4%
Topology — who called whom, in what order, with what status ~1%
Artifacts — attachments, diagnostics ~0.2%
Payloads — bodies and headers ~90%

Every command prints the first three freely and, where a payload would go, prints a pointer instead:

i12-i39  redis GET myDotnetService:v1:location*  ×28  b:4bdea521 2.7 KB

That says something exists, how big it is, that it occurred 28 times, and exactly what to ask for to see it. Nothing is silently omitted — silent omission is what sends a reader back to cat.

Payloads are not hidden. A body is very often the answer, because it is where a wrong number actually comes from. The discipline is to reach one deliberately, by address, rather than sweeping all of them into view on the way past.

Aggregate before you fetch

When the question is about a field across many calls — "what did $.status ever hold?", "did any item have a negative price?" — the values command answers it in one shot, reading every body internally and printing only distinct values with counts and addresses. Reach for http/body only when one specific payload matters; reach for values when the answer is spread across many.


Addressing

Thing Address Notes
scenario s3 ordinal in file order
interaction s3/i47 ordinal within the scenario, in capture order
step s3/2, s3/b0 the stepPath the report carries; b prefixes a background step
assertion / sub-step s3/2.1 dotted path beneath its step
body b:4bdea521 first 8 hex of SHA-1 of the content
diagram s3/d0
note within a diagram s3/d0/n12

Ordinals are deterministic for a given file but mean nothing in another run. Across runs use stableId (printed by steps) and b: hashes — both survive a re-run, and since Kronikol 3.0.47 stableId distinguishes one row of a scenario outline from another. See Generated-Reports.

Two identical bodies share one b: address. Having read it once, you have read every occurrence.


Budget and truncation

Every command writes under a byte budget, default 6000, and every truncation announces itself with the exact flags that resume:

calls: 1-24 of 127 · next: --service redis --offset 24

If you did not see a footer like that, you saw everything.

Flag Effect
--max-bytes N the budget; 0 removes it
--offset N resume a truncated listing
--limit N cap rows
--count print how many matched and nothing else
--out FILE write a payload to a file and print one line

Prefer filtering harder to paging. --service, --status 5xx, --step, --grep and --group all cut more than --offset does.


Commands

Overview

summary <report>

The run header, per-feature results, the failures, the slowest scenarios, and a diagnostics roll-up. Always the first command.

TestRunReport.json  10.4 MB  Kronikol 3.0.47
2026-08-23T09:14:02Z → 2026-08-23T09:19:41Z
19 scenarios · 1 failed · 1202 interactions · 90 distinct bodies

Orders     11 passed, 1 FAILED
Catalogue   7 passed

Failed:
  s12  Checkout fails on a wrong total  — Assert.Equal() Failure
  → failures

scenarios <report> [flags]

Flag Effect
--result Failed filter on execution result
--feature X substring match on feature name
--label L matches scenario labels, categories and feature labels
--grep T substring match on scenario name
--slower-than 5 seconds

services <report> [s3] [--sort duration\|bytes\|errors]

Per service: calls, errors, bytes, median and max duration, status mix. Scoped to one scenario when given an address.

The only command that answers a negative question. A service that is not in the table was never called, and establishing that costs no payload at all:

service                  calls errors     bytes      p50      max  statuses
redis                      412      0    1.2 MB     2 ms    31 ms  OK×412
payments                     6      1   14.2 KB   240 ms   2.1 s   OK×5 InternalServerError×1

Narrative

failures <report>

For each failing scenario: the error, the failing step in context, its assertions with their messages and file:line, the calls that happened inside that step, and any attachments. Usually the entire answer.

s12  Orders › Checkout fails on a wrong total
  Assert.Equal() Failure
  ✗ s12/2  Then the total is right
      Expected 4173 but found 3902
      at OverviewTests.cs:142
      2 calls in this step:
        s12/i8   payments  POST /charge
        s12/i10  api       GET /order/9

Assertion messages and source locations require a report from Kronikol 3.0.47 or later — see Assertion-Tracking. On an older file the tool says so in a header line.

Prints nothing failed on a green run rather than an empty response.

steps <report> s3

The step and assertion tree with statuses, durations, parameters, doc strings, bypass reasons and attachments, plus an [i12-i39] range on each step saying which calls happened inside it. Also prints the scenario's stableId and example values.

assertions <report> [s3] [--failed]

Every tracked assertion, flat: expression with resolved values, result, failure message, source location.

Assertions reach the data file only when IncludeTrackedAssertionsInStepList is on. With it off they exist only in the diagram, where note finds them.

flow <report> s3 [--step 2] [--service X] [--errors-only]

The scenario as an interleaved sequence — step bars, annotations, and one line per call with status, duration and body pointer. This replaces reading the diagram, at one or two kilobytes against 663.

annotations <report> s3

Example-row markers (Row 3) and fragments injected with DefaultTrackingDiagramOverride.InsertPlantUml, each with the interaction index it preceded. Step and assertion markers are excluded, being already structured in steps. See Tabular-Attributes.

Aggregation

values <report> [s3] --path '$.status' [flags]

SELECT value, COUNT(*) … GROUP BY value where the column is a JSON path evaluated across every matched body:

$.status across 44 response bodies (7 distinct bodies)
  "APPROVED"   ×41   e.g. s3/i12
  "DECLINED"   ×2    s3/i40, s7/i2
  (absent)     ×1    s3/i50
12 calls carried no body
Flag Effect
--path '$.x' required; the full path grammar applies — --path '$.items[*].price' counts every element of every body
--service X, --status 5xx, --method M, --step 2, --grep URI the same filters interactions takes
--where '$.status = APPROVED' body-content predicate, same grammar as interactions --where
--stats numeric summary — count/absent/non-numeric/distinct, min/median/max/sum/mean, with the address of each extreme
--request / --both target request bodies, or both (rows tagged req/resp); the default is the response body, because that is where debugging answers live

Scope is the whole run, or one scenario with an address. Counting is per occurrence, not per distinct body — the question is "what did the system see", and it saw the same body every time it arrived (each distinct body is still parsed only once). A body the path misses is counted as (absent): silence would hide exactly the bug — "one response was missing the field" — this command exists to find. Bodiless calls, calls with no response to evaluate, and non-JSON bodies are footnoted, never silently dropped.

$.total across 44 response bodies
  count 43 · absent 1 · non-numeric 0 · distinct 7
  min 12.5 (s3/i4) · median 380 · max 4173 (s3/i47) · sum 18240.5 · mean 424.2

Payloads

None of these prints a payload that was not named.

interactions <report> [s3] [flags]

One row per request: address, service, method and path, status, duration, and body pointers for the request and the response. Without an address it covers the whole run — rows print full s3/i47 addresses either way.

Flag Effect
--service X substring match
--status 5xx a class, or an exact status
--method GET
--step 2 only calls inside that step
--grep T substring match on the URI
--group fold runs of identical calls into one row with ×N
--where '$.success = false' body-content predicate — see below
--group-by
kronikol query interactions report.json --group-by service,status --sort errors
service      status               calls  errors   median      max  bodies
payments     OK                      38       0    12 ms    80 ms       4
payments     InternalServerError      2       2   230 ms   410 ms       1

Generic bucketing over the index: dimensions (comma list, any order) are service, method, status, path (URI path, query stripped), step, phase, category, kind (metaType) and capturedBy. bodies counts distinct response bodies in the bucket — 120 calls with 1 body is one fact. Status, duration and error classification come from the exactly-paired response and the shared error classifier, so a Created here is a success exactly as it is in services. Index-only unless combined with --where; composes with every filter; default sort is calls descending with --sort errors|duration. It is distinct from --group, which folds adjacent identical calls in sequence order — the two don't compose. services remains the curated view and the only answerer of negative questions; --group-by is the general form.

--where
kronikol query interactions report.json s3 --where '$.success = false'
kronikol query interactions report.json --where '$.items[*].price < 0' --where 'req:$.currency = GBP'

Grammar: [req:]PATH OP LITERAL, with ops = != < > <= >= ~ !~ exists !exists (exists takes no literal) and literals null/true/false, numbers, quoted or bare strings. Both sides numeric means a numeric comparison; otherwise case-insensitive string comparison; ~ is substring. Wildcard paths use any-semantics — $.items[*].price < 0 passes when any element satisfies. Repeated --where is AND (OR is deliberately absent: run the command twice). The default target is the response body; a req: prefix targets the request per-expression, and --request shifts the default. A call whose targeted body is missing or not JSON fails the predicate, and the footer reports how many were excluded that way. --where works on interactions and values. Single-quote the expression — > and [*] are shell-active.

Statuses, durations and response body pointers come from exact request/response pairing on requestResponseId, so interleaved parallel calls to the same service each show their own status.

http <report> s3/i47 [flags]

The interaction in full: direction, participants, method, URI, status, duration, owning step, W3C trace and span ids, phase, dependency category, capture path.

With no payload flag it describes the body and lists the cheap ways to look at it:

body: 2.7 KB · b:4bdea521 (×28 in this report)
--keys to see its shape · --path $.x to pull one value · --body for all of it · --out F to save it
Flag Effect
--headers the header block
--keys the body's shape: a line per path, with type and a sample
--path '$.a.b[2].c' one value — grammar below
--lines 20-60 a window of the pretty-printed body
--body all of it, subject to the budget
--out FILE write it out and print one line

The path grammar (shared by every command that takes a path):

Segment Meaning
.name object property
[2] array index
[*] every element — one row per match, each with its concrete path ($.items[2].price = 4173), paged
['a.b'] bracket-quoted property, for keys containing dots
.length() terminal only: array → element count, object → property count, string → char count

Single-quote paths--path '$.items[*].price' — because [*] is shell-active in both bash and PowerShell. A miss suggests the nearest key that does exist ($.data.custmers is not in this body — nearest: $.data.customers), and a result too big for the budget describes itself — kind, element count, size, the flags that window it — instead of refusing.

body <report> b:4bdea521 [same payload flags]

The same views addressed by content rather than location, plus every address the body occurs at.

note <report> s3/d0 [/n12] [--out FILE]

s3/d0 lists a diagram's notes with sizes; s3/d0/n12 prints one.

A note is what the HTML report displayed, which is a rendering of the captured content rather than a copy of it. FocusFields, phase variants, GraphQL query-only mode, note truncation and user-supplied formatting processors all change it — and a processor can add information that exists nowhere in httpInteractions. This is where to look when someone quotes something the payloads do not contain.

diagram <report> s3/d0 --out FILE

The raw PlantUML. Refuses to print to stdout — a real one is 663 KB — and points at flow instead.

Search and comparison

grep <report> "4173" [--in ...] [--values] [--count]

Returns addresses, not content.

--in defaults to bodies,uris,steps,assertions; add headers and notes. Bodies are searched once per distinct content rather than once per occurrence.

--values names the JSON path a match came from:

s3/i47  $.data.customers[2].total = 4173

This is the command for "the number on screen is wrong" — it finds where the number entered the system, which is a question a fully passing test suite still leaves open.

For numbers, add --number: the needle and every candidate token are compared numerically, across formatting — grep "4,173.00" --number finds the raw 4173, and a body's "4.173,00" (European decimal comma) is read under both separator conventions so it matches 4173 too. On JSON bodies a numeric match is always a value match, so --number always emits paths, and when the matched text differed from the needle it says so ($.display = "4,173.00" (≈ 4173)). --tolerance 0.5 (absolute) or --tolerance 1% (relative) widens the match; the default is exact with a tiny relative epsilon so 4173.0 matches 4173.

trace <report> <id | prefix | s3/i47>

Follows a W3C trace id — the activityTraceId every interaction carries since 3.0.47, the same id in your OTel traces and application logs — across the whole run. Takes the full id, an unambiguous prefix of at least 8 hex characters, or an interaction address (that call's trace):

trace 4bf92f35… — 7 calls across 2 scenarios
  +0 ms     s3/i12   api        POST /orders            202      span 00f067aa
  +12 ms    s3/i14   payments   POST /charge            200      span a1b2c3d4
  +80 ms    s7/i2    payments   POST /charge            500      span e5f60718
! spans 2 scenarios (s3, s7) — shared state or fixture leakage

Rows are chronological with offsets from the first call (file order, flagged, when a timestamp is missing). The cross-scenario warning is the command's second job: a trace id that leaks across scenarios is the classic flaky-test smell, and nothing else in the tool can see it. Parent span ids are not captured, so this is the chronology of the trace, not its tree. An ambiguous prefix exits 2 listing the candidates; an unenriched report is told to re-run on a current Kronikol.

compare <report> s3 s7

Two scenarios side by side: example values, the first differing steps, the first differing calls, and how many bodies are byte-identical — plus the address of the first differing body, ready to paste into diff (first differing body: diff s3/i12 s7/i12). A passing neighbour is the best available oracle for a failing scenario.

diff — bodies and runs

kronikol query diff report.json s3/i47 s7/i47       # two interactions' bodies, one report
kronikol query diff report.json b:4bdea521 b:9f31c02a
kronikol query diff old.json new.json               # two runs, matched on stableId
kronikol query diff old.json new.json --body s3/i47 # the same call across two runs

The body diff answers the most common debugging move — "this call succeeded in the passing scenario, what was different in mine?" — by printing only the paths that differ, never a payload:

- s3/i47  b:4bdea521  2.1 KB
+ s7/i47  b:9f31c02a  2.2 KB

$.customer.region: "EU" → null
$.items: 9 → 10 elements
$.items[4].price: 12.50 → 1250
$.items[9]: (absent) → {sku, price, qty}
$.total: 4173 → 3902

5 paths differ

Identical hashes answer byte-identical straight from the index. An added or removed subtree is one row with a shape summary, never a dump. An array where a single insert shifted every later index collapses to one honest row ($.items: elements shifted/reordered — 9 vs 10, 8 identical) rather than a page of misleading per-index rows. Non-JSON bodies fall back to a line diff. Two scenario addresses are refused with a pointer at compare. Capture-time truncation markers inside a body surface here the same way body surfaces them.

The run diff (two files) reports what broke, was fixed, got materially slower, disappeared — matched on stableId. --body s3/i47 resolves the address in the old report, matches the scenario into the new run by stableId (ordinals shift between runs), and diffs that one call's bodies across the two files.


Older reports

Reports written before Kronikol 3.0.47 carry no stepPath, no assertion failure messages and no annotations. Every command still works against them; the tool prints one header line saying what is missing, so an answer that came from thinner data is never mistaken for a complete one:

! report predates step attribution and assertion detail — stepPath, assertion messages and source
  locations are absent. Re-run the suite on a current Kronikol to get them.

A merged report (see Merging-Parallel-Reports) is also flagged, and a file declaring a mergeableFormatVersion this tool does not understand is a hard error rather than a best-effort read.

Exit codes

Code Meaning
0 answered
1 the report could not be read, or is not valid JSON
2 bad usage — unknown command, malformed address, out-of-range ordinal, ambiguous directory

Using it from an AI agent

Agents reach for Read by default, and on a Kronikol report that ends the session with the question unanswered. Two things prevent it.

The skill. templates/skills/kronikol-test-debugging/ in the Kronikol repository is a Claude Code skill carrying the rule, the four-layer model, the command ladder, recipes keyed to what a user actually says, and the traps. Copy the folder into your project's .claude/skills/:

.claude/skills/kronikol-test-debugging/
  SKILL.md
  references/commands.md
  scripts/query.py      # a smaller fallback for machines without the dotnet tool

The skill's description tells the agent to load it whenever a Kronikol report exists and the question is about test behaviour. The individual files: SKILL.md · references/commands.md · scripts/query.py.

The ladder. What the skill teaches, in one line:

summary → failures | scenarios → steps s3 → services s3 → interactions s3 → http s3/i47 --keys → --path $.x

Stop at the first rung that answers the question. Most stop at the third.

Recipes worth knowing even without the skill:

The question The command
why did these tests fail? failures
the number on screen is wrong grep "<value>" --values then http <addr> --path
did it even call X? services
which example row broke? steps s3 and annotations s3
what broke since yesterday? diff old.json new.json
show me the flow flow s3, never the diagram
the report shows X but I can't find it note s3/d0

See also

Home


Demo


Getting Started

Common Tasks

Integration Guides

Uninstrumentable / polyglot backends

Extensions

Configuration

Features

Reference

Clone this wiki locally