Ingesting External Captures

Overview

Kronikol's report generators read one thing: RequestResponseLog entries in the in-process RequestResponseLogger store. Until 3.0.44 the only way to fill that store was a .NET capturer running in the same process. NDJSON ingestion is the language-neutral on-ramp: any capturer — an out-of-process proxy tap, a Java or Node service, a shell script tailing a log — writes one JSON object per tracked request or response, and kronikol ingest (or IngestPipeline) replays the file into a full TestRunReport.html, the data files and the component diagram.

---

The interaction format

One line per request or response. The shape is exactly the httpInteraction object Kronikol already publishes in TestRunReport.json (same camelCase names), plus the attribution fields a capturer outside the test process must supply.

{"type":"Request","method":"POST","uri":"http://localhost:8081/sidekick","serviceName":"graphql","callerName":"web","content":"{\"query\":\"query Overview { overview }\"}","headers":[{"key":"Content-Type","value":"application/json"}],"traceId":"0af7651916cd43dd8448eb211c80319c","requestResponseId":"6f1c0d8e-9b7a-4c2e-8e1a-2d3f4b5c6d7e","timestamp":"2026-08-21T10:00:01.000Z","testId":"0af7651916cd43dd8448eb211c80319c","testName":"overview › renders"}
{"type":"Response","method":"POST","uri":"http://localhost:8081/sidekick","serviceName":"graphql","callerName":"web","content":"{\"data\":{}}","headers":[{"key":"Content-Type","value":"application/json"}],"statusCode":"200","traceId":"0af7651916cd43dd8448eb211c80319c","requestResponseId":"6f1c0d8e-9b7a-4c2e-8e1a-2d3f4b5c6d7e","timestamp":"2026-08-21T10:00:01.120Z","testId":"0af7651916cd43dd8448eb211c80319c"}

Property	Required	Meaning
type	✔	Request or Response.
uri	✔	Full URI of the call (the arrow label uses path + query). Relative paths are tolerated.
serviceName / callerName	✔	The two participants (receiver / caller).
testId	✔	The correlation key — must equal the report's Scenario.Id byte-for-byte. For browser-driven runs use the test's W3C trace id.
testName		Display name (cosmetic; a tests file wins when both exist).
method		HTTP verb (GET, POST, …) or any custom label (Query, generate [gemma]).
content		Body text, already decoded and capped by the capturer.
headers		[{ "key", "value" }]. Redact secrets — or rely on --redact at ingest.
statusCode		Responses only: "200" or a custom label.
requestResponseId		Pairs a request with its response (one arrow). Any string; non-UUIDs are hashed to a stable GUID.
traceId		Groups the hops of one call chain. Defaults to requestResponseId.
timestamp		ISO-8601. Used for ordering (the diagram follows timestamp order at ingest) and loop durations.
dependencyCategory / callerDependencyCategory		A DependencyCategories value (BigQuery, AI, Redis, …) for shape/colour.
phase		Setup / Action / Unknown.
metaType		Default or Event (fire-and-forget styling).
activityTraceId / activitySpanId		W3C ids for cross-linking to Tempo/Jaeger.
trackingIgnore		true to store but not draw.

Unknown properties are ignored, so capturers can add their own diagnostics. Anything you want rendered goes in content (the diagram note) — e.g. fold tokens/sec or a coalescing verdict into the JSON body.

In .NET the record is Kronikol.Ingestion.InteractionRecord (FromLog, ToLog, Pair(...), ToJson, FromJson); NdjsonInteractionWriter is an IRequestResponseSink that appends lines (thread-safe, flushed per line, tail-able), and NdjsonInteractionReader reads them back.

---

The tests format (optional)

A companion NDJSON of start / step / end events supplies each scenario's outcome, duration and steps:

{"event":"start","testId":"0af7651916cd43dd8448eb211c80319c","testName":"overview › renders","feature":"overview.spec.ts","timestamp":"2026-08-21T10:00:00Z"}
{"event":"step","testId":"0af7651916cd43dd8448eb211c80319c","text":"open the overview","timestamp":"2026-08-21T10:00:00.5Z"}
{"event":"end","testId":"0af7651916cd43dd8448eb211c80319c","status":"failed","durationMs":9000,"error":"expected 1 got 2","timestamp":"2026-08-21T10:00:09Z"}

• status accepts the Playwright / Jest / JUnit vocabularies: passed, failed, timedOut, interrupted, skipped, pending, bypassed (anything unknown is Failed).
• feature groups scenarios under one heading (a spec file, a class); without it they land in the default feature (Ingested, or --feature).
• Tests seen only in the interaction file still become scenarios (verdict Passed by default — there is no outcome information without an end record).
• Make the tests file per-run (truncate it at run start). Kronikol renders exactly what it is handed; a file accumulated across runs will render every old test too.

.NET types: TestRunRecord, NdjsonTestRunReader, FeatureSynthesizer.

---

kronikol ingest

dotnet tool install --global Kronikol.Tool
kronikol ingest ./captures --tests ./captures/tests.ndjson -o ./Reports -t "E2E run"

Option	Default
<inputs…>	—	Files, directories (searched recursively for *.ndjson / *.jsonl) or globs.
--tests <file>	—	The tests NDJSON.
-o, --output <dir>	./Reports	Output directory (absolute or relative).
--render <mode>	browserjs	browserjs (client-side, needs internet at view time), nodejs (offline SVG, needs node), local, server.
-t, --title <text>		Report title.
--feature <name>	Ingested	Feature for tests without one.
--collapse / --no-collapse, --collapse-threshold <n>	on / 2	Collapse consecutive identical calls into a loop ×N fragment.
--max-arrows <n>	unlimited	Cap pairs per diagram (… +N more calls omitted …).
--no-component-diagram		Skip ComponentDiagram.html.
--no-redact, --redact-header <name>	redact on	Capture-time redaction of credential headers during replay (Capture-Time-Redaction).
--allow-empty		Generate even when nothing was ingested.

Exit codes: 0 ok, 1 runtime failure (nothing found, malformed line), 2 usage.

Programmatic

var options = IngestPipeline.DefaultOptions();   // BrowserJs, no internal-flow, component diagram, collapse on
options.ReportsFolderPath = ".logs/kronikol";
var result = IngestPipeline.Run(new IngestRequest
{
    InteractionFiles = Directory.GetFiles(".logs/taps", "*.ndjson"),
    TestsFile = ".logs/taps/tests.ndjson",
    Options = options,
});
Console.WriteLine(result.TestRunReportHtml);

The pipeline reads everything, clears the store (ClearExistingLogs), replays in timestamp order, normalises each log's TestName from the tests file, resets the diagram cache, synthesises Feature[] and calls ReportGenerator.CreateStandardReportsWithDiagrams. Because it goes through RequestResponseLogger.Log, RequestResponseLogger.Redaction applies during replay.

---

Design notes

• Correlation is Scenario.Id == testId, byte-for-byte. testName is cosmetic.
• Order is enqueue order. The sequence diagram never sorts by timestamp itself; the pipeline sorts for you.
• Multiple hops of one call chain (web→graphql, graphql→data-insights) are separate pairs with the same testId — they all render in the same scenario, in time order.
• Parity. The same schema is the contract for the Java (Kronikol4J) and Node (@kronikol/*) ports — see JAVA_PORT_PLAN.md / NODE_PORT_PLAN.md in the repo.