Skip to content

The Clinic's run records, data-v3 (the hosted readers, adds to data-v2)

Choose a tag to compare

View all tags
@kltownsend kltownsend released this 13 Nov 00:32
· 3 commits to main since this release
data-v3
8626751

The Clinic's run records (data-v3)

The recorded conversations behind the Jev and Bivia cells on labs.layer2c.com/clinic and in Layer2C Lab 030, Examples aren't portable. Validate them per model. Released by Keith Townsend, The CTO Advisor, on 2026-11-12.

What's new since data-v2

data-v3 adds the 18 hosted-reader cells. It doesn't repeat data-v2's 28 cells, which stay current there, so download both for the full set.

  • Jev (TypeSafe's hosted System One model, jev-1.14.0) in the reader's seat: arm E zero-shot, arm E with the cb1 examples, and arm F zero-shot, on payment, schedule and doctor. 9 cells.
  • Bivia (Articul8's hosted System One model, dec-1, fast mode) in the same seat, the same 9 cells.

Each is 48,000 conversations on the same frozen bench as Lab 029's cells (instrument hash 0f3e1c09488db424), with no infrastructure failures left.

How these cells differ from the others

  • The reader is a hosted API. The clients that called Jev and Bivia are mine. Each closed question went to Jev as one choice question and to Bivia's /choice endpoint, with the driver's options plus "unclear." Bivia's abstain is read as unclear. Jev has no extraction type, so identity extraction returns nothing for Jev; Bivia's goes to its /extract endpoint.
  • The cb1 examples went into the state field. Neither API has a field for examples, so the six labeled cases per question went in as text ahead of the case. That's my adaptation, not either vendor's tuning.
  • Every response's model id is logged. Each session's usage.model_ids records the id the vendor returned, and every cell was checked against it before it was built.
  • The reader's logged answers include the vendor's raw response: Jev's choice, confidence and probabilities, Bivia's value, probability, status and abstain reason.

What's here

  • One archive per cell, <cell id>.tar.gz, 18 in all, the same layout as data-v2: <id>/sessions/*.json, <id>/summary.json and <id>/cell.json, cleaned of machine addresses.
  • MANIFEST.json: every archive's SHA-256, its variants, its session count, and the session-set hash the Clinic's source manifest pins.

Check a download with:

shasum -a 256 <cell id>.tar.gz

and compare it with the cell's entry in MANIFEST.json.

What the records are, and aren't

  • The patients are synthetic. No real patient or health data is in any record.
  • Each cell is one run of its reader. Run-to-run variation isn't in any interval, and a hosted model can change behind the same id.
  • Two known defects are in the patient lines, left in by ruling. A clear refusal can name the slot just offered ("No, I can't do that one. Here's what I can do. Thursday at 8:30 in the morning works for me.") without saying what was refused. A contradictory disclosure sounds like a clean correction. Lab 030 reports every metric with and without the first.
  • Jev and Bivia ran arm F without examples only.

Disclosure

Jev ran on my own TypeSafe account; TypeSafe billed $17.68. Bivia ran on Articul8's free tier. Articul8 is a past sponsor of The CTO Advisor's work. It didn't pay for these runs, but at my request it raised the account's request limit partway through them, on November 10, 2026. Neither vendor reviewed these records.

Assets 21
Loading