Skip to content

Operability, vocabulary teach-back, language pin, reframe, and dry-run fixes - #14

Merged
daveh-beep merged 5 commits into
mainfrom
agent/teaching-vocabulary
Jul 30, 2026
Merged

Operability, vocabulary teach-back, language pin, reframe, and dry-run fixes#14
daveh-beep merged 5 commits into
mainfrom
agent/teaching-vocabulary

Conversation

@daveh-beep

@daveh-beep daveh-beep commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

What setup experience does this improve?

One bundle, five independently reviewable commits. Gaps addressed: (1) #12 landed OPERABILITY.md but nothing routed to it; (2) setup Q4 surfaced everyday synonyms with nowhere to put them; (3) nothing let another repository consume a chalkline system; (4) the README undersold the problem scope; (5) the first end-to-end dry run surfaced seven protocol ambiguities.

Commit 1 — Wire the operability loop into the docs agents and users actually read

  • README — describe the current loop (consult, surface gaps or conflicts, draft or stop, review and decide, revise, recalibrate) and mention the reviewer-facing usage report.
  • AGENTS.md — new Usage reports section pointing at OPERABILITY.md: optional, kept separate from requested copy, never proof of compliance.
  • setup/PROTOCOL.md — Phase 5 growth paths include the operability handoff.

Commit 2 — Vocabulary and teach-back

Slimmed re-cut of the red-teamed teaching-loop experiment:

  • New optional reference: references/vocabulary.md — everyday phrases → canonical terms; governs conversation, not output.
  • AGENTS.md teach-back contract — fulfill first; one teach-back per reply max; conversational framing only; urgency/opt-out/teaching: "off" gates; never a mistake record.
  • Calibration pairs gain a one-line "What this teaches" note; Meridian example gains a completed vocabulary.md.

Kept from red-team hardening: countable cap, deliverable fence, urgency gate. Dropped: the in-repo conformance harness — template stays runtime-free.

Commit 3 — Language pin: let other repos declare this writing system

A consuming repository adds a short block to its own AGENTS.md naming the language repo and a commit. Agents there consult the pinned revision (unreadable = reported gap, not a guess), cite it in usage reports, and surface multi-repo conflicts to a human. Bumping the commit is how approved language changes reach consuming repos. Wired in README, AGENTS.md, OPERABILITY.md, and setup Phase 5.

Commit 4 — Reframe the problem (two sentences)

Tagline scopes chalkline to the part of a team's language it can actually record, inside the larger truth that teams brief agents, reach customers, and define products in words. Problem statement names prompt-restatement drift as the failure mode.

Commit 5 — Protocol fixes from the first end-to-end dry run

Scripted setup run (fresh fictional brand so the shipped Meridian example couldn't be cribbed) plus a consumer-phase test through a language pin. Full field report on #2. Fixes:

  • Q2 probes skip what the answer already settled
  • Q6: channels.md requires channel-specific rules, not named surfaces
  • Q7: record where canonical exact-wording copy lives (stable link/path, or owning source when paper-only)
  • Phase 2: edited rules count as approved-as-edited; product-fact candidates routed to product docs, never offered as writing rules; absence-inferred patterns are weak candidates
  • Phase 3: vocabulary disambiguation note when one everyday phrase maps to two canonical terms
  • Phase 4: ambiguous supplied facts ("06/02") — ask or keep original form; reformatting is language, reinterpreting is a fact decision
  • Phase 5: diff review pairs the full diff with a one-line-per-file summary
  • AGENTS.md: frontmatter contract now states the file-default + inline (must) enforcement model the examples already practiced

Verified in the dry run: fact/language boundary held both directions (setup refused product facts; consumer flagged missing closing time instead of inventing it); exactly one teach-back fired, conversationally, never in copy; pin consumer verified the revision and cited it in its usage report.

Docs only across all five commits; no runtime, schema, or new tooling. Replaces #13.

Safety and verification

  • I used only fictional, redacted, or explicitly publishable material.
  • I did not include secrets, personal or customer data, confidential material, or private links.
  • Examples preserve supplied facts and do not invent product state, causes, dates, deadlines, guarantees, names, amounts, eligibility, or actions.
  • I ran or documented the relevant setup or harness check (end-to-end dry run: setup, generation, calibration pair, pin consumption, teach-back, usage reports — report on Dry-run: Meridian setup end-to-end #2).
  • My commits include a DCO sign-off (git commit -s).

daveh-beep and others added 2 commits July 29, 2026 14:58
PR #12 added OPERABILITY.md, but nothing routed to it: README still
described the pre-#12 loop (consult, apply, report gaps), AGENTS.md
never mentioned usage reports, and setup ended without handing the
user to the operating loop.

- README: describe the current loop (consult, surface gaps or
  conflicts, draft or stop, review and decide, revise, recalibrate)
  and mention the usage report.
- AGENTS.md: add a short 'Usage reports' section pointing at
  OPERABILITY.md — optional, reviewer-facing metadata, kept separate
  from requested copy, never proof of compliance.
- setup/PROTOCOL.md: Phase 5 growth paths now include the operability
  handoff.

Docs only; no new files, rules, or runtime.

Co-Authored-By: goose <goose@block.xyz>
Signed-off-by: David Hamilton <daveh@squareup.com>
… term gently

Adds an optional references/vocabulary.md mapping everyday phrases to
canonical terms so agents can meet people in their own words, plus a
conversation-only teach-back contract in AGENTS.md:

- fulfill the request first; at most one teach-back per reply
- conversational framing only, never inside deliverables
- skip when the canonical term was already used, when the request is
  urgent, when asked to stop, or when teaching: "off"
- never log, score, or report anyone's word choices

Setup Q4 now records everyday synonyms without judgment and generates
vocabulary.md only when the interview surfaces real phrase pairs.
Calibration pairs gain a one-line "What this teaches" note so each
pair doubles as a teaching artifact.

Slimmed re-cut of the red-teamed teaching-loop experiment
(origin/teaching-loop): keeps the one-per-reply cap, deliverable fence,
urgency gate, and no-mistake-record rules; drops the in-repo
conformance harness to keep the template runtime-free.

Co-Authored-By: goose <goose@block.xyz>
Signed-off-by: David Hamilton <daveh@squareup.com>
@daveh-beep daveh-beep changed the title Vocabulary and teach-back: understand everyday words, teach the house term gently Operability follow-through + vocabulary teach-back Jul 29, 2026
@daveh-beep
daveh-beep changed the base branch from docs/operability-alignment to main July 29, 2026 19:19
The convention that turns a chalkline repo from a place agents visit
into a dependency other repos declare. A consuming repository adds a
short block to its own AGENTS.md naming the language repo and a commit:

- agents there consult the pinned revision, not whatever is newest
- if the revision is unreadable, that is a reported gap, not a guess
- the pinned revision is cited in usage reports so reviewers can
  reconstruct what was read
- bumping the commit is how an approved language change reaches
  consuming repos — deliberate, visible, diffable
- multiple applicable language repos surface to a human; precedence
  across systems is not an agent decision

Wired in four places so it reads as one convention: README (the pin
block itself), AGENTS.md (how to behave when routed here by a pin),
OPERABILITY.md (scope and boundaries), setup Phase 5 (growth path).

Docs only; no runtime, schema, or new files.

Co-Authored-By: goose <goose@block.xyz>
Signed-off-by: David Hamilton <daveh@squareup.com>
@daveh-beep daveh-beep changed the title Operability follow-through + vocabulary teach-back Operability follow-through, vocabulary teach-back, and the language pin Jul 29, 2026
The tagline now scopes chalkline to the part of a team's language it
can actually record — voice, terminology, vocabulary, channel rules —
inside the larger truth that teams brief agents, reach customers, and
define products in words. The problem statement names the concrete
failure mode: every prompt, agent file, and skill restates the team's
voice from memory, drifting a little each time.

Two sentences changed; no new claims. Writing stays the proven first
domain.

Co-Authored-By: goose <goose@block.xyz>
Signed-off-by: David Hamilton <daveh@squareup.com>
@daveh-beep

Copy link
Copy Markdown
Collaborator Author

🤖 Added commit 4: two-sentence README reframe — the tagline scopes chalkline to 'the part of your team's language you can agree on' (voice, terminology, vocabulary, channel rules), and the problem statement names prompt-restatement drift as the failure mode. No new claims.

A scripted setup run (fictional Larkspur tool-lending co-op, fresh
brand so the shipped Meridian example could not be cribbed) plus a
consumer-phase test through a language pin surfaced seven protocol
ambiguities. Fixes, each pointing at the observed stall:

- Q2 probes: skip any probe the answer already settled — chatty users
  answer all three adjectives at once
- Q6: create channels.md only when channel-specific rules surface;
  naming places writing shows up isn't enough ("kind of everywhere?
  email, flyers..." previously matched both clauses)
- Q7 exact wording: ask where the canonical copy lives; record a
  stable link or path, or name the owning source when it's paper-only
- Phase 2: edited rules count as approved-as-edited (provenance
  tally); product facts (prices, hours, durations, mechanics) are
  named as facts and routed to product docs, never offered as writing
  rules; absence-inferred patterns are weak candidates, dropped
  without argument
- Phase 3 vocabulary: one everyday phrase mapping to two canonical
  terms ("booking" = loan and hold) gets a disambiguation note
- Phase 4: ambiguous supplied facts ("06/02") — ask or keep the
  original form; reformatting is language, reinterpreting is a fact
  decision
- Phase 5: diff review needs a one-line-per-file summary; 200 added
  lines is not a reviewable unit on its own

AGENTS.md: the frontmatter contract now states what the examples
already practiced — file-level enforcement is the default, a single
rule may carry an inline (must) at the same bar, unmarked rules
inherit the file default.

Full run transcript and findings filed on issue #2.

Co-Authored-By: goose <goose@block.xyz>
Signed-off-by: David Hamilton <daveh@squareup.com>
@daveh-beep daveh-beep changed the title Operability follow-through, vocabulary teach-back, and the language pin Operability, vocabulary teach-back, language pin, reframe, and dry-run fixes Jul 29, 2026
@delkc
delkc self-requested a review July 30, 2026 13:19
@daveh-beep
daveh-beep merged commit 2a8a2c2 into main Jul 30, 2026
2 checks passed
@daveh-beep
daveh-beep deleted the agent/teaching-vocabulary branch July 30, 2026 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants