Skip to content

OntoSmith 0.1.2

Choose a tag to compare

@Dongbumlee Dongbumlee released this 02 Nov 03:46
· 131 commits to main since this release
0647611

Added

  • Identity seat seam — guard_triples accepts an
    optional identity_seat: an injected judge that
    decides rule 4's completion question (whether a
    fragment mention completes to a longer census name)
    per candidate, from a code-computed feature envelope.
    The seat answers complete / stand / no opinion; no
    opinion defers that candidate to the rule chain. The
    default is no seat — the deterministic rule chain,
    unchanged. The batch path wires the seat from a
    profile key, identity_seat_decisions: a frozen
    JSONL file of verdicts an offline judge produced for
    the corpus, keyed by document hash. A declared file
    that cannot be loaded fails the stage closed, and
    the run metrics record the file's hash and how often
    the seat was consulted and ruled. The seam is the
    measured outcome of the identity-surgery program
    (2026-10-30): on a frozen 245-point gold the seat
    matched the rules' accuracy with zero
    referent-changing completions (the rules made two),
    and on fresh documents it completed the legitimate
    full names the rule chain refused (4/4 against 0/4)
    at the cost of one title misnaming. The probe replay
    through this wiring reproduces the measured seat
    outcomes byte for byte.

Changed

  • Upstream attestation program — the candidate
    guard now reads a document the way a reader does:
    against the entities the document itself establishes,
    not raw strings alone. Attestation is tiered
    (verbatim; the document's stipulated aliases —
    defined terms and acronym pairs collected into a
    per-document registry; punctuation-spacing collapse
    for tokenized corpora; a proposer-declared derivation
    surface, behind the profile-gated declared-surface
    prompt variant). A defined term is an identity
    inside its document: the role-word guard fires only
    for undefined role nouns, and a defined term with a
    stated referent canonicalizes to it. The
    theme-presence block is retired in favor of grading:
    a verb+preposition proposal whose object attests in
    the document but not in its quote is kept with a
    theme_grade for the judge instead of being blocked
    wholesale (theme_attest_chunk declarations remain
    accepted and are subsumed). Identity resolution no
    longer rewrites a complete attested name into a
    longer name that merely contains it (a city is not
    its city council), never resolves into a coordination
    surface, and still completes surnames and generational
    suffixes. Duplicates are recorded in the blocked
    sidecar (duplicate) instead of vanishing uncounted.
    Measured on recorded corpora replayed offline
    (current guard vs fixed guard, identical inputs):
    contract-slice role-word blocks 89 → 19, kept
    candidates 4,965 → 5,193; novels theme blocks 626 → 0
    (491 graded, the rest attesting in their evidence
    window); wiki seat kept-stage recall 0.186 → 0.210.
  • Documentation polish — the README gains a
    worked-examples section (three edges from the
    project's own records: a head of government named
    inside another article, a novel's character entering
    by relation, a contract's defined term as an
    identity), and the project site is rebuilt around
    the same examples, with the hero graph drawn as an
    animated SVG. The Connections recommended-models
    table no longer lists the retired GLiNER spotter as
    a current seat. No measured claim changes.

Added

  • Open Domain pack 0.5.0: TN-3/4/5 data adopted (RE-TN-1) —
    packs/open-domain/tn_table.json goes to version 2: the
    TN-3 relation groups, TN-4 inverse expansions and TN-5
    subsumptions whose code support landed with the scorer
    change below are now adopted as data, under registered
    evaluation RE-TN-1 (conformance entry DC-6), for the
    EXTENDED figure only. Dual reporting is mandatory: the
    frozen TN-1/TN-2 figure is always reported beside the
    extended one, and the delta is labeled an instrument
    change, never a quality gain (on the frozen cap-b records
    against gold v1.1.0: kept predictions 273 → 335, proposal
    union 301 → 364; baseline hits lost 0, unverified new
    hits 0). The pack manifest pins the table as an
    in_force artefact, and verify_cli.load_table now
    reads the pack-native tn_table format directly, so the
    shipped file works with --normalization as-is. The
    extraction payload is unchanged (same sha256 pin).

  • Scorer: alias-aware matching + opt-in TN-3/4/5 layers —
    match_records now also matches a predicted record under
    the attested alias surfaces in its guard-emitted
    canonicalized field (the guard's identity resolution
    rewrites endpoints to one canonical surface; the frozen
    scorer matched only the emitted surface, which read the
    upstream-attestation fix as a recall regression on the
    cap-b replay: 262 vs the old guard's 273 — with alias keys
    the same output scores 301). Alias keys are the cross
    product of attested subject x object surfaces, mirroring
    gold_keys, so a record canonicalized at BOTH endpoints
    still matches its gold pair. NormalizationTable gains
    three caller-data layers, all default-empty (byte-identical
    to the TN-1/TN-2 instrument): TN-3 relation groups, TN-4
    inverse expansions, TN-5 subsumptions. Grouping ADDS keys
    on both sides — verbatim keys are always retained, so a
    group can never break a TN-2 pairing (verified on the
    frozen cap-b corpus: baseline hits lost 0; hits beyond the
    frozen instrument plus the hand-verified variant set 0).
    verify_cli.load_table reads the new keys. Enabling the
    layers as a pack's table is a new registered evaluation
    (the open-domain table is frozen; dual reporting per its
    effect_reporting rule) — measured on cap-b/gold v1.1.0, the
    full stack would read 366 candidate-level hits vs 273
    frozen (union 364 vs 301); the table data itself is NOT
    adopted here.

  • Mention layer, first cut (issue #54) — profile-gated
    (mention_layer in setup_config; absent = off, and the
    undeclared path is byte-identical). Entities register by
    noun surface forms, so facts stated via pronouns, definite
    descriptions, or cross-sentence evidence had no citable
    path to an entity and drops recorded only a score. With
    the layer declared, the free proposer emits per-endpoint
    mention spans (verbatim, as they appear in the quote),
    mention types (named / pronominal / descriptive), and —
    for non-named mentions — a resolution proposal: an
    antecedent span, verbatim from the document. The guards
    verify both spans with the same discipline as quotes;
    an unverifiable span blocks the candidate by name
    (mention_missing / mention_span /
    mention_antecedent), never silently. GATES judges each
    resolution as its own Jev decision before the fact:
    clear proceeds, ambiguous (review band) is capped at
    Tier-2 review, rejected drops with reason class
    resolution_rejected — distinguished from
    fact_rejected in the drop accounting. The quote is
    never rewritten; resolution travels as annotation.
    Measured (Re-DocRED 40-doc A/B vs the cap-25 recipe,
    pre-registered criteria): the first cut does not ship
    as a recall feature.
    Candidate recall was flat
    (0.1189 vs 0.1183, with a genuinely different proposed
    slice — 46 gold proposed only with the layer on), and
    resolution judging behaved as designed (58 clear /
    3 rejected / 1 review of 62), but Tier-1 recall fell
    to 0.0772 from 0.1066 (paired net −43): the guards
    blocked 69 of 257 proposals as mention_missing
    because the proposer's mention-field compliance is
    unreliable, culling candidates before judging. Raw
    Tier-1 precision rose (0.4912 → 0.6038) and cost held
    ($0.00268/doc vs a $0.00312 ceiling). The layer stays
    an experimental opt-in with that measured trade-off;
    the full record is in the cap-recall measurement
    (Phase 2 section) and issue #54.

  • Entity sweep (issue #58) — profile-gated
    (entity_sweep in setup_config; absent = off, and
    the undeclared path is byte-identical; a non-boolean
    value fails closed). An additive second proposal
    pass inside CANDIDATES' free-proposer branch: a
    deterministic code census names a document's
    entities, the ones appearing as no endpoint of any
    chunk-pass item become anchors, and each anchor is
    re-asked in its own mention view under the chunk
    prompt plus one anchor rule and one evidence rule:
    every sweep triple must carry a verbatim evidence
    span copied from the text, verified by code against
    the full document at proposal time — a missing or
    unverifiable span voids the proposal before the
    guards, counted in the finish record as
    evidence_missing. Sweep items pass the
    same guards and merge behind the chunk channel —
    duplicates and off-anchor items are counted, never
    shadowing or replacing chunk provenance — and the
    finish record restates the pass (candidates_sweep:
    anchors, proposed, duplicates, off-anchor,
    evidence_missing, blocked
    by reason, shipped Tier-1). Measured on the
    pre-registered 8-document density pilot against lab
    gold v1.1.0 (issue #58): the pass recovered 33 of
    139 never-proposed gold labels at Tier-1 (paired
    net +61 Tier-1 over the pilot docs), but the pooled
    new-Tier-1 adjudicated precision was 0.676 against
    a 0.90 bar and the arm cost $0.02423/doc against a
    $0.00375 ceiling — the pilot failed on precision
    and cost, the full-corpus run was not licensed, and
    the pass stays an experimental opt-in pending a
    redesign decision. A pre-registered cut 2
    (2026-10-29) made the evidence span load-bearing —
    the void fired zero times on the re-run (the
    proposer attached a verifying span to all 272
    proposals) and precision measured 0.657 against
    the same 0.90 bar, so the pilot failed on
    precision again; yield and cost passed (P1
    35/139, paired net +67, $0.0240/doc against a
    re-registered $0.0323 bar), and the sweep line
    returns to a close-or-redesign decision. The full
    record is in the
    cap-recall measurement (Entity sweep sections) and
    issue #58.

Fixed

  • Identity completion damaged attested names the
    word list did not know.
    The candidate guard's
    completion step rewrites a fragment mention to the
    longer census name that contains it; a curated list
    of generic elements decided which containing phrases
    were safe to absorb into. The list missed glued
    surfaces in other shapes — verb and prose glue,
    demonym prepends, a show title wrapped around a
    country — and a per-language list cannot travel to
    the next genre or language anyway. The list is
    deleted. A subset rewrite of a mention now requires
    document-supplied identity evidence for the added
    material, whatever the mention's length; the one
    exception is an infix completion (middle-name
    material) of a mention that occurs nowhere outside
    the candidate, where refusing would protect no
    competing name and cost the fact. Leading
    completions (glue, specifiers) and trailing
    completions (derived entities) always need
    evidence. A mention with its own textual life in
    the document and no evidence for the longer form
    stands as proposed — the triple is kept, the name
    uncompleted — instead of being rewritten. The
    regression record is
    tests/test_identity_surface_damage.py.

  • The cost-cap preflight checked the wrong stage set.
    _preflight_cost_cap selected its seats by testing
    stage in stages (completed stages) where its own
    docstring and both sibling preflights use pending-stage
    semantics (not in). On a fresh run under a USD cap with
    an unmetered proposer binding, the preflight therefore
    examined nothing, and the refusal arrived only from
    _record_cost — after CANDIDATES had already spent.
    The three conditions now use pending semantics, with
    regression tests pinning which seats a fresh run, a
    live-decide run, and a resumed run must check. Found in
    the 2026-10-29 implementation audit.

  • Review-band cases were keyed by document alone.
    The interactive reviewer's resume set and undo jump
    used doc_id as a case's identity, so a document's
    second band case was silently retired by its first
    case's label (never shown, never recorded) and undo
    could land on a different triple from the same
    document. Cases are now keyed by the full triple
    (doc_id, subject, relation, object); label rows carry
    the key, and legacy doc_id-only rows still suppress
    their whole document (the data cannot say which case
    they covered).

  • Stale install and diagram text after the pipeline
    retirement.
    The README's install section still
    offered the removed [nlp] extra ("the pipeline
    candidate path's local NER") in the present tense —
    inherited by the PyPI variant, whose generator anchor
    pinned the stale paragraph, which is why the doc
    guards stayed green. The paragraph and the anchor now
    name only the surviving extras ([viz], [neo4j]). The
    pipeline-flow diagram's NER subtitle ("entity roster
    (mentions)") and CANDIDATES subtitle ("pairs
    enumerated") likewise described retired mechanics;
    they now read "contract rows only (local NER
    retired)" and "LLM proposer (the only source)", and
    the figure is re-rendered.

  • Queue claim success read a connection cumulative.
    Queue.claim tested total_changes (cumulative for
    the connection, effectively always true) instead of
    the statement's rowcount; the UPDATE was correctly
    guarded, so no duplicate processing was observed, but
    a racing second claimer could have double-processed
    an item. Now cursor.rowcount. Also from the audit:
    the ontology_judge return annotation claimed a
    3-tuple; it returns (and callers unpack) two values.

  • Fragment endpoints became separate graph nodes.
    The proposer types endpoints in the surface the local
    text uses — a first name alone, a nickname, a surname
    alone, a name missing its suffix — and the graph keys
    nodes by name string, so one entity fragmented into
    several nodes (and a bare surname could wire a
    relation to the wrong bearer). The census roster
    itself fed the problem: it glued a title line to the
    body across a paragraph break, glued sentence-initial
    prose words onto names ("During Alger"), split a
    quoted nickname out of its name ("Wilfried" /
    "Willi" / "Schneider" as three surfaces for
    Wilfried "Willi" Schneider), and split spaced-hyphen
    compounds ("Bernheimer - Haus"). The census now
    keeps sequences inside paragraphs, drops
    sentence-initial prose words, bridges quoted
    nicknames (recording them, and the title form, as
    aliases of the full name), and bridges spaced
    hyphens; and guard_triples — the one funnel every
    proposal channel passes — resolves each endpoint
    against the roster before attestation: a fragment
    naming exactly one canonical surface is rewritten
    to it (recorded as canonicalized), one naming
    several is flagged (identity_ambiguous) and never
    guessed, and attestation reads the effective
    endpoint through the entity's recorded mention
    forms so the canonical name is not blocked for
    never occurring contiguously. Blocked records still
    carry the item exactly as proposed. Deterministic
    code only; no model is consulted on identity.
    Replaying the sweep pilot's six adjudicated
    identity failures through the fixed guard: three
    now resolve to the canonical entity, three are
    flagged ambiguous (their documents genuinely carry
    several bearers of the name) instead of shipping
    silently. Issue #58.

  • The serving index silently covered only part of a
    build's corpus.
    INDEX projected chunks only for
    documents that had produced Tier-2-routed candidates,
    so a build whose candidates landed mostly Tier 1 served
    a subset of its documents — in the multilingual pilot's
    build, 1 of 16 — while VERIFY passed. INDEX now chunks
    every built document (each chunk tagged with its
    document's best tier), and VERIFY carries a
    serving_coverage check that re-derives the expected
    coverage from the batch's texts and fails by name when
    it is partial. A green VERIFY now means coverage is
    complete. Found by the multilingual pilot (issue #50);
    the flagship build's published recall figures were
    re-checked against its serving projection and stand
    (509/509 documents).

  • ER pairs held for human settlement were invisible in
    the batch's own accounting.
    WRITE noted the held
    count, but the finish line said nothing, and a build
    could ship with every held pair unactioned in silence
    (the pilot: 44 of 49 ER decisions). The finish line now
    accounts for ER decisions by terminal state (merged /
    settled merge / settled keep-separate / unactioned),
    records the account in the batch metrics, and prints
    an ER SETTLEMENT PENDING line naming the settle
    command when unactioned holds remain. The settlement
    policy itself is unchanged — the review band still
    never auto-merges (issue #51).

  • CANDIDATES guard blocks were invisible in the
    batch's finish record.
    The stage totals its blocks
    by reason in its own metrics and names them on its
    log line, but the finish — the record a reader signs
    off on — restated only the ER account, so a guard
    cliff was indistinguishable at the finish from an
    ordinary sparse run (the mention layer's measured
    Phase 2 cull: 69 of 257 proposals blocked as
    mention_missing). The finish now carries the
    by-reason block counts in its metrics
    (finish.candidates_blocked) and names them on its
    log line, alongside the ER account. Batch-level tests
    pin the surfacing, characterize the cliff under a
    scripted partial-compliance proposer at the measured
    27% mention-field omission rate, and pin equal
    candidate survival for compliant input with the layer
    on and off. The guard policy itself is unchanged
    (issue #54).

  • Extraction density on long documents was capped and
    unreported.
    The free proposer's per-chunk ceiling
    (10 proposals per 20,000-char chunk, a CUAD-recipe
    constant) bound every pack and document shape, and
    nothing reported it: batch metrics now carry the
    per-document candidate distribution (min / median /
    max over every built document) and the CANDIDATES log
    prints it plus the count of chunks that reached the
    ceiling. The ceiling is now recipe data a profile may
    declare (proposer_max_per_chunk, registered and
    validated like every setup_config key); undeclared
    keeps the core default, byte-identical prompts
    (issue #52).

  • Non-Latin queries retrieved nothing. The query
    tokenizer was [a-z0-9]+, so a Korean-only query
    produced zero tokens — the lexical arm never fired and
    the graph arm's overlap sets were empty — and the
    arms' length filter also dropped dense two-character
    Hangul content words. Tokenization is now script-aware
    (Unicode word tokens; byte-identical streams on ASCII
    input, so the flagship evaluation mirror is unchanged),
    the content-token rule counts two-character non-ASCII
    tokens, and the full-text indexes accept a per-deployment
    analyzer via ONTOSMITH_FULLTEXT_ANALYZER (unset keeps
    the stock analyzer and the pre-fix schema). Whitespace
    tokenization is not morphology; the limit is stated in
    the serving guide (issue #53).

  • Hands-on Track 1 §4 named the wrong store. The
    finish line printed the file-store serve banner and
    promised ranked documents unconditionally; a reader
    arriving from §3 with the store variables still
    exported is served by the graph store instead, and
    its Documents section depends on whether the build
    projected Tier-2 chunks. The section now shows the
    graph-store banner, states the environment rule that
    picks the store, and conditions the ranked-documents
    promise on chunks being present. Found by the
    retroactive vanilla-VM verification of Tracks 1–3.

  • Hands-on Track 3 could not be followed verbatim.
    §1 printed a stale skeleton schema hash (the draft
    tool's deterministic output has moved since the
    capture); §2's validate block dropped the field-path
    prefixes the tool prints; §4's emit command named an
    approval file dated from the capture, which a reader
    running the track today does not have. The hash is
    refreshed, §2's block matches the tool's output, §3
    now states that the approval file's name and the
    schema hash are the reader's own, and §4 takes the
    approval file by placeholder. Found by the same
    retroactive verification.

Changed

  • Mention layer: absence now falls back instead of
    blocking (issue #54, cut 2).
    The first cut's guard
    treated an endpoint whose mention data was absent the
    same as an invalid mention, and the Phase 2 A/B
    measured the cost: 69 of 257 proposals culled as
    mention_missing on the proposer's unreliable field
    compliance, Tier-1 recall 0.1066 -> 0.0772. The layer
    is now additive by policy: an endpoint with no mention
    data proceeds under the legacy evidence path — judged
    on its quote exactly as without the layer — while a
    mention that is declared but invalid (span outside
    the quote, antecedent absent / not verbatim / naming
    another entity, malformed type) still blocks by name.
    Declared, valid resolutions are judged by GATES
    exactly as before. Every fallback is counted by
    endpoint in the stage metrics and restated in the
    finish record (candidates_fallback), and candidates
    name their fallback endpoints on the record, so the
    fallback rate is always visible. Measured (the
    same pre-registered A/B as Phase 2, criteria
    unchanged): the fallback policy removes the first
    cut's failure point but still does not ship as a
    recall feature.
    mention_missing blocks fell
    69 → 0, the 18 endpoints without mention data were
    judged on their quotes, and judged-file candidate
    recall recovered (0.0793 → 0.0943); raw Tier-1
    precision held (0.5588) and cost held ($0.00291/doc
    vs a $0.00312 ceiling). But Tier-1 recall reached
    0.0895 against the baseline's 0.1066 (paired net
    −25): this draw's proposal channel was thinner
    (249 proposals vs the baseline draw's 327; candidate
    paired net already −23 before judging), and
    resolution judging cost 7 rejected + 4 review-capped
    of 81. The guard semantics stay (absence is not
    invalidity, and the fallback accounting stands);
    the layer remains an experimental opt-in with the
    measured trade-off. Full record: the cap-recall
    measurement (cut-2 section) and issue #55.
    Precision claim, measured and closed (issue #56).
    The raw-precision edge above (0.5588 vs 0.4912) was
    tested under a separate pre-registration with blinded
    adjudication (the flagship correction's instrument):
    adjudicated Tier-1 precision is 0.9562 with the layer
    and 0.9561 without — a gain of +0.0001 against a
    registered +0.05 bar. The raw gap was gold's error
    distribution, not the layer (~90% of both arms'
    disagreements are in fact supported). The layer does
    not ship as a precision feature either; the record is
    the cap-recall measurement (precision program
    section) and issue #56.

  • Hands-on guide restructured for a lighter first
    read.
    The appendix track (reproduce the reference
    build, steps A1–A9) moved to its own document,
    docs/hands-on-appendix.md; the front matter was
    compressed (the four tracks now open in a compact
    table, Track 1 starts at the top of the guide, and
    the profile/pack glossary moved into Track 1's
    opening); the pipeline-path install block moved to
    the end of the guide as an advanced note; and the
    example-document-sets table was slimmed and moved
    ahead of Track 2. Layout only — no step text
    changed: every command, quoted output, and
    checkpoint is the VM-verified text, relocated
    verbatim. README, llms.txt, and the benchmarks
    appendix now point at the appendix document.

Removed

  • The legacy scale1k graph writer
    (pipeline/graph_write.py) is removed.
    It wrote
    name-keyed entities with per-relation edge types
    directly from a gate-results file — the pre-product
    identity discipline the shipped store contract
    replaced (entity_id keys, one REL type, a
    batch-stamped single graph, replace-at-WRITE) — and
    derived its model/gate provenance from the results
    filename. Nothing referenced it: no import, test,
    doc, or CLI dispatch. The batch WRITE stage through
    the graph store is the only write path.
  • The GLiNER+spaCy pipeline candidate source is
    retired, and the candidate_source setting is
    removed with it; the LLM proposer is the only
    candidate source.
    The pipeline path was the
    product default but was never measured at the
    standard the LLM path carries, its structural
    (RER) pass was English-bound by construction
    (hand-written dependency patterns — each new
    language would have multiplied channels), and it
    matched extracted spans to NER mentions with its
    own order-dependent substring rule, a second
    identity discipline beside the guard funnel's (the
    same span could land on different people in
    different documents, and full names could collapse
    to a fragment listed earlier). Rather than grow a
    third implementation of the identity law for it,
    the channel is removed: pipeline/rer.py is
    deleted, the NER stage writes empty contract rows
    (its ner.jsonl consumers read tolerantly), the
    ontosmith[nlp] extra is removed from the package,
    and the --candidate-source flag and the
    candidate_source config field are gone — with
    one source there is nothing to select. A stored
    config still naming a retired value refuses by
    name at load; one carrying "llm" (or nothing)
    loads silently. All measured builds were already
    LLM-path builds; serving is unchanged.