Skip to content

Proposal: evidence-trail convention and anti-pattern concept type for safety-critical bundles #204

Description

@kaeferboeck

Summary

Two additive frontmatter conventions for domains where a wrong answer breaks machines rather than conversations (industrial automation, robotics, medical tooling):

  1. an evidence trail documenting how a concept was verified, and
  2. anti-patterns (negative knowledge) as a first-class concept type.

Both use only §4.1's arbitrary-key allowance — no spec change required, no impact on consumers that ignore the keys. What I'm asking: does this fit as a documented convention (or sample bundle), so producers in safety-critical domains converge on the same keys instead of inventing private dialects?

Problem

OKF standardizes how agents read curated knowledge, and deliberately says nothing about whether it is true. In safety-critical niche domains that gap is the failure mode: LLMs interpolate from mainstream languages/APIs into proprietary ones, and an agent that confidently repeats an unverified claim generates production-breaking code. Two things kept this in check in our production use (details below): every verified claim carries its evidence, and disproven assumptions are knowledge too.

Proposal 1: evidence trail

status: verified          # draft | verified | disputed | deprecated
evidence:
  - "wait_t(500) blocked the cell ~8.3 min instead of 0.5 s (reproduced 2026-07-16, v2.4 simulator)"
  - "language reference §4.2: 'duration: FLOAT, unit seconds'"
verified_by: kk
last_verified: 2026-07-16
support_status: unsupported   # for capability claims: supported | unsupported | partial

Contract: status: verified without evidence + verified_by + last_verified fails CI ("no evidence, no merge"). support_status: unsupported deserves emphasis: proven-absent features are the single most valuable entries, because they are exactly what models hallucinate.

Proposal 2: type: anti-pattern

A concept documenting a plausible but false assumption, with three mandatory sections: Misconception (what a reasonable person or LLM assumes) → Observed failure (with evidence) → Correct alternative (linked concept). Since type values are not centrally registered, this costs the spec nothing — but naming the convention in docs or a sample would let consumers (linters, routing agents) treat these as hard generation constraints.

Relationship to existing threads

Origin & offer

The convention comes from a private production KB for a proprietary industrial robot-programming language (2025–2026, predates OKF; converged on the markdown-plus-frontmatter pattern independently). It measurably stopped a class of agent errors: invented functions, wrong unit conventions, alarm-text-taken-literally diagnoses. The employer-internal KB can't be published, so the method is demonstrated on a fictional language in a small OKF bundle with a CI validator: https://github.com/kaeferboeck/okf-kb-framework

If there's interest, I'd submit a sample bundle PR (like #185) — "safety-critical industrial automation" — demonstrating both conventions end-to-end.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions