Seeking feedback: information interop schema

[Jeffallan]

Soliciting feedback on a draft schema that defines the universal frontmatter / top-level JSON convention for all artifacts produced by and ingested into the claude-skills plugin. This schema is load-bearing for v0.5.0+ — adapter authors, memory system consumers, and Workflow Profile evidence artifacts all conform to it.

Why this matters

Every artifact the plugin produces or ingests — backend adapter output (Jira tickets, GitHub issues, Notion pages, local markdown), Workflow Profile evidence (review verdicts, merge decisions), memory system nodes (dialectical, procedural, episodic), article production artifacts, external source adapters (Slack, email, PDF) — carries the same universal fields. One contract, two serializations (YAML frontmatter for markdown files, top-level JSON for structured artifacts).

The bet: if the schema is right, adapter authors can write against a stable contract. Memory retrieval, audit walks, GRC tooling, and cross-system interop all traverse schema-conformant content with one parser. If the schema is wrong, every consumer rewrites their own.

What I'm asking for

Most valuable feedback is from anyone who would (or has) authored an adapter against an external system. Specifically:

1. Missing fields. What external-system metadata does your adapter need to preserve that the current schema doesn't capture?
2. Friction points. Is there a field that will be hard to populate from real-world external sources?
3. Type discriminator vocabulary. Does the type / subtype list match what you'd produce? Anything missing or over-specified?
4. Ingestion extension (original_metadata). Per-subtype sub-schema (I recommend this) vs. free-form object?
5. Confidence derivation. Currently arbitrary 0.0–1.0. How would you calibrate it for your source?
6. The two serializations (YAML frontmatter + top-level JSON) question. Any tooling concerns?

What's out of scope

• Storage layout decisions (directory structure, vault federation)
• Retrieval library choices (engraph, claude-mem, etc.)

Draft schema

---

Base Schema: Universal Frontmatter Convention

Status: Draft
Created: 2026-04-21
Author: Jeff Smolinski

---

Purpose

A universal frontmatter convention that every artifact in the claude-skills ecosystem conforms to. One contract, two serializations (YAML frontmatter for markdown files, top-level JSON for structured artifacts), many consumers.

Why this is load-bearing:

• Ingestion: external sources (Notion, Jira, Confluence, GitHub, Slack, email, PDFs) are normalized to the base schema by adapters. No adapter = no ingestion. No schema = no adapters.
• Interop: memory retrieval, audit walks, compliance tooling, and external integrations all traverse schema-conformant content. One parser serves all.
• Decoupling: consumers (memory system, Workflow Profile, article cycle) don't depend on each other — they depend on the schema. Substrate changes without rewrites.

Why this is a v0.5.0 prerequisite:
Layer 1 adapters (Chunk 1, #119) read/write tickets and docs to external systems. Those reads and writes need a target/source format. The base schema is that format. Chunk 1 cannot meaningfully land without the schema being defined first.

---

Architecture

┌──────────────────────────────────────────────────────┐
│  CONSUMERS                                           │
│  • Workflow Profile artifacts (v0.6.0+)              │
│  • Adversarial review verdicts (v0.7.0+)             │
│  • Memory system (v0.9.0+)                           │
│  • Article production cycle                          │
│  • Future: ADR tooling, RFC lifecycle, spec tooling  │
├──────────────────────────────────────────────────────┤
│  ADAPTERS (normalize external ↔ schema)              │
│  • L1 ticketing adapters (Jira, GitHub, local)       │
│  • L1 docs adapters (Confluence, Notion, local)      │
│  • Memory ingestion adapters (email, Slack, PDF)     │
├──────────────────────────────────────────────────────┤
│  BASE SCHEMA (this document)                         │
└──────────────────────────────────────────────────────┘

---

Universal Fields (required in every artifact)

Every schema-conformant artifact MUST carry these fields, regardless of type or serialization.

Field	Type	Description
id	string	Globally unique identifier. Slug or hash. Stable across edits.
type	enum	Top-level type discriminator (see Type Discriminators below).
schemaVersion	string	Semver-like version of this schema the artifact was authored against. Starts at "1".
created	ISO-8601 datetime	When the artifact was first produced. Immutable after creation.
updated	ISO-8601 datetime	Last modification timestamp. Bumped on every edit.
source	array	Provenance — where the content came from (see Provenance below).
sensitivity	enum	Access-control tier (see Sensitivity below).

A file missing any of these is non-conformant. Consumers MAY reject non-conformant input.

---

Strongly Recommended Fields

Most artifacts should carry these. Consumers rely on them for retrieval, indexing, and promotion.

Field	Type	Description
subtype	string	Secondary discriminator within the top-level type (see extensions).
ttl	ISO-8601 date or "never"	Staleness horizon. After this date, content should be re-validated.
cache_key	string	Hash over relevant source refs. Invalidation signal for derived content.
tags	array of strings	Unstructured classification. Used for subgraph scoping and search.
project	string or null	Project slug if scoped; null for general-vault / cross-project content.
workstream	string or null	Sub-project clustering (e.g., a specific engagement, sprint, or epic).
session	string or null	Session identifier if the artifact was produced in an agent session.
confidence	number 0.0–1.0	For artifacts with an epistemic status (dialectical, semantic). See Open Questions for derivation rules.
style	atomic or log	Indexing hint. atomic = one subject per file; log = chronological write-append.
promoted_to_general	boolean	Whether this project-scoped artifact has been promoted to the general vault.

---

Type Discriminators

Top-level type values and their defined subtypes:

type	Description	Subtypes
dialectical	Decision node with thesis/antithesis/synthesis	node (default), decision, architecture
procedural	Patterns, rules, operational instructions	runbook, playbook, process-notes, rule
semantic	Compiled knowledge: facts, concepts, entities	definition, concept, entity
episodic	Timestamped record of what happened	session-log, event, observation
article-artifact	Writing-cycle production artifact	raw-material, whirlybird, outline, draft, judge-report, fool-report, final-draft
workflow-evidence	Workflow Profile artifact produced at a phase boundary	impl-plan, review-verdict, qa-verdict, merge-decision, compliance-artifact, test-results
feature	Feature specification (from feature-forge)	spec, requirements
decision	Architecture Decision Record	adr, decision-record
topic	Reusable concept node for injection into project vaults	stub, canonical
ingested	Artifact ingested from an external source	notion-page, confluence-page, jira-ticket, github-issue, github-pr, slack-message, email, pdf, url

New types may be added over time. Consumers MUST ignore unknown type values rather than erroring.

---

Type-Specific Extensions

Fields below are appended to the universal set when type matches.

dialectical

thesis: slug-or-identifier
thesis_statement: "Prose form of the thesis"
thesis_src: path/to/source/or/session-ref
antithesis: slug-or-identifier
antithesis_statement: "Prose form of the antithesis"
antithesis_src: path/to/source/or/session-ref
synthesis: "Prose form of the resolution"
defeated_position: "What was rejected and why"
evidence: path/to/supporting/evidence.md
resolved: 2026-04-21          # Date the synthesis was reached
edge_type: resolved_via_synthesis | superseded | in_tension

procedural

Base procedural extension:

applies_to: [discovery, planning, execution]   # Phases/contexts where this applies
supersedes: null | <id>                         # If this replaces an earlier version
review_cadence: per-epic | per-sprint | annual | ad-hoc

Subtype process-notes (chronological workstream log):

style: log                                      # Always log for process-notes
state_marker: "2026-04-21 ~14:30"               # Where next session resumes
promotion_candidates:                            # Observations flagged for distillation
  - id: sandbox-boot-pattern
    kind: runbook-stub
  - id: scp-over-base64-pattern
    kind: runbook-stub
  - id: alliteration-rule
    kind: claude-md-patch

Subtype runbook:

trigger: "When X happens"                        # Entry condition
steps_ref: "#steps"                              # Points to body section with steps
rollback_ref: "#rollback"                        # Points to rollback section

Subtype playbook:

scenario: "Situation this addresses"
pattern: "High-level strategy name"
inversions: ["what-not-to-do-1", "what-not-to-do-2"]

article-artifact

version: 4                                       # Numeric revision within the cycle
parent: outline-04.md                            # Immediate ancestor
derived-from:                                    # All antecedents contributing content
  - raw-material.md
  - whirlybird-c-stop-outsourcing-your-brain.md
key-decisions:                                   # Decisions shaping this artifact
  - "Whirlybird C selected"
  - "B+C throughline blend"
  - "Voice shifted to we"
status: in-progress | preserved | superseded | final

workflow-evidence

ticket: TICKET-123                               # Associated ticket ID
phase: review | qa | merge                       # Phase producing this artifact
verdict: blocking | warning | advisory | pass    # For review/qa/merge verdicts
findings: []                                     # Type-specific array of findings

ingested

origin_url: https://notion.so/...                # Or non-URL ref if no URL exists
origin_format: notion-page                       # Matches one of the ingested subtypes
original_metadata:                               # Preserves external structured data verbatim
  notion_page_id: abc123
  notion_last_edited: 2026-04-20T10:00:00Z
  notion_author: user@example.com
ingestion_date: 2026-04-21
ingestion_adapter: notion-adapter@1.2.0          # Which adapter produced this artifact

---

Sensitivity Levels

Level	Meaning	Promotion allowed?
public	No restrictions. Can be published anywhere.	Yes
internal	Inside the organization only.	Yes
client-confidential	Bound to a specific client engagement.	No — must be sanitized first
restricted	Legally/contractually restricted (PII, PHI, regulated).	No — never promotes

Promotion rule: artifacts with sensitivity: client-confidential or restricted MUST be sanitized (engagement-specific IDs stripped, client identifiers removed, roles substituted for specifics) before any content is promoted to a general vault. The /promote command is responsible for enforcing this rule.

Sanitization output: a new artifact with sensitivity: internal (or public if further cleared), referencing the original only by workstream and date — never by client name or asset ID.

---

Provenance: the source Field

Every artifact records its origins. source is an array (artifacts can have multiple upstream sources).

source:
  - ref: raw/specs/realtime-sync.md              # Local markdown reference
    type: local-markdown
  - ref: https://notion.so/workspace/page-id     # External URL
    type: notion-page
  - ref: JIRA-1234                                # Ticket reference
    type: jira-ticket
  - ref: session:2026-04-21-session-42            # Agent session
    type: session
  - ref: tool:the-fool                            # Skill/tool output
    type: skill-output

Each source entry carries:

• ref — string identifier (path, URL, ticket key, session ID, tool name)
• type — matches the ingested subtype vocabulary or one of local-markdown, session, skill-output, derived

Derived content (e.g., a runbook distilled from process notes) records its upstream process-notes files as sources.

---

Cache / Staleness

Two orthogonal mechanisms:

TTL (absolute expiration): ttl: 2026-10-21 means the artifact should be re-validated after that date. ttl: never means no automatic expiration.

Cache key (content-derived invalidation): cache_key is a hash (typically SHA-256) of the relevant source refs. When any source changes, consumers recompute the expected cache_key; mismatch flags the artifact as potentially stale regardless of TTL.

Typical usage:

• Raw ingested content: ttl: 2026-10-21, cache_key derived from external version/etag
• Dialectical nodes: ttl: never (resolved decisions persist), cache_key derived from thesis/antithesis sources
• Derived specs / runbooks: ttl: per-epic-cadence, cache_key derived from upstream dialectical and process-note refs
• Process notes: ttl: never, cache_key bumped on each entry

Stale-content handling is the consumer's responsibility. A retrieval layer may de-rank stale artifacts; a spec pipeline may refuse to use them; an audit walk may surface them for review.

---

Two Serializations

YAML Frontmatter (markdown files)

Standard --- fenced YAML at the top of .md files:

---
id: dialectical-websockets-for-realtime-sync
type: dialectical
subtype: node
schemaVersion: "1"
created: 2026-04-15
updated: 2026-04-15
sensitivity: internal
tags: [realtime, transport, architecture]
project: acme-payments
workstream: acme-payments-api
session: 2026-04-15-session-42
confidence: 0.92
style: atomic
promoted_to_general: false
ttl: never
cache_key: sha256:abc...
source:
  - ref: raw/specs/realtime-sync.md
    type: local-markdown
thesis: websockets-for-realtime-sync
thesis_statement: "WebSockets are the right transport for real-time sync"
# ... full dialectical extension ...
---

# Dialectical Node: WebSockets for Real-Time Sync

(body content goes here)

Top-Level JSON (structured artifacts)

Universal and type-specific fields live as top-level keys in the JSON:

{
  "id": "TICKET-123-review-verdict",
  "type": "workflow-evidence",
  "subtype": "review-verdict",
  "schemaVersion": "1",
  "created": "2026-04-21T14:00:00Z",
  "updated": "2026-04-21T14:00:00Z",
  "sensitivity": "internal",
  "tags": ["review", "TICKET-123"],
  "project": "acme-payments",
  "workstream": "acme-payments-api",
  "session": "2026-04-21-session-42",
  "ttl": "never",
  "cache_key": "sha256:def...",
  "source": [
    {"ref": "git-diff:TICKET-123", "type": "derived"},
    {"ref": "test-results.json", "type": "derived"}
  ],
  "ticket": "TICKET-123",
  "phase": "review",
  "verdict": "blocking",
  "findings": [
    {"severity": "blocking", "category": "security", "description": "..."}
  ]
}

---

Validation and Conformance

A JSON Schema describing this base schema and its extensions SHALL be maintained alongside this document.

Adapter obligations:

• Adapters MUST emit conformant artifacts. Validation is part of the adapter contract.
• Adapters MUST fail loudly if they cannot produce a conformant artifact from an input.
• Adapters SHOULD preserve external structured data in original_metadata for round-trip fidelity.

Consumer permissions:

• Consumers MAY rely on conformance — no need to handle missing universal fields.
• Consumers SHOULD ignore unknown type-specific fields (forward compatibility).
• Consumers MUST ignore unknown type values rather than erroring.

---

Schema Versioning

• schemaVersion: "1" is the initial release.
• Breaking changes bump the version. Artifacts retain their original schemaVersion.
• Consumers SHOULD be version-aware: either handle multiple versions explicitly or refuse to consume unsupported versions.
• Migrations between versions are scripted; a migration tool walks the vault, rewrites frontmatter/JSON, and bumps schemaVersion + updated.

---

Open Questions

1. Confidence score derivation. The confidence: 0.0–1.0 field is currently arbitrary. Before any promotion logic relies on it, a calibration rule is needed. Candidates: evidence-weighted score from the-fool's evidence audit; agent-reviewer consensus; elapsed time since last challenge; human calibration rating. Unscoped.

2. Promotion threshold(s). promoted_to_general: true gating uses a confidence threshold (0.85 suggested). Needs calibration alongside Add GraphQL Expert skill #1.

3. original_metadata schema per ingested subtype. Should each subtype (notion-page, jira-ticket, etc.) have its own sub-schema for original_metadata, or is it a free-form object? Recommend per-subtype sub-schemas so adapters can validate their output and consumers can query known fields.

4. Non-textual content. Images, PDFs, binary attachments — do they live alongside the markdown (as referenced files) or are they represented as schema artifacts with type: ingested? Recommend the former for most cases; schema only when the binary IS the artifact.

5. External adapter registry. How do third-party adapters declare themselves to the system? A manifest file? A convention-based directory scan? TBD.

6. ID stability and migration. id is supposed to be stable across edits, but what happens when a vault is renamed or a project is merged with another? Recommend: IDs are stable within a vault scope; merge operations generate redirect records. Unscoped for v1.

---

How to respond

• Field-level feedback: quote the field + line and propose a change
• Structural feedback: top-level comment
• "I'll write an adapter against this": comment saying which external system and I'll circle back for a test integration

Thanks for reading. This is a sketch — pushback is the point.