Encrypted / encoded (opaque) message bodies with transparent metadata

[zircote]

ADR Precursor — Structured MADR seed. Recast of this ideation seed into Structured MADR shape. Still an ideation seed / details plan — not a spec proposal, deliberately undecided (status: proposed, Decision: PENDING). Posted to gather design input before anything is written into SPECIFICATION.md or schema/mif.schema.json. When it matures, it drops into adr/ADR-NNN-...md (next free number) with the frontmatter below.

title: "Opaque (Encoded/Encrypted) Message Bodies with Transparent Metadata"
description: "Whether to add an OPTIONAL body-envelope so a MIF record can carry a compressed/encoded/encrypted body while metadata stays transparent — likely scoped to the JSON-LD projection, not canonical Markdown."
type: adr
category: data
tags:
  - body-envelope
  - encryption
  - encoding
  - json-ld
  - round-trip
  - level-3
status: proposed
created: 2026-06-28
updated: 2026-06-28
author: MIF Maintainers
project: MIF
technologies:
  - json-ld
  - jwe
  - age
  - cose
audience:
  - developers
  - architects
related:
  - ADR-011-markdown-canonical-derived-jsonld.md
  - ADR-014-document-reference-not-embed.md
  - ADR-013-provenance-lightweight-core-optional-prov-layer.md

ADR Precursor: Encrypted / Encoded (Opaque) Message Bodies with Transparent Metadata

Status

Proposed — precursor / undecided. No option is selected. Seed for design input.

Context

Background and Problem Statement

Today a MIF record carries its body as cleartext canonical Markdown in content. There is no standard way to ship a body that is compressed (large prose / transcripts), encoded (binary-safe transport), or encrypted (confidential payload) while keeping the surrounding metadata — conceptType, namespace, relationships, provenance, temporal — fully transparent and machine-legible.

The proposal: a body-envelope where the metadata stays transparent but the body becomes opaque (compressed and/or encrypted), self-described by metadata the record carries — mimeType + encoding chain + a cipher/key descriptor (public-key fingerprint, token, or JWT/JWK reference — never the secret itself).

Sketch of the envelope (Level 3, all OPTIONAL — default stays today's plaintext)

Field	Purpose
mimeType / mediaType	What the decoded body is (text/markdown, application/json, …)
contentEncoding	Transfer/compression chain: identity, base64, zlib, gzip, base64+zlib
contentCipher	Encryption descriptor: alg (age, jwe, AES-256-GCM) + key reference (pubkey fingerprint, JWK URL, JWT/token claim) — not the secret
contentHash	Integrity over the decoded (or encoded) bytes — reuse the DocumentReference.hash {algorithm, value} shape
contentLength / compressedAt	Sizing + reuse of the existing §5.6 timestamp

Framing Axis (the crux)

This likely applies solely to the JSON-LD projection — the derived/interchange document — not the canonical Markdown source.

MIF holds Markdown as canonical (SPECIFICATION.md §6, Invariant 2; the JSON-LD form "MUST round-trip losslessly back to markdown" — see ADR-011). An opaque body would break that for cleartext records — no grep, no OKF *.md ingestion, no embeddings over the prose. Scoping the envelope to the JSON-LD projection sidesteps that: the human-authored Markdown stays plaintext-canonical, and the encoded/encrypted body is a property of the JSON-LD document that gets converted into / regenerated from Markdown on the trusted side. The source-of-truth question for the encrypted case — does JSON-LD become the carrier of record when the Markdown can't be materialized? — is the first thing to nail down.

Decision Drivers

Primary Decision Drivers

1. Preserve Invariant 2 (Markdown-canonical, lossless round-trip) while still enabling opaque bodies. Scoping to the JSON-LD projection is the proposed mitigation and must be made airtight.
2. Confidentiality / transport without leaking metadata — the envelope must keep conceptType/namespace/relationships/provenance/temporal transparent even when the body is opaque.
3. Durable decryptability — a static record can only describe the key; the descriptor must carry enough to stay decryptable later.

Secondary Decision Drivers

1. Reuse over invention — prefer existing primitives (DocumentReference.hash, an existing cipher standard) to a MIF-native descriptor.
2. Avoid primitive confusion with existing Compression (§5.6) and with DocumentReference.

Scope

In scope: the metadata envelope (mimeType + encoding + cipher descriptors), how it rides on the JSON-LD content, conformance level, and the round-trip / source-of-truth story.

Out of scope (for this seed): mandating specific ciphers or a KMS, key distribution/rotation, and any decryption tooling in mif_convert.

Considered Options

Option 1: JSON-LD-projection-scoped body envelope

Description: Add the OPTIONAL sibling fields above to content only in the JSON-LD projection; Markdown stays plaintext-canonical. For the encrypted case, define whether JSON-LD becomes the source-of-truth carrier when Markdown cannot be materialized.

Advantages: keeps Invariant 2 intact for cleartext; metadata stays transparent; additive/optional (default unchanged).
Disadvantages: bifurcates the source-of-truth story for the encrypted case; new fields to validate; round-trip semantics for opaque bodies need precise definition.
Risk: Technical: Medium–High (Invariant 2 interplay). Schedule: Medium. Ecosystem: Low.

Option 2: Push opaque/large bodies out via DocumentReference (reuse, don't add inline)

Description: Instead of an inline opaque content, carry the payload by reference using the existing DocumentReference (pointer + integrity metadata) per ADR-014 ("reference, not embed").

Advantages: reuses an accepted primitive and its hash shape; keeps the record body small; aligns with ADR-014.
Disadvantages: indirection for what may be small inline payloads; doesn't cover the "opaque but still inline" use case; resolver/availability concerns.
Risk: Technical: Low–Medium. Schedule: Low. Ecosystem: Low.

Option 3: Adopt an existing cipher/encoding standard wholesale (JWE / age / COSE/RFC 9165)

Description: Rather than a MIF-native descriptor, the contentCipher is a JWE/age/COSE structure; MIF only references it.

Advantages: battle-tested crypto + key-reference semantics; least invention; interop with existing tooling.
Disadvantages: heavier dependency/conformance surface; couples MIF to an external format's evolution.
Risk: Technical: Medium. Schedule: Medium. Ecosystem: Medium.

Option 4 (exculpatory — do nothing): Keep cleartext-only bodies

Description: Make no schema change. Confidentiality/compression handled out-of-band (encrypted-at-rest storage, encrypted transport, or pre-encrypting and pushing to a DocumentReference without new fields).

Advantages: zero risk to Invariant 2, grep, OKF *.md ingestion, and embeddings; no new validation surface; no key-durability burden on the record; smallest spec.
Disadvantages: no standard way to carry an opaque body with transparent metadata inline — the original gap remains; each consumer reinvents it.
Risk: Technical: Low. Schedule: Low. Ecosystem: Low.

Integration Points

• schema/mif.schema.json — content plus the new sibling fields.
• DocumentReference.hash (mif.schema.json:152) — reuse pattern for integrity; also clarify body-inline-opaque vs push-to-DocumentReference.
• Compression §5.6 — must be disambiguated: existing compression is lossy summarization (summary + compressed_at, content replaced by ≤500-char summary); this is lossless encode/encrypt. Two different primitives that must not be conflated.
• The JSON-LD round-trip invariant (SPECIFICATION.md §6; ADR-011).
• scripts/validate-memories.py — the frontmatter validation gate.

Trade-offs / Risks

1. Invariant 2 conflict — "Markdown is canonical / lossless round-trip" assumes a readable body. Scoping to the JSON-LD projection is the proposed mitigation; needs to be made airtight.
2. Static record + encryption = key management lives elsewhere. The record can only describe the key. Durability risk: "can't decrypt this years later." What is the minimum the descriptor must carry to stay decryptable?
3. Overlap/confusion with existing Compression (§5.6) and with DocumentReference — when is an inline opaque body correct vs. pushing the payload out by reference?

Decision

PENDING. Deliberately undecided — this seed records the option space and open questions; default behavior stays today's plaintext.

Open Questions

• Inline opaque body, or always push encrypted/large bodies to a DocumentReference?
• Should an encoded/encrypted body force the JSON-LD form (since Markdown-canonical can't hold it)? Does JSON-LD become source-of-truth in that case?
• Reuse an existing standard (JWE, age, COSE/RFC 9165) for the cipher descriptor, or define a minimal MIF-native descriptor?
• New conformance sub-level, or extend Level 3?
• Does contentHash cover the plaintext, the ciphertext, or both?

Links

• SPECIFICATION.md §6 (round-trip), §5.6 (Compression) · ADR-011 (Markdown-canonical) · ADR-014 (DocumentReference) · ADR-013 (provenance layer)
• schema/mif.schema.json · scripts/validate-memories.py

[zircote]

🙄