Proposal: content_hash integrity check on the generated Hermes plugin

[Tryboy869]

Why a Discussion and not a direct PR

Per CONTRIBUTING.md, new integration formats and changes touching build
tooling go through a Discussion before code. This is exactly that case --
hence this post rather than a PR, even though the patch below is already
written and tested.

Observation

scripts/build-hermes-plugin.py generates a Hermes plugin that serves
agent["body"] as-is through agency_agents_load / agency_agents_delegate
/ agency_agents_inspect (when include_body=true). Nothing verifies that
data/agents.json still matches the source .md files by the time it's
read -- whether because it was hand-edited after generation, or shipped
from a source other than this repo.

This is the same shape as the CS10 case study in Agents of Chaos
(arXiv:2602.20021, Shapira/Bau et al.): a shared, editable context file,
once modified, has its injected instructions executed and then propagated
to other agents. data/agents.json has the same shape -- a shared, editable
file whose content is injected as instructions by several downstream tools.

Proposal

• parse_agent() computes content_hash = sha256(body) at generation time.
• The generated plugin recomputes this hash on read and refuses to serve
  body if it no longer matches (success: false, error: integrity_check_failed)
  instead of silently composing a prompt from unverified content.
• Backward compatible: an agents.json generated before this change (no
  content_hash) is treated as unverifiable rather than crashing anything.

Deliberately minimal scope

One file changed (scripts/build-hermes-plugin.py), none of the 254 agent
.md files touched, no new script, no new dependency (just hashlib,
stdlib). check-tools.sh passes unmodified.

Tested

• Normal generation: content_hash present, agency_agents_load works as
  before.
• data/agents.json edited directly (instruction injected into a body,
  hash left stale -- this is exactly the CS10 scenario): agency_agents_load
  returns integrity_check_failed, the tampered content never reaches a
  composed prompt.

Open question for maintainers

This starts with Hermes only, since it's the closest thing to a runtime
that consumes the generated JSON at a distance from this repo. Does it make
sense to generalize to claude-code/openclaw/mcp-memory afterward, or
is Hermes a special enough case not to generalize yet?

Full patch attached below.

Patch (scripts/build-hermes-plugin.py)

From 97f0dccff65c8a1c46864ccf219c0867c20e8314 Mon Sep 17 00:00:00 2001
From: "Context Manager Agentic (draft)" <placeholder@example.com>
Date: Wed, 15 Jul 2026 07:12:38 +0000
Subject: [PATCH] feat(hermes): verify agent content_hash before serving body

Adds a sha256(body) content_hash at generation time (parse_agent) and
a runtime integrity check in the generated plugin, gating the three
handlers that can return an agent's full body: inspect (include_body),
load, and delegate. If data/agents.json is edited outside this script
(or shipped from an untrusted source) and a body no longer matches its
recorded hash, the handler returns success:false with
error=integrity_check_failed instead of composing a prompt from
unverified content.

Motivated by Agents of Chaos (arXiv:2602.20021) case study CS10: a
shared, editable context file with injected instructions was executed
and then propagated to other agents. The generated plugin currently
has no way to notice data/agents.json changed underneath it between
generation and use.
---
 scripts/build-hermes-plugin.py | 44 ++++++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)

diff --git a/scripts/build-hermes-plugin.py b/scripts/build-hermes-plugin.py
index 14e4bc4..b655a95 100644
--- a/scripts/build-hermes-plugin.py
+++ b/scripts/build-hermes-plugin.py
@@ -9,6 +9,7 @@ skill catalog.
 from __future__ import annotations
 
 import argparse
+import hashlib
 import json
 import re
 import shutil
@@ -54,6 +55,11 @@ def parse_agent(path: Path, repo_root: Path) -> dict[str, str] | None:
         return None
     rel = path.relative_to(repo_root)
     division = rel.parts[0]
+    # content_hash lets the runtime plugin detect a body that changed after
+    # generation (e.g. data/agents.json edited directly, bypassing this
+    # script). It intentionally covers only `body` — the text actually
+    # injected into a caller's prompt — not the full frontmatter, so cosmetic
+    # metadata edits don't require regeneration to stay verifiable.
     return {
         "slug": slugify(name),
         "name": name,
@@ -64,6 +70,7 @@ def parse_agent(path: Path, repo_root: Path) -> dict[str, str] | None:
         "vibe": fields.get("vibe", "").strip(),
         "source_path": str(rel),
         "body": body,
+        "content_hash": hashlib.sha256(body.encode("utf-8")).hexdigest(),
     }
 
 
@@ -110,6 +117,7 @@ def init_py() -> str:
     return r'''"""Hermes plugin: lazy router for The Agency agents."""
 from __future__ import annotations
 
+import hashlib
 import json
 import math
 import re
@@ -129,6 +137,22 @@ def _load_agents() -> list[dict[str, Any]]:
     return _AGENTS
 
 
+def _integrity_ok(agent: dict[str, Any]) -> bool:
+    """True if agent['body'] still matches the hash recorded at build time.
+
+    Catches data/agents.json being hand-edited after generation (or shipped
+    from an untrusted source) — this is the only gate between on-disk agent
+    content and it being served as instructions to a caller. A missing
+    content_hash (plugin built before this field existed) is treated as
+    unverifiable rather than trusted.
+    """
+    expected = agent.get("content_hash")
+    if not expected:
+        return False
+    actual = hashlib.sha256(agent.get("body", "").encode("utf-8")).hexdigest()
+    return actual == expected
+
+
 def _tokens(text: str) -> set[str]:
     return {token.lower() for token in _WORD_RE.findall(text or "")}
 
@@ -159,6 +183,20 @@ def _not_found(identifier: str) -> dict[str, Any]:
     }
 
 
+def _integrity_failed(agent: dict[str, Any]) -> dict[str, Any]:
+    return {
+        "success": False,
+        "error": "integrity_check_failed",
+        "agent": agent.get("slug"),
+        "detail": (
+            "This agent's body no longer matches its recorded content_hash. "
+            "data/agents.json may have been edited outside "
+            "scripts/build-hermes-plugin.py. Regenerate the plugin from "
+            "source and re-install before using this agent."
+        ),
+    }
+
+
 def _score(agent: dict[str, Any], query_tokens: set[str], query_text: str) -> float:
     haystack_fields = [
         agent.get("name", ""),
@@ -319,6 +357,8 @@ def register(ctx):
             return _json(_not_found(identifier))
         payload = {"success": True, "agent": _summary(agent)}
         if bool(args.get("include_body", False)):
+            if not _integrity_ok(agent):
+                return _json(_integrity_failed(agent))
             payload["body"] = agent.get("body", "")
         return _json(payload)
 
@@ -328,6 +368,8 @@ def register(ctx):
         agent = _agent_lookup(identifier)
         if not agent:
             return _json(_not_found(identifier))
+        if not _integrity_ok(agent):
+            return _json(_integrity_failed(agent))
         return _json({
             "success": True,
             "agent": _summary(agent),
@@ -341,6 +383,8 @@ def register(ctx):
         task = str(args.get("task", "")).strip()
         if not agent:
             return _json(_not_found(identifier))
+        if not _integrity_ok(agent):
+            return _json(_integrity_failed(agent))
         if not task:
             return _json({"success": False, "error": "task is required"})
         composed = _specialist_prompt(agent, task)
-- 
2.43.0