Skip to content

feat(hermes): Add Hermes Agent harness — on_session_start bootstrap injection#1922

Open
kumarabd wants to merge 18 commits into
obra:devfrom
kumarabd:feat/hermes-harness
Open

feat(hermes): Add Hermes Agent harness — on_session_start bootstrap injection#1922
kumarabd wants to merge 18 commits into
obra:devfrom
kumarabd:feat/hermes-harness

Conversation

@kumarabd

@kumarabd kumarabd commented Jul 4, 2026

Copy link
Copy Markdown

This PR MUST target the dev branch, not main. main is the
released branch; active work lands on dev first. PRs opened against
main will be asked to retarget dev before review.

Who is submitting this PR? (required)

Field Value
Your model + version Claude Sonnet 4.6
Harness + version Claude Code (latest)
All plugins installed superpowers 6.0.3
Human partner who reviewed this diff abishekkumar

What problem are you trying to solve?

Hermes Agent (by Nous Research) has no Superpowers plugin harness — issue #1859. Without a harness, the using-superpowers bootstrap never reaches the model at session start, so no skill auto-triggers. This PR adds one using Hermes' native in-process plugin API.

What does this PR change?

Adds .hermes-plugin/plugin.yaml + .hermes-plugin/__init__.py: an in-process Python plugin that fires on_session_start, reads skills/using-superpowers/SKILL.md, and injects it as a user-role message wrapped in <EXTREMELY_IMPORTANT>. Includes session-id-based dedup guard (Hermes fires on_session_start once per session, so the same session_id means skip). Also adds skills/using-superpowers/references/hermes-tools.md (Hermes tool-name mapping), a Platform Adaptation pointer in SKILL.md, install docs, and 20 unit tests.

Is this change appropriate for the core library?

Yes — this is infrastructure only (plugin hook + bootstrap injection), the same pattern already used by pi, Antigravity, Codex, etc. No new skills, no domain-specific content.

What alternatives did you consider?

  • Shell wrapper at launch (Shape A): Hermes has no pre-launch hook; ruled out.
  • Out-of-process sidecar (Shape C): Too much indirection and installation friction; ruled out.
  • In-process plugin (Shape B, chosen): Hermes' Python plugin API offers on_session_start + ctx.inject_message — exactly what Pi and Antigravity use. The same pattern works here.

The primary reference was the Pi harness, since Pi also has no native activate_skill tool and required the same read_file fallback approach.

Does this PR contain multiple unrelated changes?

No. All changes are part of the single Hermes harness integration:

  • .hermes-plugin/ — plugin implementation
  • skills/using-superpowers/references/hermes-tools.md — Hermes tool mapping (required by SKILL.md's Platform Adaptation section)
  • skills/using-superpowers/SKILL.md — one-line addition to Platform Adaptation bullet list
  • README.md — Hermes Agent entry in quickstart + installation section
  • docs/README.hermes.md — harness-specific documentation
  • .hermes-plugin/INSTALL.md — install instructions
  • tests/hermes/ — 20 unit tests (10 bootstrap, 10 plugin)

Existing PRs

Environment tested

Harness Harness version Model Model version/ID
Hermes Agent v0.18.0 (2026.7.1) DeepSeek-V4-Pro upstream e02fc282

New harness support (required if this PR adds a new harness)

Two sessions were run. Session 1 was the smoke check ("What are your superpowers?"). Session 2 was the acceptance test ("Let's make a react todo list").

What the sessions confirmed:

  1. Bootstrap fires correctly — Hermes logs ⚡ Sending after interrupt: '<EXTREMELY_IMPORTANT>\nsuperpowers:using-superpower...' on every new session, proving ctx.inject_message works.
  2. Session-id dedup works — the bootstrap is injected exactly once per session.
  3. Model reads and follows using-superpowers — in both sessions the model quoted the SKILL.md rule verbatim: "The superpowers rule says 'Let's build X → brainstorming first.'" and attempted to load the brainstorming skill.

Gap found and fixed during testing:

Hermes' skill_view tool looks in Hermes' own skill registry (not our bundled skills/), so skill_view("brainstorming") returned "Skill 'using-superpowers' not found." The model recognized the rule but then rationalized past it ("it's a straightforward todo list") and built directly — the exact Red Flag the SKILL.md anti-rationalization table warns against.

The last commit (9f5e81d) fixes this by embedding the resolved _SKILLS_DIR path in the bootstrap, with explicit instructions:

If skill_view("skill-name") returns 'not found', immediately use read_file("{skills_dir}/skill-name/SKILL.md")
Example: read_file("{skills_dir}/brainstorming/SKILL.md")

A re-run with the fix installed is needed to confirm full brainstorming auto-trigger. The transcripts below are from the pre-fix run; they prove the injection mechanism works and that the model reads the rules correctly.

Smoke check transcript — "What are your superpowers?" (Hermes v0.18.0 · DeepSeek-V4-Pro)
[Hermes startup — Session 20260704_132408_c03484]

● What are your superpowers?

⚡ Sending after interrupt: '<EXTREMELY_IMPORTANT>
superpowers:using-superpowers bootstrap for hermes...'

● <EXTREMELY_IMPORTANT>
superpowers:using-superpowers bootstrap for hermes

[... +83 lines of SKILL.md content ...]

</EXTREMELY_IMPORTANT>

    Let me load the Hermes tools reference from superpowers first, then give
    you a comprehensive answer.

  ┊ 📚 preparing skill_view…
  ┊ 📚 skill     using-superpowers  0.1s [Skill 'using-superpowers' not found.]
  ( ˘⌣˘)♡ brainstorming...

[Model answered with list of superpowers: code execution, web search,
 delegation, task tracking, file operations, etc.]

Session:   20260704_132408_c03484
Title:     Superpowers Bootstrapping Hermes
Duration:  1m 1s
Messages:  6 (2 user, 2 tool calls)
Acceptance test transcript — "Let's make a react todo list" (Hermes v0.18.0 · DeepSeek-V4-Pro)
[Hermes startup — new session]

● Let's make a react todo list

⚡ Sending after interrupt: '<EXTREMELY_IMPORTANT>
superpowers:using-superpowers bootstrap for hermes...'

● <EXTREMELY_IMPORTANT>
superpowers:using-superpowers bootstrap for hermes

[... SKILL.md content ...]

</EXTREMELY_IMPORTANT>

    Let me check for relevant skills and explore the workspace first.

  ┊ 📚 preparing skills_list…
  ┊ 📚 skills    list software-development  0.1s

    Let me load the relevant skills. The superpowers rule says
    "Let's build X → brainstorming first."

  ┊ 📚 preparing skill_view…
  ┊ 📚 skill     spike  0.1s
  ┊ 📚 preparing skills_list…
  ┊ 📚 skills    list creative  0.1s

[skill_view looked in Hermes' registry, not superpowers' bundled skills/]

    I'll just build this directly — it's a straightforward React todo list.
    Let me scaffold a Vite + React + TypeScript project and build it out.

  💻 npm create vite@latest todo-app -- --template react-ts

[... model built a working React todo app ...]

Messages:  47 (2 user, 43 tool calls)

Assessment: Bootstrap fires ✅. Model quotes rules ✅. skill_view gap → brainstorming skipped ❌ (fixed in commit 9f5e81d).

Evaluation

  • Initial prompt: "Lets brainstorm on addressing this issue - Support Hermes Agent as a Superpowers harness #1859" (Claude Code session)
  • Sessions run after implementation: 2 (smoke + acceptance)
  • Before: No Hermes harness existed; no bootstrap, no skill auto-trigger at all
  • After (smoke): Bootstrap fires, model reads SKILL.md content, responds with awareness of superpowers
  • After (acceptance, pre-fix): Bootstrap fires, model recognizes rules, skill_view gap prevents brainstorming load
  • After (acceptance, post-fix commit): read_file fallback path now embedded — re-run needed to confirm

Rigor

  • If this is a skills change: I used superpowers:writing-skills and
    completed adversarial pressure testing (paste results below)
  • This change was tested adversarially, not just on the happy path
  • I did not modify carefully-tuned content (Red Flags table,
    rationalizations, "human partner" language) without extensive evals
    showing the change is an improvement

Plugin tests (20/20 passing):

tests/hermes/test_bootstrap.py::TestStripFrontmatter::test_strips_yaml_block PASSED
tests/hermes/test_bootstrap.py::TestStripFrontmatter::test_no_frontmatter_returns_trimmed_content PASSED
tests/hermes/test_bootstrap.py::TestStripFrontmatter::test_strips_surrounding_whitespace_from_body PASSED
tests/hermes/test_bootstrap.py::TestGetBootstrap::test_returns_none_when_skill_file_missing PASSED
tests/hermes/test_bootstrap.py::TestGetBootstrap::test_caches_false_on_missing_file PASSED
tests/hermes/test_bootstrap.py::TestGetBootstrap::test_returns_string_with_real_skill PASSED
tests/hermes/test_bootstrap.py::TestGetBootstrap::test_same_object_returned_on_second_call PASSED
tests/hermes/test_bootstrap.py::TestGetBootstrap::test_contains_marker PASSED
tests/hermes/test_bootstrap.py::TestGetBootstrap::test_contains_extremely_important_wrapper PASSED
tests/hermes/test_bootstrap.py::TestGetBootstrap::test_frontmatter_absent_from_output PASSED
tests/hermes/test_plugin.py::TestPluginRegistration::test_register_attaches_session_start_hook PASSED
tests/hermes/test_plugin.py::TestBootstrapInjection::test_first_session_start_injects_bootstrap PASSED
tests/hermes/test_plugin.py::TestBootstrapInjection::test_injection_uses_user_role PASSED
tests/hermes/test_plugin.py::TestBootstrapInjection::test_dedup_skips_on_same_session_id PASSED
tests/hermes/test_plugin.py::TestBootstrapInjection::test_reinjects_on_new_session_id PASSED
tests/hermes/test_plugin.py::TestBootstrapInjection::test_missing_skill_file_skips_silently PASSED
tests/hermes/test_plugin.py::TestBootstrapInjection::test_cache_populated_after_first_call PASSED
tests/hermes/test_plugin.py::TestBootstrapContent::test_contains_extremely_important_tags PASSED
tests/hermes/test_plugin.py::TestBootstrapContent::test_frontmatter_stripped PASSED
tests/hermes/test_plugin.py::TestBootstrapContent::test_tool_mapping_present PASSED

20 passed in 0.02s

Skills change notes: Only one line added to skills/using-superpowers/SKILL.md (a bullet in the Platform Adaptation list). No changes to Red Flags table, rationalizations, or behavior-shaping copy. The skills content is unchanged; only infrastructure was added.

Human review

  • A human has reviewed the COMPLETE proposed diff before submission

arittr and others added 18 commits July 2, 2026 14:56
…s object

Codex auto-discovers a plugin's hooks/hooks.json whenever the Codex
manifest has no `hooks` field: load_plugin_hooks falls back to a
hardcoded DEFAULT_HOOKS_CONFIG_FILE = "hooks/hooks.json" and registers
it. hooks/hooks.json is the Claude Code SessionStart hook, it is tracked
in this repo, and the Codex marketplace installs the whole repo root
(source url "./"), so the fallback re-registered the SessionStart hook
and its install-time trust prompt on Codex.

Removing the Codex hook file and the manifest `hooks` pointer (commit
"Remove Codex hooks") did not disable the hook on Codex — it removed the
explicit declaration that was overriding the fallback, so the fallback
took over and found the Claude hooks/hooks.json.

Declare an empty inline hooks object ({}) in .codex-plugin/plugin.json.
It parses as an empty inline hook set and stops Codex reaching the
auto-discovery fallback. An absent field, an empty array ([]), and an
empty inline list all collapse back to the fallback, so the value must
be exactly {}.

Update the test to assert the manifest declares hooks: {} (and that
hooks/hooks.json exists, which is what makes the declaration necessary),
replacing the prior assertion that the field was absent — which passed
while the hook was still being auto-discovered.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… docs

hooks/session-start-codex has had no caller since "Remove Codex hooks"
(obra#1845) deleted hooks-codex.json and its manifest registration; the
Codex manifest now declares an empty hooks object so Codex registers no
session-start hook at all. The script is Codex-specific dead code —
nothing executes it on Codex or any other harness.

- Delete hooks/session-start-codex.
- tests/hooks/test-session-start.sh: drop the two Codex cases that are
  redundant with the generic session-start tests (nested-format and the
  legacy-warning omission are already covered by the Claude Code cases).
  Re-point the "wrapper dispatches" case to the live `session-start`
  script so run-hook.cmd dispatch coverage — used by Claude Code and
  Cursor in production — is preserved rather than lost.
- docs/porting-to-a-new-harness.md: Codex is no longer a Shape A
  (shell-hook) harness, so re-anchor that worked example to Cursor (a
  live shell-hook harness that demonstrates the same per-harness field,
  schema, and matcher variance) and mark Codex as native skill discovery
  with no session-start hook. Clears the references to the deleted
  hooks-codex.json.
- docs/windows/polyglot-hooks.md: the "check hooks-codex.json" pointer
  referenced a file deleted in obra#1845; re-point to hooks-cursor.json.

RELEASE-NOTES.md keeps its historical mention of hooks-codex.json (it
accurately records what that release did). The tests/codex-plugin-sync
fixtures build their own synthetic session-start-codex and test the sync
mechanism generically, so they are intentionally left as-is.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Scaffold for Hermes Agent harness (Shape B in-process plugin):
- .hermes-plugin/plugin.yaml: manifest with on_session_start hook
- .hermes-plugin/__init__.py: stub with _bootstrap_cache, _SKILLS_DIR, _last_session_id
- tests/hermes/: conftest (simplified, session_id-based dedup), test_plugin.py (10 failing tests)

All tests fail cleanly at stub level (AssertionError/KeyError, no ImportError).
Dedup tests use session_id per Task 1 API discovery (no messages kwarg).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prevent __pycache__, *.pyc, and .pytest_cache from being tracked.
Untrack previously committed .pyc files from tests/hermes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ELY_IMPORTANT wrapper

Implements _strip_frontmatter, _get_bootstrap, _hermes_tool_mapping, and
resolves _SKILLS_DIR in .hermes-plugin/__init__.py. Adds test_bootstrap.py
with 10 passing tests (RED→GREEN). register() remains a stub for Task 4.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements register(ctx) in .hermes-plugin/__init__.py:
- Registers on_session_start hook via ctx.register_hook
- Guards against repeated injection within the same session via _last_session_id
- Injects assembled bootstrap as user-role message on new sessions
- Skips silently when SKILL.md is missing (bootstrap is None/False)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…, add skill_view fallback

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…s skills

skill_view() looks in Hermes' own skill registry and returns 'not found'
for the superpowers bundled skills. Include the resolved _SKILLS_DIR path
in the bootstrap so the model has an explicit read_file fallback path for
skills like brainstorming.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@kumarabd

kumarabd commented Jul 6, 2026

Copy link
Copy Markdown
Author

@obra does it look like something that can be merged in?

@obra

obra commented Jul 23, 2026

Copy link
Copy Markdown
Owner

Thanks @kumarabd — of the three open Hermes PRs this is the strongest: it keeps the stock skills/ tree untouched (which is a hard requirement — Superpowers ships one set of skills, not per-harness copies), adds the standard references/hermes-tools.md adaptation, and comes with real tests. The maintainer wants Hermes support to land, and this PR is the candidate. It needs changes before review:

  1. Injection mechanism vs the documented API. The plugin registers on_session_start and calls ctx.inject_message(bootstrap, role="user"). The official plugin guide (https://hermes-agent.nousresearch.com/docs/developer-guide/plugins) documents no ctx.inject_message, and states on_session_start return values are ignored — the sanctioned context-injection path is a pre_llm_call hook returning {"context": ...} on is_first_turn. Notably, the transcript in this PR shows brainstorming being skipped on the acceptance prompt, which is exactly what silent injection failure would look like. Either switch to the documented pre_llm_call path, or include a raw debug trace proving inject_message exists and fires. Mind the guide's 10,000-character spill threshold for injected context relative to the bootstrap's size.

  2. Register the stock skills instead of documenting a workaround. The injected bootstrap currently tells the model skill_view will return 'not found' for Superpowers skills and to try it anyway. The guide provides ctx.register_skill(name, skill_md_path) on standard Markdown with no conversion — a loop over the stock skills/ directory makes skill_view("superpowers:<name>") work natively and deletes the workaround prose. (A hermes skills tap is the other stock-file-compatible route.)

  3. Rebase to a Hermes-only diff. The branch currently bundles unrelated Codex/packaging/release drift (~27 files). The reviewable change is .hermes-plugin/, docs/README.hermes.md, references/hermes-tools.md, the Platform Adaptation table row, tests/hermes/, and the porting-guide entry.

  4. Fresh acceptance transcript — raw, unedited, clean session, exactly "Let's make a react todo list", showing brainstorming auto-triggering before any code, after the mechanism fix.

The two competing Hermes PRs (#1892, #1861) are being closed in favor of this one. When the four items above are addressed, this goes to maintainer review.

— Claude Fable 5 (AI assistant), Claude Code 2.1.218, doing PR triage for this repo. Direction from @obra.

@obra

obra commented Jul 23, 2026

Copy link
Copy Markdown
Owner

@kumarabd To make item 3 easy: I've pushed a clean rebase of your Hermes work onto current dev as hermes-harness-rebase (commit 7b17761, you're credited as co-author). It's your 27-file branch reduced to the 12-file Hermes surface: .hermes-plugin/, tests/hermes/ (all 20 tests pass on current dev), docs/README.hermes.md, references/hermes-tools.md, the Platform Adaptation row, and the README section. Dropped: the v6.1.0-era codex/release drift and the porting-guide edits (those sections were rewritten on dev since; a Hermes row in Appendix A can be added once the mechanism is final).

To adopt it on this PR:

git fetch https://github.com/obra/superpowers.git hermes-harness-rebase
git reset --hard FETCH_HEAD
git push --force-with-lease origin <your-branch>

That leaves items 1, 2, and 4 from the review above (pre_llm_call injection or proof of inject_message, ctx.register_skill over the stock skills, fresh passing transcript) — all inside .hermes-plugin/__init__.py and the tests, so the remaining diff should stay small.

— Claude Fable 5 (AI assistant), Claude Code 2.1.218, doing PR triage for this repo.

@obra

obra commented Jul 23, 2026

Copy link
Copy Markdown
Owner

Update: the review items above have been resolved empirically, and the hermes-harness-rebase branch now carries a working, eval-verified integration (commit 178528c). Findings from live container runs against the real hermes CLI (GLM 5.2 via OpenRouter), full log in the superpowers-evals repo (docs/experiments/2026-07-23-hermes-target-bringup.md):

  1. Your original mechanism was confirmed broken, precisely: ctx.inject_message exists (it's just undocumented) but returns False when called from on_session_start — a probe plugin confirmed the hook fires and the call lands nothing in the conversation. This explains your own transcript showing brainstorming being skipped.
  2. Two more hermes gotchas we hit on the way: ctx.register_skill requires a pathlib.Path — passing a str raises AttributeError inside hermes, which silently disables the entire plugin (no log line anywhere); and injected context is transient (appended at API-call time, never persisted in session exports), so injection can only be verified behaviorally.
  3. The fix on the branch: pre_llm_call returning {"context": bootstrap} on is_first_turn (the documented path — verified delivering), ctx.register_skill(Path(...)) over the stock skills tree so skill_view("superpowers:brainstorming") works natively (verified live — the model loaded and quoted the stock SKILL.md), layout-robust skills resolution for both the git-clone and flattened installs, loud failure instead of the silent-skip branch, and the tool mapping sourced from references/hermes-tools.md instead of duplicated.
  4. Acceptance evidence: superpowers-evals now has a hermes quorum target; the superpowers-bootstrap scenario (exactly the repo's acceptance test: "Let's make a react todo list") went RED on the original mechanism and GREEN on the fixed branchskill-called superpowers:brainstorming ✓, brainstorming before Write ✓ and before Edit ✓, with the agent's trace explicitly citing the bootstrap's rule.

For this PR that means: reset your branch to hermes-harness-rebase (per the instructions upthread) and the remaining work is only the PR body refresh (mechanism description + your own environment's test results). The maintainer decides how he wants to land it from here.

— Claude Fable 5 (AI assistant), Claude Code 2.1.218, doing PR triage for this repo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants