Skip to content

Feat user skills#139

Merged
simongdavies merged 4 commits into
hyperlight-dev:mainfrom
simongdavies:feat-user-skills
May 15, 2026
Merged

Feat user skills#139
simongdavies merged 4 commits into
hyperlight-dev:mainfrom
simongdavies:feat-user-skills

Conversation

@simongdavies
Copy link
Copy Markdown
Member

This pull request implements the user-generated skills feature in HyperAgent, allowing users to persist learnings from a session as reusable skills (SKILL.md files) that override system skills and survive upgrades. The changes introduce new commands, update documentation, and add core logic for saving, loading, and managing user skills. It also includes comprehensive testing instructions and validation for edge cases.

Key changes:

User Skills Feature Implementation

  • Added support for user-generated skills: users can save session learnings as skills via the /save-skill command, which persists a structured SKILL.md under ~/.hyperagent/skills/<name>/ and optionally a companion module. User skills override system skills with the same name and are loaded automatically on startup.

  • Introduced the generate_skill tool, which handles validation, user approval, and writing of user skills and companion modules. It ensures no silent overwrites and provides detailed feedback to the user.

  • Updated the session configuration to include both system and user skill directories, enabling user skills to be loaded and override system skills on name collision.

Command and REPL Enhancements

  • Extended the /skills command set with subcommands for listing, showing, editing, and deleting user skills. Added the /save-skill command for persisting new skills. User skills are marked with a 👤 badge and overrides are flagged in the UI.

  • Updated the command help text and documentation to reflect the new user skills functionality and management commands.

Documentation and Testing

  • Added a comprehensive TESTING-USER-SKILLS.md guide detailing end-to-end testing, boundary cases, override behavior, and troubleshooting for the user skills feature.

  • Updated SKILLS.md with a new section explaining user skills, how to save them, when to use them, and management commands.

Core and Utility Logic

  • Refactored skill loading logic to support loading from multiple directories (system and user), and updated the approach resolver to handle the new skill directory structure.

  • Added tracking for modules registered during a session, so they can be suggested as companion modules when saving a skill.

  • Introduced a cap on tool-call history to limit memory usage and ensure only recent session activity is summarized for skill generation.

These changes collectively enable users to capture, reuse, and manage domain-specific expertise directly within HyperAgent, making workflows more efficient and persistent across sessions.

marked v15's use() iterates 'for (prop in pack.renderer)' and validates every enumerable key against its known renderer method list, throwing "renderer 'o' does not exist" at module init.

The legacy 'new TerminalRenderer(opts)' route assigns config to own enumerable properties (this.o, this.tab, ...), so the first iteration hits an unknown key and crashes. This broke the agent on every 'just start' since PR hyperlight-dev#135 landed; CI never noticed because no test imports the module.

Switch to the modern markedTerminal() factory which returns a clean MarkedExtension containing only renderer method keys, and add a regression test that import-loads the module and smoke-tests rendering so a future bump can't reintroduce this class of crash.

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
Lets users persist what the agent learned in a session as a reusable skill at ~/.hyperagent/skills/<name>/SKILL.md, surviving upgrades and overriding system skills with the same name.

Triggered via:
- /save-skill [name]  - slash command that builds a synthetic prompt from
  session context (tool history, MCP servers, modules registered, recent
  errors) and asks the LLM to call generate_skill()
- 'save this as a skill' (natural language) - system message documents
  the generate_skill tool so the LLM can call it directly

Components added:
- src/agent/skill-writer.ts: validation + CRUD for user skills,
  with HYPERAGENT_USER_SKILLS_DIR env override for tests
- src/agent/session-context.ts: pure extractor that rolls up tool
  history, MCP servers, modules registered, and recent errors into a
  prompt-ready string
- generate_skill tool: registered in all three gating points
  (tools[], ALLOWED_TOOLS, availableTools[]) with interactive approval
- /skills enhanced with 'info <name>', 'edit <name>', 'delete <name>',
  override-detection badge for user skills
- skill-loader now supports loading from multiple directories with
  override semantics (later dirs win)
- state.ts tracks toolCallHistory (capped FIFO), mcpServersUsed,
  modulesRegistered, pendingPrompt; populated by onPostToolUse hook
  and registerModuleImpl
- system-message.ts documents the saving workflow for the LLM
- docs/SKILLS.md adds 'User Skills (Persist What You Learn)' section

Tests: 39 new (skill-writer 22, session-context 9, skill-loader +8). All 2443 TS tests pass; 124 Rust tests pass; lint clean.
Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
Standalone walkthrough at docs/TESTING-USER-SKILLS.md covering smoke test, full workout, override behaviour, boundary cases, and likely failure modes. Intended to be passed to reviewers / testers who want to exercise the feature without reading the implementation.

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 14, 2026 22:59
@simongdavies simongdavies added the enhancement New feature or request label May 14, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds user-generated skills to Hyperagent so session learnings can be saved as reusable SKILL.md files under the user library and loaded alongside bundled skills.

Changes:

  • Adds user skill persistence, multi-directory skill loading, and /save-skill generation flow.
  • Extends slash commands and agent state to track session context for skill authoring.
  • Adds documentation and tests for skill loading/writing, session summaries, and markdown rendering.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 18 comments.

Show a summary per file
File Description
src/agent/index.ts Wires user skills into session config, adds generate_skill, and tracks tool/session context.
src/agent/skill-writer.ts Adds validation, rendering, CRUD, and persistence for user skills.
src/agent/skill-loader.ts Adds source/file metadata and multi-directory loading with override precedence.
src/agent/session-context.ts Adds session activity summarization for /save-skill.
src/agent/slash-commands.ts Adds /save-skill and user-skill management subcommands.
src/agent/state.ts Adds session-learning state fields.
src/agent/commands.ts Updates help text for skills and save-skill commands.
src/agent/approach-resolver.ts Allows approach resolution from multiple skill directories.
src/agent/tool-gating.ts Allows the new generate_skill tool.
src/agent/system-message.ts Instructs the LLM when to save learned skills.
src/agent/markdown-renderer.ts Switches to markedTerminal() extension setup.
tests/skill-writer.test.ts Adds coverage for user skill validation and CRUD.
tests/skill-loader.test.ts Adds coverage for source metadata and user override loading.
tests/session-context.test.ts Adds coverage for session context extraction/rendering.
tests/markdown-renderer.test.ts Adds markdown renderer smoke/regression tests.
tests/intent-matcher.test.ts Updates skill fixtures for new fields.
tests/approach-resolver.test.ts Updates skill fixtures for new fields.
docs/TESTING-USER-SKILLS.md Adds manual testing guide for user skills.
docs/SKILLS.md Documents saving and managing user skills.
Comments suppressed due to low confidence (2)

src/agent/index.ts:5618

  • Saving the skill before validating/registering the companion module can leave a persisted skill that references a module that was denied or failed validation. That partial state makes the new skill broken for future sessions; either register the module first or roll back/update the skill when the companion module step fails.
      if (params.companionModule) {
        const m = params.companionModule;
        const moduleSaveResult = (await registerModuleImpl({
          name: m.name,

docs/TESTING-USER-SKILLS.md:104

  • This override test does not actually shadow a bundled skill because code-review is not present in the repository's skills/ directory. Use an existing system skill name (for example one listed in skills/CLAUDE.md) so the test verifies override behavior.
User skills must override system skills with the same name. Drop a user
skill that shadows an existing system one:

```bash
mkdir -p "$HYPERAGENT_USER_SKILLS_DIR/code-review"

Comment thread src/agent/slash-commands.ts
Comment thread src/agent/index.ts Outdated
Comment thread src/agent/index.ts
Comment thread src/agent/skill-writer.ts
Comment thread src/agent/skill-writer.ts Outdated
Comment thread src/agent/session-context.ts
Comment thread src/agent/slash-commands.ts
Comment thread src/agent/slash-commands.ts
Comment thread src/agent/index.ts
Comment thread src/agent/skill-writer.ts
Security & correctness
- skill-writer: cap on UTF-8 byte length (not String.length) so a
  multi-byte payload can't bypass the 64 KB limit
- skill-writer: reject reserved /skills subcommand names (info, edit,
  delete, list) to prevent shadowing the CLI surface
- skill-writer: reject description/triggers containing newlines or
  a bare '---' line so they can't break out of YAML frontmatter
- slash-commands /skills info|edit|delete: validate <name> via
  validateSkillName before any filesystem join — closes the path
  traversal vector pointed out by the reviewer

UX correctness
- index.ts generate_skill: surface an 'Overwrite existing user
  skill?' confirmation when overwrite=true and the file already
  exists
- slash-commands /save-skill: pass skipAutoSuggest=true so the
  synthetic prompt's scaffolding terms don't trigger unrelated
  skills via runSuggestApproach
- slash-commands /new: also reset currentUserPrompt + lastGuidance
- slash-commands /resume: reset toolCallHistory, mcpServersUsed,
  modulesRegistered, currentUserPrompt, lastGuidance — local
  session-learning state can't be reconstructed from a resumed
  remote session
- slash-commands /save-skill: fix 'distinct tools' status line to
  count the full tool history, not the bounded topTools view
- session-context: truncate currentUserPrompt to 2000 chars with
  an ellipsis so a giant paste can't dominate the prompt

MCP session-learning correctness
- mcp/plugin-adapter: add optional onCall observer; agent wires it
  to state.mcpServersUsed so calls made from inside
  execute_javascript via host:mcp-<name> imports are now tracked
- state.ts: add skipNextAutoSuggest flag (consumed in
  onUserPromptSubmitted)

Documentation
- docs/TESTING-USER-SKILLS.md: drop branch-name reference, switch
  override example from non-existent 'code-review' to bundled
  'kql-expert', clarify '/skills edit' prints a path (no $EDITOR),
  describe the now-correct overwrite confirmation flow, note that
  the override badge surfaces in '/skills' list view, fix approval
  prompt wording (summary, not full content)

Tests
- Reserved-name rejection
- YAML-unsafe newline rejection (description + trigger)
- UTF-8 byte-length cap (32 KB of 4-byte chars)
- User-prompt truncation contract

Quality gate: 2448 TS tests pass (+5), 124 Rust tests pass.

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
@simongdavies simongdavies merged commit 8cfb28a into hyperlight-dev:main May 15, 2026
12 checks passed
simongdavies added a commit that referenced this pull request May 15, 2026
* docs: changelog for v0.6.0

Move [Unreleased] section to [v0.6.0] - 2026-05-15 covering the 23
commits landed on upstream/main since v0.5.0:

Added:
- User-generated skills from session learnings (#139)
- KQL expert skill with requires-mcp frontmatter and Kusto highlighting (#137)
- Terminal markdown rendering via marked + marked-terminal (#135, #136)
- Verbose/debug gating for diagnostic output (#137)
- execute_bash large output interception (#134)

Fixed:
- marked v15 + marked-terminal v7 incompat in markdown-renderer (#138)
- HybridFs sandbox /tmp path mapping + adapter refactor (#134)
- Prettier mangling of nested template literals in styled output (#134)
- Full error text now wrapped in C.err() across 11 paths (#135)
- /markdown toggle now flips sessionNeedsRebuild (#136)
- looksLikeMarkdown false positives on bold/unordered-list (#136)

Changed:
- Dependency bumps: msal-node 5.2.1, tsx 4.22.0, @types/node 25.8.0,
  tokio in code-validator guest (#142, #143, #146, #147)

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>

* fix: address PR #148 review feedback (3 issues)

CHANGELOG.md:
- Fabric RTI CLI flag: `--mcp setup-fabric-rti` was wrong, actual
  flag in src/agent/cli-parser.ts is `--mcp-setup-fabric-rti`
  (single hyphenated token). Users copy-pasting the old form would
  hit "unknown argument"
- Terminal markdown rendering default: said "raw streaming remains
  the default" but cli-parser.ts sets `markdown: process.env.HYPERAGENT_MARKDOWN
  !== "0"` (default ON) and `--[no-]markdown` help text confirms
  default: on. Reworded to make default-on explicit and document
  the three opt-out paths
- Missing link reference: added
  `[v0.6.0]: https://github.com/hyperlight-dev/hyperagent/releases/tag/v0.6.0`
  to match the existing reference-style link section

No code changes. Prettier clean.

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>

---------

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants