Skip to content

Improvements to H1 Audit skill#163

Draft
mandreko-bitwarden wants to merge 2 commits into
mainfrom
h1-audit-improvements
Draft

Improvements to H1 Audit skill#163
mandreko-bitwarden wants to merge 2 commits into
mainfrom
h1-audit-improvements

Conversation

@mandreko-bitwarden

Copy link
Copy Markdown
Contributor

📔 Objective

This includes some fixes I've been using in my working copy to make the H1 audit run faster and more correct. Some edge cases have been found over the last weeks that excluded specific tickets, which this helps to include. It also adds parallelism to make things run faster (albeit at the cost of readability)

  • major improvements for timing, duplicate reduction, and status changes
  • feat(bitwarden-security-engineer): surface Verified VULN tickets for close out

…close out

The auditing-hackerone-vulns skill previously excluded Verified VULN tickets
from the audit query entirely. A Verified VULN has had its fix confirmed in
production but its Jira ticket still needs to be moved to Closed, so those
tickets silently dropped out of the report.

Stop excluding Verified from the JQL and route these tickets to a new 🏁 Close
Out action token. Verified tickets skip the child-item and GitHub release
checks since the fix is already confirmed, and go straight to the close-out
step. Adds the token to the classification table, decision tree, summary
counts, and output template.

Bumps the plugin to 1.3.0 with changelog entry, and adds "aspirationally" and
"heredocs" to the cspell dictionary.
@github-actions

github-actions Bot commented Jul 13, 2026

Copy link
Copy Markdown

🤖 Plugin Validation — PR #163

Result: ✅ PASS — no errors (must-fix), a few minor warnings (should-fix). Safe to merge.

Scope: bitwarden-security-engineer plugin (v1.3.0)
Changed files:

  • plugins/bitwarden-security-engineer/.claude-plugin/plugin.json
  • plugins/bitwarden-security-engineer/CHANGELOG.md
  • plugins/bitwarden-security-engineer/skills/auditing-hackerone-vulns/SKILL.md

1. Plugin Structure Validation (plugin-validator)

PASS. No critical or major issues.

  • plugin.json — valid JSON; name kebab-case; version 1.3.0 valid semver; description, author (name/url), homepage, repository, keywords, agents all well-formed.
  • Version consistency (target 1.3.0) — PASS:
    • .claude-plugin/marketplace.json1.3.0
    • plugins/bitwarden-security-engineer/.claude-plugin/plugin.json:31.3.0
    • Agent AGENT.md carries no version field (nothing to sync) ✓
    • Descriptions match between marketplace.json and plugin.json ✓
  • CHANGELOG.md — PASS: ## [1.3.0] - 2026-07-10 entry present (CHANGELOG.md:8), Keep a Changelog format, correct ### Changed category, accurately describes the Verified/Close-Out skill change. Descending version order maintained.
  • Directory structure / auto-discovery — PASS.
  • Agent frontmatter — PASS: agents/bitwarden-security-engineer.md — valid name, description, model: opus, color: red, tools includes Skill, substantial system prompt.
  • Credentials — PASS: no hardcoded secrets.

Minor warnings (should-fix, both pre-existing — not introduced by this PR):

  • ⚠️ agents/bitwarden-security-engineer.md:6-12 — the auditing-hackerone-vulns skill is not listed in the agent's skills: block (missing since 1.2.0). If it is intended as a standalone user-invoked skill, no action needed; otherwise register it so the agent can route to it.
  • ⚠️ README.md:11-19 — the Skills table omits auditing-hackerone-vulns (documentation drift since 1.2.0). Consider adding a row for discoverability.

2. Skill Review (skill-reviewer)

PASS. No critical or major errors. File: skills/auditing-hackerone-vulns/SKILL.md (~2,807 words / ~20KB — within the 1,000–3,000 target).

  • Frontmatter — PASS: name matches directory; description present; allowed-tools well-scoped (least-privilege).
  • Description — PASS: third-person, specific trigger phrases (incl. a verbatim quoted query), states output, includes anti-bypass directive.
  • Writing style — PASS: consistently imperative/infinitive; clear Step 1–6 flow.
  • Referenced files — PASS: no external file references, so no broken links.
  • 🏁 Close Out token (the PR's core change) — consistent and correct across the tokens table (:14), Step 1 routing note (:43), Step 4 gate (:100), Step 5 decision tree (:186-188), and Step 6 output — both the summary count row (:208) and the dedicated section (:233-237). The summary count table (the most commonly forgotten spot) was correctly updated.

Minor warnings (should-fix):

  • ⚠️ SKILL.md:187 — in the Step 5 decision tree the action reads 🏁 Move to Closed, whereas every sibling branch uses the token label form (e.g. :172 🔴 Update VULN to In Progress, :173 🟡 Mark Remediated). Change to 🏁 Close Out for parallelism. Cosmetic; meaning is unambiguous.
  • ⚠️ SKILL.md:3 (optional) — description could add an explicit trigger for closing out already-verified tickets to reflect the 1.3.0 scope change ("ready to verify or close" only partially covers it).
  • ⚠️ Progressive disclosure (forward-looking) — the file is within budget now, but Step 4 release-inclusion logic (:135-160) and the Step 6 output template (:198-265) are extraction candidates for references//examples/ if the file grows past 3,000 words.
  • ℹ️ Pre-existing, no regression: label rephrasing between the tokens table (:11-17) and the Step 6 summary table (:206-211) — e.g. "Monitor" vs "Monitoring". The new 🏁 entry follows this same established convention.

3. Security Validation (reviewing-claude-config)

PASS. No critical or important security issues.

  • ✅ No settings.local.json committed.
  • ✅ No hardcoded credentials/secrets. plugin.json holds only public URLs; CHANGELOG.md is docs; SKILL.md URLs are illustrative API endpoint templates.
  • allowed-tools scoping (SKILL.md:4) — all entries are read-only and well-scoped: three read-only MCP Atlassian tools plus Bash(gh api --method GET *) (GET-only, no mutations), Bash(gh pr view *), Bash(gh api repos/bitwarden/*/compare/*), Bash(gh search prs *). No write/delete capability.
  • This PR narrows permissions — it removed Bash(gh release list *) and replaced its usage with a gh api --method GET call already covered by an existing pattern (least-privilege improvement).
  • ✅ No dangerous command auto-approvals. The skill body explicitly forbids file writes, heredocs, interpreters (python3/node/ruby), shell parallelism, and 2>/dev/null error suppression.
  • ✅ No overly broad file access.

Summary

Check Result Errors Warnings
Plugin structure ✅ PASS 0 2 (pre-existing)
Skill review ✅ PASS 0 2 + 2 notes
Security ✅ PASS 0 0

No must-fix issues. Version 1.3.0 is consistent across all three required files, the CHANGELOG entry is correct, the new 🏁 Close Out token is implemented consistently throughout the skill, and the change is security-clean (and in fact tightens allowed-tools). Warnings are minor/cosmetic or pre-existing documentation drift and do not block this PR.

Note: scripts/validate-plugin-structure.sh and scripts/validate-marketplace.sh required interactive approval in this environment and were not executed. Manual inspection indicates both would pass; run them in CI to confirm.

@withinfocus withinfocus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still hoping to see the prompting for dates when a code change happens. Automation in Jira can be added for some of that to actually prompt someone, but if you are already querying GitHub I would think this can call out what to set a confirmation date to.

@@ -11,109 +11,153 @@ allowed-tools: mcp__plugin_bitwarden-atlassian-tools_bitwarden-atlassian__search
| 🔴 | **Update VULN Status** | Child item has progressed (In Progress/Review) but VULN is still at a lower status |
| 🟡 | **Mark Remediated** | Child item is Done — set Remediation Date to merged PR date and move VULN to Remediated |
| 🟢 | **Verify & Close** | Fix is in a release that has already shipped — verify in prod, add Confirmation Date, close HackerOne |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎨 As we discussed in chat, the confirmation date usually gets set shortly after one is remediated, so you know when the release will go out and therefore the validation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants