Repository navigation
Releases: Skillkeel/skillkeel-starter
Releases · Skillkeel/skillkeel-starter
Release list
v0.1.13
- guard-bash and secret-scan now refuse when they cannot read the hook input. Before, guard-bash fell back to matching the raw JSON when python3 was not on PATH, and that let
rm -rf /,rm -rf ~,rm -rf .,git reset --hardandmkfsthrough with exit 0; secret-scan exited 127, which Claude Code treats as a non-blocking error, so the write went ahead. guard-bash now reads the command with python3, else jq, and exits 2 if neither can parse the payload; secret-scan exits 2 when python3 is missing. The session overrides (SKILLKEEL_ALLOW_DANGEROUS=1,SKILLKEEL_ALLOW_SECRETS=1) still work without python3. - README: the npx installer runs from GitHub (
npx github:skillkeel/skillkeel-starter); the package is not on the npm registry yet. - Tests: 89 hook cases (was 79), including runs with python3 and jq removed from PATH, jq alone, and an unparseable payload.
Also in this release, from 0.1.12 (2026-10-01), which had no release page:
- npx installer:
npx skillkeel-starterrunsclaude plugin marketplace add skillkeel/skillkeel-starterandclaude plugin install skillkeel-starter@skillkeelfor you (--scope user|project|local, default user), andupdateanduninstallcall the matchingclaude plugincommands. It copies no files and edits no settings itself; Claude Code does the install. Packaged for npm asskillkeel-starterbut not published there yet; until it is, run it asnpx github:skillkeel/skillkeel-starter(cli/install.js, no dependencies, Node 18 or later). Tested on Claude Code 2.1.286 in an empty home directory: install, a second install, update and uninstall. - evals: a rerun on the same day no longer overwrites the earlier attempt.
evals/run.shmoves the previous record, transcripts and stderr of that case intoevals/<case>/attempts/with the time in the name. On 2026-09-29 one claude-md-init attempt listed a directory under~/.cacheand the escapes check flagged it, then the passing rerun wrote over that record, so the finding was gone from the counts.
v0.1.11
- secret-audit: new step 7. Before the final message, the skill re-reads its whole reply and masks every key-shaped string still shown in full (first 4 + last 2), placeholders and documentation examples included. On Claude Code 2.1.284 the findings table was masked, but a note quoted the AWS docs example key pair in full, although step 6 already forbade it. The masked grader caught it. Found by our own eval run on Claude Code 2.1.284.
- evals: the pr-description grader
no-footernow matches the footer itself (Generated with [or the robot line). The description had no footer, and the grader had tripped on the model's sentence saying it left the "Generated with Claude Code" footer off. The claude-md-init gradernot-npm-testpasses a line that rulesnpm testout ("There is notestscript, sonpm testwill not work") and still fails any other line that names it. Both were grader faults. Found by our own eval run on Claude Code 2.1.284. - CHANGELOG: the 0.1.9 entry quoted the secret-scan test fixture, a Postgres URL with a password in it, and a scanner reads that as a committed secret. The entry now describes the fixture and says the same thing. Reported by sgharlow's skillcrossroads scan.
- Each SKILL.md has a Verify section (was Eval) with the command that runs its case,
bash evals/run.sh /tmp/skillkeel-evals <skill>, and the folderevals/<skill>/that holds the records and transcripts. - Transcript run on 2.1.284: 8 of 8 after the fixes (docs/compat.md). 79 hook tests, 32 record checks, CLI verb snapshot of 2026-09-28 (trufflehog 3.97.9). Hooks unchanged.
v0.1.10
- docs: the 0.1.9 ZIP carried a docs/compat.md that still said 72 hook tests next to a README that said 79 (the table is generated after the eval run, which ran after the tag). Regenerated table with the 2.1.278 row of 2026-09-21 (8 of 8) and the 79 count; README release line. No code change; hooks, skills and tests as in 0.1.9.
v0.1.9
- secret-scan: a credential name with a prefix now counts. The generic pattern started with a word boundary right before
api_key,secret,passwordortoken, and an underscore is a word character, soSUPABASE_SERVICE_TOKEN = "...",NOTION_SECRET = "..."andCLIENT_PASSWORD = '...'were written without a word (1 of 5 .mcp.json-shaped lines caught on 2026-09-21). The pattern now takes any[A-Za-z0-9_]prefix;token_count = "..."andMAX_TOKENS_PER_REQUESTstill pass because the name has to end at the key word. - secret-scan: new pattern for a connection URL with the password inline,
scheme://user:pass@host(postgres://app:S3cretPassw0rd9@db.internal:5432/app). A password without a digit is read as a placeholder, sopostgres://user:password@localhostin an.env.examplepasses. 79 hook tests (was 72). Found while checking a claim for an X reply about hardcoded secrets in public MCP configs; no user report. - evals: Claude Code 2.1.278 ships agents-md as a built-in plugin (init line path "builtin");
evals/record.pyno longer counts built-in plugins as a leak in the isolation check, so a run with only built-ins beside this plugin passes. 32 record checks. Transcript run on 2.1.278: all cases pass (docs/compat.md). - guard-bash, session-start and the skills unchanged.
v0.1.8
- guard-bash: rule 3b, ORM-level database wipes. The framework drops every table itself, so no DROP appears in the command and rule 3 never saw it:
php artisan migrate:fresh,migrate:refresh,db:wipe;rails db:drop,db:reset,db:schema:load;prisma migrate reset;manage.py flushandsqlflush;sequelize db:dropanddb:migrate:undo:all;knex migrate:rollback --all;alembic downgrade base. Plainmigrate,db:migrate,prisma migrate devand single-step rollbacks pass. Found through the r/ClaudeCode thread wheremigrate:fresh --seed --force --env=testingresolved to the dev MariaDB and dropped every table (1wlawz8, 2026-09-20); the command passed guard-bash 0.1.7. 72 hook tests (was 64). - Skills and the other hooks unchanged.
v0.1.7
- README: "This plugin stops it" is now "blocks the commands and writes listed below (a guard, not a sandbox; the tamper-cases corpus shows what it catches and what it does not)"; the Kit price line says "plus VAT where it applies, added at checkout".
- evals:
evals/record.pyrecordsescapes, tool calls whose path (Read, Edit, Write, Glob, Grep) or Bash command before any heredoc names the plugin checkout, the home directory, a parent directory or a sibling case; one escape fails the case andcluster.pyfiles it under runner as "reads outside the fixture". Text the run writes is not checked, so a CLAUDE.md that mentions~/.claudepasses. The 0.1.6 claim (no transcript read the grader) was a grep; this makes it a record field. Today's eight records re-graded from their streams: 0 escapes. 30 record checks. - Hooks and skill texts unchanged, 64 hook tests.
v0.1.6
- evals: trigger mode.
bash evals/run.sh --triggerruns every case with the fixture'strigger:line, a request that never names the skill ("Commit the staged changes."), so the skill-fired grader tests the description and not only the invocation path; records carrymode: triggerand docs/compat.md has a Trigger prompts column. First run on 2.1.276: 8/8 fired. Two graders were stricter than the skill text and are fixed: claude-md-initno-install-runmatchedpnpm installinside the heredoc that wrote CLAUDE.md (now command position only); readme-refreshdeadlink-handleddemanded the dead link removed while the skill allows a<!-- TODO: verify -->marker (now accepts the marked form, before or on the same line as the link; the named rerun on 2.1.276 put it after the link). Named rerun on 2.1.276: 8/8. Found through the HN thread on skill triggering (item 49744398), no user report. - evals:
evals/record.pyrecords the plugin list from the run's init event; any plugin besides skillkeel-starter fails the case as "not isolated" andcluster.pyfiles it under runner with the plugin names. The 09-17 leak (a user-level hook loaded into every run while the suite stayed green) would have failed on the record line instead of in a transcript read. 25 record checks. - Hooks and skill texts unchanged, 64 hook tests. No transcript in the 21 eval runs kept in this repo reads the grader, the fixture script or any path outside the fixture directory (checked 2026-09-19); the grader still sits within reach of the run through the plugin directory, so this is an observation, not a guarantee.
v0.1.5
- secret-audit: a note that a found value matches a known placeholder (the AWS docs example key) names the placeholder and keeps the masked form. On Claude Code 2.1.274 the skill masked the key in the findings table and then quoted it in full in a note; the masked grader caught it.
- pr-description: the body ends after the Risk / rollback section and carries no attribution line even when the harness asks for one; users who want attribution off everywhere are pointed to the includeCoAuthoredBy setting. On 2.1.274 the "Generated with Claude Code" footer came back (first seen and fixed 2026-09-13); the no-footer grader caught it.
- evals: dependency-audit grader
bumpedaccepts"lodash":"4.18.1"without a space (npm writes compact JSON; the pattern expected a space, a grader fault); commit-message graderno-trailermatches the trailer formCo-Authored-By:and the footer link, not the words (the model's sentence "I omitted the Co-Authored-By trailer" had tripped it).evals/run.shpasses--setting-sources projectso the maintainer's user-level plugins and hooks stay out of the run, and setsGIT_CONFIG_GLOBAL=/dev/nullplus a fixture git identity so the model never sees or commits with the maintainer's; run records replace the case directory, the home directory and any global git identity with placeholders before they are written (a user-level "be terse" hook had turned a report into "Done.").evals/record.pyreads the Claude Code version from the init event and judges regex graders withtraceand{source: file}targets locally (19 checks). - docs/compat.md: table "tested against Claude Code versions", generated from the eval results in this repo; the README links it. 2.1.274 (2026-09-17): 3/3 native git-free cases, 5/5 transcript cases after the two skill fixes above.
- Hooks unchanged, 64 tests.
v0.1.4
- guard-bash: git settings that make git run a command are now refused when set through
git configorgit -c:core.fsmonitor,core.hooksPath,core.sshCommand,core.pager,core.editor,core.askPass,core.gitProxy,core.alternateRefsCommand,diff.external, difftool and mergetool commands, merge drivers, filter clean/smudge/process, credential helpers, gpg programs,sequence.editor, uploadpack and receive hooks, and shell aliases (alias.x '!...'). Shell writes into a.git/path (>> sub/.git/config,> .git/hooks/pre-commit) andmv,cp,lnorrsyncwith a.gittarget are refused too. Reads (--get,--list,cat .git/config),--unset, and plain keys such asuser.emailorcore.autocrlfpass. Prompted by accomplish.ai's Beltdown write-up (2026-09-11): a repo-plantedcore.fsmonitorin a nested.git/configran outside the Claude Code sandbox because the harness runs its own git there (fixed upstream in 2.1.247). Before this release all five commands from that chain passed guard-bash. - tests: 20 new guard cases (64 total).
- Known, on purpose: a
.git/configedited through the Write or Edit tool is not covered by this hook; Claude Code's own file tools refuse.gitwrites.
v0.1.3
- tests: CLI verb gate.
tests/cli-verbs.jsonrecords the subcommands and flags that the latest releases of gitleaks, trufflehog, pip-audit and gh accept, read from each binary's own help output.python3 tests/test-cli-verbs.pygreps every<cli> <verb> --flagout of the skill texts and fails on anything the snapshot does not know, so a renamed subcommand (gitleaksdetectin 0.1.1) fails on the commit that touches the skill instead of in a transcript. Shape from u/EvalRaccoonDev on r/ClaudeCode. - evals: run record.
evals/run.shnow keeps the stream-json output of each run and writesrecord-<date>.jsonper case (exit code, seconds, every tool call, skill invocations, last message) with the verdict of each grader that needs no model (tool_used, regex, file_exists).evals/cluster.pygroups failed cases by their first mechanical mismatch (runner, skill not invoked, tool count, file, text, judge) before anyone opens a transcript. Also from the thread with u/EvalRaccoonDev. - Hooks and skills unchanged, 44 hook tests; 17 checks for the record and cluster tools; the verb gate checks 6 commands in 8 files. Run records scrub the plugin path and the home directory before they are written.