Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
af8e0ab
Release 0.7.1: the tier 0.7.0 announced was never reachable
MongLong0214 Aug 8, 2026
4ae1f6f
Make the terminal answer what the hook answers
MongLong0214 Aug 8, 2026
cb960d0
Merge pull request #485 from MongLong0214/release-071
MongLong0214 Aug 8, 2026
e1a3c92
Make a doctor row carry the observation its verdict rests on
MongLong0214 Aug 8, 2026
661d953
Merge pull request #487 from MongLong0214/doctor-evidence
MongLong0214 Aug 8, 2026
afd48df
Say what a read-only run actually does
MongLong0214 Aug 8, 2026
5ffa5c0
Merge pull request #488 from MongLong0214/prd-writes-wording
MongLong0214 Aug 8, 2026
b910dba
Publish the study the README's claim actually rests on
MongLong0214 Aug 8, 2026
e69a4b8
Merge pull request #489 from MongLong0214/readme-sources-m5
MongLong0214 Aug 8, 2026
d24a284
Say which failure is the cause and which are its echoes
MongLong0214 Aug 8, 2026
a66141e
Merge pull request #490 from MongLong0214/doctor-collapse
MongLong0214 Aug 8, 2026
7dab3e3
Ask the release gate which commitlore the plugin actually ran
MongLong0214 Aug 8, 2026
ce5162a
Merge pull request #491 from MongLong0214/plugin-resolution-notice
MongLong0214 Aug 8, 2026
5b7a823
Check the manifest the canonical install path actually resolves
MongLong0214 Aug 8, 2026
3216a54
Qualify the release before it exists rather than after
MongLong0214 Aug 8, 2026
8ffb31c
Refuse a tag that is not on main or whose CI did not pass
MongLong0214 Aug 8, 2026
9dc7b72
Merge pull request #494 from MongLong0214/version-gate-all-manifests
MongLong0214 Aug 8, 2026
68f3415
Merge pull request #495 from MongLong0214/release-gate-fail-closed
MongLong0214 Aug 8, 2026
ea66f50
Merge pull request #497 from MongLong0214/release-publish-prerequisites
MongLong0214 Aug 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "commitlore",
"displayName": "CommitLore",
"version": "0.7.0",
"version": "0.7.1",
"description": "Recorded decisions from git history, delivered to the agent before it edits. Constraints, alternatives already ruled out, and warnings left by whoever was here last.",
"author": {
"name": "MongLong0214",
Expand Down
184 changes: 183 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@ permissions:
contents: write
id-token: write
attestations: write
# `exact-head-ci` reads the check runs GitHub recorded for the tagged commit.
# Reading a branch's green badge would answer a different, weaker question.
checks: read

jobs:
# Refuses the whole release before a single binary is built. A release
Expand All @@ -37,13 +40,192 @@ jobs:
- name: Tag, package.json, and `commitlore --version` agree
run: node scripts/check-release-version.mjs "$GITHUB_REF_NAME"

# A version-consistent tag can still name a commit that exists only on dev.
# `merge-base` can answer that only from complete history: a shallow checkout
# is not a smaller proof, it is an incomplete graph that the script refuses.
# The accepted commit is an output so the API gate below queries the commit an
# annotated tag resolves to, rather than assuming the event SHA is one.
release-target:
runs-on: ubuntu-latest
outputs:
commit: ${{ steps.ancestry.outputs.sha }}
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.ref }}
fetch-depth: 0
- uses: actions/setup-node@v4
with:
node-version: 22
- id: ancestry
name: The pushed tag is already contained in main
run: node scripts/check-release-target.mjs "$GITHUB_REF_NAME" main --github-output

# A tag push does not create a new CI result for the commit it names. The
# relevant evidence is the existing set of check runs at that exact commit,
# not the last green run on main and not a local suite this release happens
# to start. The script has a fixed required list and treats every missing or
# non-success result as a block; an empty API response is therefore a failure.
exact-head-ci:
needs: release-target
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.ref }}
fetch-depth: 1
- uses: actions/setup-node@v4
with:
node-version: 22
- name: Every required CI check succeeded at the tagged commit
env:
GITHUB_TOKEN: ${{ github.token }}
run: node scripts/check-exact-head-ci.mjs "$GITHUB_REPOSITORY_OWNER" "${GITHUB_REPOSITORY#*/}" "${{ needs.release-target.outputs.commit }}"

# RELEASE-GATE §4 used to be a human checklist run after `gh release create`.
# That made a failed row a correction to something already published: the
# release existed, but its documented installation did not work. This job
# makes the six checks a prerequisite instead, so publication cannot get
# ahead of the artifact it claims to qualify.
install-gate:
runs-on: ubuntu-latest
steps:
# The default checkout follows the event SHA today, but qualifying the
# branch that happened to contain that SHA would weaken the claim when
# checkout's default changes. The pushed tag is the artifact users name.
- uses: actions/checkout@v4
with:
ref: ${{ github.ref }}
fetch-depth: 0
- uses: actions/setup-node@v4
with:
node-version: 22

# A checkout can hide exactly the distribution failures this protects
# against: an untracked bundle, a path that reaches back to the source
# tree, or dependencies left over from CI. Clone the pushed tag anew,
# then run every check from that clone as an installer does.
- name: The tagged installation passes RELEASE-GATE section 4
run: |
set -euo pipefail
clone_dir="$(mktemp -d)/commitlore"
git clone --quiet --branch "$GITHUB_REF_NAME" --depth 1 \
"$GITHUB_SERVER_URL/$GITHUB_REPOSITORY.git" "$clone_dir"
cd "$clone_dir"

# The bundle is the shipped CLI; no build or npm install belongs in
# this job, because either would conceal a clone missing what ships.
clone_version="$(node dist/commitlore.mjs --version)"
printf 'fresh clone version: %s\n' "$clone_version"

if validation_output="$(printf 'Subject\n\nBlast: wide\n' | node dist/commitlore.mjs validate 2>&1)"; then
echo "expected the invalid message to be rejected"
exit 1
else
validation_exit=$?
fi
printf '%s\n' "$validation_output"
test "$validation_exit" -eq 1 || {
echo "validate exited $validation_exit, expected 1"
exit 1
}
case "$validation_output" in
*Blast*) ;;
*)
echo "validate rejected the message without reporting its Blast violation"
exit 1
;;
esac

# A fresh clone is allowed to warn about unfetched notes and setup it
# has not opted into. `doctor` exits zero exactly when none of those
# findings is a fail, which is the RELEASE-GATE row's condition.
node dist/commitlore.mjs doctor

# The installed stub records node's absolute path. This command
# removes node from PATH while retaining git, so a rejection proves
# the hook used that recorded interpreter rather than ambient PATH.
git config user.name "Release gate"
git config user.email "release-gate@example.invalid"
node dist/commitlore.mjs hooks install
if hook_output="$(env -i PATH=/usr/bin:/bin git commit --allow-empty \
-m 'Release gate invalid message' -m 'Blast: wide' 2>&1)"; then
echo "expected the PATH-less hook to reject the invalid commit"
exit 1
else
hook_exit=$?
fi
printf '%s\n' "$hook_output"
test "$hook_exit" -ne 0 || {
echo "the PATH-less hook accepted an invalid commit"
exit 1
}
case "$hook_output" in
*Blast*) ;;
*)
echo "the hook rejected the commit without running validation"
exit 1
;;
esac

# This is the earlier, marker-bearing stub body. Installing first
# records an otherwise healthy target; replacing only the body makes
# a real stale-stub fixture rather than merely testing a missing hook.
hook_path="$(git rev-parse --git-path hooks/commit-msg)"
printf '%s\n' '#!/bin/sh' '# commitlore:commit-msg:v1' \
'exec commitlore validate "$1"' > "$hook_path"
chmod +x "$hook_path"

# Doctor deliberately runs a hook probe with PATH=/usr/bin:/bin.
# This historical body therefore also produces the expected separate
# hook-runtime fail and doctor exits 1. The row being qualified is
# commit-msg-hook, whose required verdict is warn rather than ok, so
# read that row instead of mistaking the fixture's other finding for
# this row's outcome.
if stale_report="$(node dist/commitlore.mjs doctor --json)"; then
stale_doctor_exit=0
else
stale_doctor_exit=$?
fi
printf '%s\n' "$stale_report"
printf 'stale-hook doctor exit: %s\n' "$stale_doctor_exit"
printf '%s' "$stale_report" | node -e '
let input = "";
process.stdin.on("data", (chunk) => { input += chunk; });
process.stdin.on("end", () => {
const report = JSON.parse(input);
const hook = report.checks.find((check) => check.id === "commit-msg-hook");
if (hook?.status !== "warn") {
console.error("expected the stale commit-msg hook to warn, got:", hook);
process.exit(1);
}
console.log("stale commit-msg hook: warn");
});
'

# The runner intentionally prefers a CLI on PATH, so keep PATH narrow
# and include only the node directory needed for the clone fallback.
# Comparing versions catches a different installation that still
# answers --version successfully (#483).
node_dir="$(dirname "$(command -v node)")"
plugin_version="$(env PATH="/usr/bin:/bin:$node_dir" \
CLAUDE_PLUGIN_ROOT="$clone_dir" "$clone_dir/scripts/commitlore-run.sh" --version)"
printf 'plugin version: %s\n' "$plugin_version"
test "$plugin_version" = "$clone_version" || {
echo "plugin resolved $plugin_version, expected clone version $clone_version"
exit 1
}

# Nothing is compiled and nothing is attached. ADR-0026 makes the plugin the
# canonical install path and `install.sh`/`install.ps1` the secondary one, and
# both take a pinned source checkout -- so the release itself is the tag, and the
# tag is what those installers resolve. A release with no assets is the whole
# artifact set, not an incomplete one.
publish:
needs: version-consistency
# A skipped or failed prerequisite skips this job by GitHub's default needs
# semantics. Keep it free of `if:`: a condition such as `always()` would
# turn a failed qualification into a path that can still publish.
needs: [version-consistency, install-gate, release-target, exact-head-ci]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
Expand Down
36 changes: 36 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,41 @@
# Changelog

## 0.7.1

### `[directive]` did not work in 0.7.0

0.7.0's headline change made the `[directive]` tier reachable. **It did not
reach anyone.** `commander` declares `--trusted-author` with a default of `[]`,
so the flag arrived as an empty array rather than `undefined` when it was
absent, the nullish fallback to the author `init` records never fired, and
every record on every install still graded `[claim]`.

That is the defect #415 was opened about, reintroduced one layer up by the fix
for it.

Reproduced against the released artefact: `commitlore inject --path <p>` — the
form the hook runs — rendered `[claim]`, while the same command with an
explicit `--trusted-author` rendered `[directive]`.

The tests that passed drove `buildInjection` with options assembled by hand and
never went through the command line, which is the only path the hook uses.
`test/trusted-authors.test.ts` now spawns the built CLI. Its own header had
already warned that a unit test one layer down would have passed throughout the
period the original bug existed; the same sentence applied one layer up and was
not heard.

### Fixed

- `package-lock.json` still declared `0.1.0` while the manifests read `0.7.0`.
Stale since the first release.

### Corrections to 0.7.0's own record

The promotion PR said 132 commits; it was **137**. It said `RELEASE-GATE.md` §4
lists seven install checks; it lists **six**. Both were miscounts in the
evidence submitted for review, and both are corrected here rather than left in
the history unremarked.

## 0.7.0

### The behaviour claim is measured: 2.8% against 18.8%
Expand Down
51 changes: 33 additions & 18 deletions README.ja.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ Claude Code · Codex · Cursor · Gemini CLI · OpenCode · Windsurf
**その他のコーディングエージェント** — CLI をインストールします:

```bash
curl -fsSL https://raw.githubusercontent.com/MongLong0214/commitlore/v0.7.0/install.sh | sh
curl -fsSL https://raw.githubusercontent.com/MongLong0214/commitlore/v0.7.1/install.sh | sh
```

どの host に対応しているか、各インストール経路が何を必要とするか: [docs/COMPATIBILITY.md](docs/COMPATIBILITY.md)。
Expand Down Expand Up @@ -138,11 +138,11 @@ commitlore context .

```bash
# installer を固定してダウンロードし、確認してから実行します。
curl -fsSLO https://raw.githubusercontent.com/MongLong0214/commitlore/v0.7.0/install.sh
sh install.sh v0.7.0
curl -fsSLO https://raw.githubusercontent.com/MongLong0214/commitlore/v0.7.1/install.sh
sh install.sh v0.7.1

# あるいはスクリプトを使わずに。スクリプトが作るチェックアウトは自分でも作れます。
git clone --depth 1 --branch v0.7.0 https://github.com/MongLong0214/commitlore
git clone --depth 1 --branch v0.7.1 https://github.com/MongLong0214/commitlore
node commitlore/dist/commitlore.mjs --version
```

Expand Down Expand Up @@ -337,40 +337,55 @@ path の履歴は `commitlore context <path>` で読みます。より小さな
<!-- Generated by `node bench/report.ts --section` from the result logs named below. Do not edit by hand:
CI regenerates this block and fails if a single byte differs (scripts/check-readme-numbers.mjs). -->

**112 runs recorded.** No manifest declares how many runs the matrix was meant to produce, so completeness cannot be checked from the logs alone.
**1160 measurements across 1240 rows.** 80 row(s) are superseded by a re-run of the same task, arm and seed, and the analysis counts the survivor. No manifest declares how many runs the matrix was meant to produce, so completeness cannot be checked from the logs alone.

| Where it comes from | |
|---|---|
| Results | `bench/results/t702-m4-final.jsonl` (112 rows) |
| Run id | `20260727T120103Z-aa5eab`, `20260728T025523Z-db4659`, `20260728T025635Z-e3d669`, `20260728T025817Z-d8d0dc` |
| Results | `bench/results/m5-seeds-1-10-rerun.jsonl` (200 rows), `bench/results/m5-seeds-11-20-rerun.jsonl` (200 rows), `bench/results/m5-seeds-21-30.jsonl` (200 rows), `bench/results/m5-seeds-31-40.jsonl` (200 rows), `bench/results/m5-seeds-41-50.jsonl` (200 rows), `bench/results/m5-seeds-51-58.jsonl` (160 rows), `bench/results/m5-seeds-55-58-rerun.jsonl` (80 rows) |
| Run id | `20260802T124657Z-ae3ba0`, `20260802T230855Z-00da79`, `20260803T100356Z-aeb38a`, `20260803T203631Z-77df15`, `20260806T230824Z-60e31e`, `20260807T095937Z-bf2b05`, `20260807T234037Z-6dd0a2` |
| Driver | `claude-headless` |
| Model | not recorded |
| Matrix | 8 tasks, seeds 1, 2, 3, 4, 5, 6, 7 |
| Status | final (declared in `bench/report.ts`, pending a manifest field) |
| Model | `sonnet` |
| Matrix | 10 tasks, seeds 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58 |
| Status | final (declared in `bench/report.ts`, pending a manifest field) (`bench/results/m5-seeds-1-10-rerun.jsonl`), final (declared in `bench/report.ts`, pending a manifest field) (`bench/results/m5-seeds-11-20-rerun.jsonl`), final (declared in `bench/report.ts`, pending a manifest field) (`bench/results/m5-seeds-21-30.jsonl`), final (declared in `bench/report.ts`, pending a manifest field) (`bench/results/m5-seeds-31-40.jsonl`), final (declared in `bench/report.ts`, pending a manifest field) (`bench/results/m5-seeds-41-50.jsonl`), final (declared in `bench/report.ts`, pending a manifest field) (`bench/results/m5-seeds-51-58.jsonl`), final (declared in `bench/report.ts`, pending a manifest field) (`bench/results/m5-seeds-55-58-rerun.jsonl`) |

**Re-proposal and violation rates, every recorded run:**

| Condition | n | Re-proposed | Re-proposal rate | Runs with violations | Violation rate | Mean turns | Mean tokens |
|---|---|---|---|---|---|---|---|
| `commitlore-guard` | 56 | 41 | 0.732 | 0 | 0.000 | 14.8 | 18965 |
| `commitlore-on` | 56 | 35 | 0.625 | 0 | 0.000 | 14.2 | 18091 |
| `commitlore-off` | 620 | 110 | 0.177 | 58 | 0.094 | 19.3 | 38488 |
| `commitlore-on` | 620 | 16 | 0.026 | 7 | 0.011 | 16.2 | 39265 |

**Analysis set — all 112 rows.** Nothing was excluded: no simulated rows, no failed runs, no run that never started.
**Analysis set — 1169 of 1240 rows** (71 excluded: error = 71). A row that failed carries `reproposed: false` because the field is required, not because the agent declined to re-propose; leaving it in the denominator would let the arm that crashed more often look like the arm that behaved better. The excluded runs are counted here, never dropped silently.

**Significance:** not computed — guard exposure is unknown for 112 analysis rows
| Condition | n | Re-proposed | Re-proposal rate | Runs with violations | Violation rate | Mean turns | Mean tokens |
|---|---|---|---|---|---|---|---|
| `commitlore-off` | 584 | 110 | 0.188 | 58 | 0.099 | 20.4 | 40842 |
| `commitlore-on` | 585 | 16 | 0.027 | 7 | 0.012 | 17.1 | 41568 |

**Significance:**

| Quantity | Value |
|---|---|
| Arms | `commitlore-on` (treatment) vs `commitlore-off` (baseline) |
| Re-proposed / did not | `commitlore-on` 16/569, `commitlore-off` 110/474 |
| Fisher exact, two-tailed | p = 1.52e-20 |
| Rate difference, treatment minus baseline | -16.1pp, 95% CI [-19.6pp, -12.7pp] |
| Odds ratio | 0.1212 |
| Paired (task, seed) cells | 579 |
| Rows excluded from the analysis set | 71 |

**How the runs ended** — failures are reported, not filtered:

| Condition | completed | timeout | over-turns | over-tokens | error |
|---|---|---|---|---|---|
| `commitlore-guard` | 56 | 0 | 0 | 0 | 0 |
| `commitlore-on` | 55 | 0 | 1 | 0 | 0 |
| `commitlore-off` | 414 | 3 | 157 | 10 | 36 |
| `commitlore-on` | 459 | 2 | 109 | 15 | 35 |

**Read these numbers with their limits:**

- No model is recorded — neither on the rows nor in a manifest. A re-proposal rate whose model is unknown is not a comparable number, and these figures must not be quoted against another model's.
- Every rate here is conditional on the model that produced it. Re-proposal is a behaviour, and behaviours differ between models, so these figures are not evidence about any other model.
- 112 runs in the analysis set: this matrix is only powered to detect a large effect, so a non-significant result from it is a statement about the sample size, not about CommitLore. The exact power table is in [`bench/README.md`](bench/README.md).
- 585 and 584 runs per arm: this matrix is only powered to detect a large effect, so a non-significant result from it is a statement about the sample size, not about CommitLore. The exact power table is in [`bench/README.md`](bench/README.md).
- Fisher exact treats the runs as independent while the design is paired by (task, seed). It is the pre-registered result, but it is not a valid paired-data test. See the correction in [`docs/VERDICT-M4.md`](docs/VERDICT-M4.md).
<!-- BENCH:END -->

</details>
Expand Down
Loading
Loading