feat(release): add LLM-generated prose summary to release notes by jdx · Pull Request #421 · jdx/usage

jdx · 2026-01-18T22:11:07Z

Summary

Adds LLM-generated editorialized release notes to both CHANGELOG.md and GitHub releases using Claude Code.

New script scripts/gen-release-summary.sh: Generates user-friendly editorialized release notes using Claude Code (sandboxed with read-only tools)
Updated tasks/release-plz: Integrates LLM generation into the release process, replacing raw git-cliff output with editorialized content
Updated .github/workflows/publish-cli.yml: Extracts release notes from CHANGELOG.md instead of regenerating

How it works

During release-plz, the script generates editorialized release notes using Claude Code CLI
The editorialized content is used for:
- CHANGELOG.md (with standard header format)
- Release PR body
GitHub releases extract the same content from CHANGELOG.md (single source of truth)

Security

The Claude Code agent is sandboxed with --allowedTools "Read,Grep,Glob" to prevent any file modifications or command execution.

Example output format

The LLM produces release notes with:

1-2 paragraph summary of important changes
Organized sections (Highlights, Bug Fixes, etc.)
User-focused language (not commit messages)
Contributor attribution where appropriate

Test plan

Verify gen-release-summary.sh produces well-formatted output locally
Run through a release cycle in staging/test environment
Confirm CHANGELOG.md and GitHub release have matching content

🤖 Generated with Claude Code

Note

Introduces AI-authored release notes for tagged releases and streamlines the release automation.

Workflow (.github/workflows/publish-cli.yml): Installs Claude Code, computes previous tag, and generates release notes via scripts/gen-release-notes.sh; falls back to CHANGELOG.md if unavailable; switches to --notes-file; ensures full history with fetch-depth: 0.
New script: scripts/gen-release-notes.sh builds a prompt from git-cliff output and calls Claude in a read-only sandbox to produce user-friendly notes.
Release task (tasks/release-plz): Keeps git-cliff as the source of truth for CHANGELOG.md; derives PR_BODY from unreleased notes (sans header); updates PR create/edit logic to use this body and the release head branch.

^{Written by Cursor Bugbot for commit 9682074. This will update automatically on new commits. Configure here.}

- New script `scripts/gen-release-summary.sh`: Generates user-friendly editorialized release notes using Claude Code (sandboxed with read-only tools) - Updated `tasks/release-plz`: Integrates LLM generation into the release process, replacing raw git-cliff output with editorialized content - Updated `.github/workflows/publish-cli.yml`: Extracts release notes from CHANGELOG.md instead of regenerating Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Copilot

Pull request overview

This PR adds LLM-generated, user-friendly release notes to the release process using Claude Code. Instead of using raw git-cliff output, the release workflow now generates editorialized prose summaries that explain changes in user-focused language.

Changes:

Introduces a new script that uses Claude Code CLI to transform git-cliff changelog into editorialized release notes
Updates the release-plz task to integrate LLM generation, formatting, and insertion into CHANGELOG.md
Modifies the GitHub release workflow to extract notes from CHANGELOG.md instead of regenerating them

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
tasks/release-plz	Integrates LLM generation script, handles CHANGELOG.md insertion, and updates PR body with editorialized notes
scripts/gen-release-summary.sh	New script that invokes Claude Code to generate user-friendly release notes from git-cliff output
.github/workflows/publish-cli.yml	Extracts release notes from CHANGELOG.md for GitHub releases

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-18T22:11:47Z

-gh pr create --title "chore: release $version" --body "$changelog" --label "release" ||
-  gh pr edit --title "chore: release $version" --body "$changelog"
+
+if [[ "$(gh pr list --label release)" == "" ]]; then


The command gh pr list --label release returns output even when no PRs exist (e.g., headers or formatting). Use --json output with --jq to reliably check for empty results, or add | grep -v '^$' to filter empty lines.

Suggested change

if [[ "$(gh pr list --label release)" == "" ]]; then

if [[ "$(gh pr list --label release --json number --jq 'length')" == "0" ]]; then

Copilot · 2026-01-18T22:11:47Z

+EDITORIALIZED_NOTES="$(./scripts/gen-release-summary.sh "$version" "$prev_tag")"
+
+# Validate we got non-empty release notes
+if [[ -z $EDITORIALIZED_NOTES ]]; then


The variable should be quoted to prevent word splitting issues: [[ -z \"$EDITORIALIZED_NOTES\" ]].

Suggested change

if [[ -z $EDITORIALIZED_NOTES ]]; then

if [[ -z "$EDITORIALIZED_NOTES" ]]; then

Copilot · 2026-01-18T22:11:48Z

+
+# Create the release header with appropriate URL format
+RELEASE_DATE="$(date +%Y-%m-%d)"
+if [[ -n $prev_tag ]]; then


The variable should be quoted to prevent word splitting issues: [[ -n \"$prev_tag\" ]].

Suggested change

if [[ -n $prev_tag ]]; then

if [[ -n "$prev_tag" ]]; then

Copilot · 2026-01-18T22:11:48Z

+	}
+' CHANGELOG.md >CHANGELOG.md.tmp || awk_exit_code=$?
+
+if [[ $awk_exit_code -ne 0 ]]; then


The variable should be quoted for consistency: [[ \"$awk_exit_code\" -ne 0 ]].

Suggested change

if [[ $awk_exit_code -ne 0 ]]; then

if [[ "$awk_exit_code" -ne 0 ]]; then

codecov · 2026-01-18T22:12:53Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 50.08%. Comparing base (5424214) to head (9682074).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #421   +/-   ##
=======================================
  Coverage   50.08%   50.08%           
=======================================
  Files          47       47           
  Lines        6100     6100           
  Branches     6100     6100           
=======================================
  Hits         3055     3055           
  Misses       1513     1513           
  Partials     1532     1532

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

cursor

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}

cursor · 2026-01-18T22:20:37Z

+  gh pr create --title "chore: release $version" --body "$EDITORIALIZED_NOTES" --label "release" --head release
+else
+  gh pr edit --title "chore: release $version" --body "$EDITORIALIZED_NOTES"
+fi


PR check uses repo-wide label query instead of branch-specific

Low Severity

The PR existence check uses gh pr list --label release which queries ALL open PRs with that label repo-wide, not just PRs from the release branch. If an unrelated PR has the "release" label, the condition evaluates incorrectly, causing gh pr edit to fail (no PR exists for the current branch). The previous fallback pattern gh pr create ... || gh pr edit ... was more robust since it handled both cases correctly.

Add prompt instruction and runtime validation to prevent LLM from generating "## [" patterns that would corrupt changelog processing: - Added explicit instruction in prompt to never use "## [" - Added grep validation that rejects output containing the pattern - Exits with error if pattern detected Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Use printf-based prompt construction to avoid backtick command substitution - Add instructions for PR links and documentation links in release notes Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}

…releases - gen-release-summary.sh: concise 1-paragraph + bullets for CHANGELOG.md - gen-release-notes.sh: rich detailed notes for GitHub releases - Update publish-cli.yml to generate rich notes at release time Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}

LLM-generated changelog content should only be created after the release PR is merged, not during PR creation. Now: - release-plz uses git-cliff directly for CHANGELOG.md - LLM editorialization only happens in release.yml for GitHub releases - Removes gen-release-summary script (no longer needed) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

cursor

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}

- Add fetch-depth: 0 to checkout step so git describe and git cliff have access to full git history - Validate Claude output is non-empty before using it, exit with error if empty so workflow falls back to CHANGELOG.md Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The PR title already contains the version, so strip the redundant version header (## [version] - date) from the PR body. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Log version, previous version, and changelog length before calling Claude - Capture stderr from Claude CLI and display on failure - Helps diagnose why LLM generation fails in CI Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}

cursor · 2026-01-19T02:09:53Z

+		--model claude-opus-4-20250514 \
+		--output-format text \
+		--allowedTools "Read,Grep,Glob" 2>&1
+); then


Stderr redirect may corrupt release notes output

Medium Severity

The 2>&1 redirect on line 62 captures both stdout and stderr from the Claude CLI into $output. If Claude CLI outputs any informational messages, progress indicators, or warnings to stderr during successful operation, those would be mixed into the release notes output (echoed on line 75). This would silently corrupt the release notes without any error indication, making it difficult to debug.

@jdx

### 🚀 Features - **(release)** add LLM-generated prose summary to release notes by [@jdx](https://github.com/jdx) in [#421](#421) - add LLM-generated release notes for GitHub releases by [@jdx](https://github.com/jdx) in [#423](#423) - add spec lint command by [@jdx](https://github.com/jdx) in [#430](#430) ### 🐛 Bug Fixes - replace unsafe path unwrap chains with proper error handling by [@jdx](https://github.com/jdx) in [#424](#424) - pass positional args through to executed scripts by [@jdx](https://github.com/jdx) in [#425](#425) - replace unimplemented!() with proper errors for unsupported shells by [@jdx](https://github.com/jdx) in [#432](#432) - update claude CLI model and add bypassPermissions by [@jdx](https://github.com/jdx) in [#435](#435) ### 🚜 Refactor - remove unused double-shebang support by [@jdx](https://github.com/jdx) in [#426](#426) - replace once_cell with std::sync::LazyLock by [@jdx](https://github.com/jdx) in [#428](#428) - improve code quality with safety and lint fixes by [@jdx](https://github.com/jdx) in [#427](#427) ### ⚡ Performance - use Arc for flag/arg keys in ParseOutput to reduce cloning by [@jdx](https://github.com/jdx) in [#422](#422) ### 🔍 Other Changes - update insta snapshots to newer format by [@jdx](https://github.com/jdx) in [#429](#429) - fix legacy inline snapshot format warnings by [@jdx](https://github.com/jdx) in [#433](#433) - replace TODO with doc comment for subcommand_lookup by [@jdx](https://github.com/jdx) in [#434](#434) ### 📦️ Dependency Updates - update actions/setup-node digest to 6044e13 by [@renovate[bot]](https://github.com/renovate[bot]) in [#419](#419) - replace dependency @tsconfig/node22 with @tsconfig/node24 by [@renovate[bot]](https://github.com/renovate[bot]) in [#418](#418)

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [usage](https://github.com/jdx/usage) | minor | `2.12.0` → `2.13.1` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>jdx/usage (usage)</summary> ### [`v2.13.1`](https://github.com/jdx/usage/blob/HEAD/CHANGELOG.md#2131---2026-01-19) [Compare Source](jdx/usage@v2.13.0...v2.13.1) ##### 🐛 Bug Fixes - use correct PowerShell casing in enum variant by [@jdx](https://github.com/jdx) in [#438](jdx/usage#438) ### [`v2.13.0`](https://github.com/jdx/usage/blob/HEAD/CHANGELOG.md#2130---2026-01-19) [Compare Source](jdx/usage@v2.12.0...v2.13.0) ##### 🚀 Features - **(release)** add LLM-generated prose summary to release notes by [@jdx](https://github.com/jdx) in [#421](jdx/usage#421) - add LLM-generated release notes for GitHub releases by [@jdx](https://github.com/jdx) in [#423](jdx/usage#423) - add spec lint command by [@jdx](https://github.com/jdx) in [#430](jdx/usage#430) - add PowerShell completion support by [@jdx](https://github.com/jdx) in [#431](jdx/usage#431) ##### 🐛 Bug Fixes - replace unsafe path unwrap chains with proper error handling by [@jdx](https://github.com/jdx) in [#424](jdx/usage#424) - pass positional args through to executed scripts by [@jdx](https://github.com/jdx) in [#425](jdx/usage#425) - replace unimplemented!() with proper errors for unsupported shells by [@jdx](https://github.com/jdx) in [#432](jdx/usage#432) - update claude CLI model and add bypassPermissions by [@jdx](https://github.com/jdx) in [#435](jdx/usage#435) ##### 🚜 Refactor - remove unused double-shebang support by [@jdx](https://github.com/jdx) in [#426](jdx/usage#426) - replace once\_cell with std::sync::LazyLock by [@jdx](https://github.com/jdx) in [#428](jdx/usage#428) - improve code quality with safety and lint fixes by [@jdx](https://github.com/jdx) in [#427](jdx/usage#427) ##### ⚡ Performance - use Arc for flag/arg keys in ParseOutput to reduce cloning by [@jdx](https://github.com/jdx) in [#422](jdx/usage#422) ##### 🔍 Other Changes - update insta snapshots to newer format by [@jdx](https://github.com/jdx) in [#429](jdx/usage#429) - fix legacy inline snapshot format warnings by [@jdx](https://github.com/jdx) in [#433](jdx/usage#433) - replace TODO with doc comment for subcommand\_lookup by [@jdx](https://github.com/jdx) in [#434](jdx/usage#434) ##### 📦️ Dependency Updates - update actions/setup-node digest to [`6044e13`](jdx/usage@6044e13) by [@renovate\[bot\]](https://github.com/renovate\[bot]) in [#419](jdx/usage#419) - replace dependency [@tsconfig/node22](https://github.com/tsconfig/node22) with [@tsconfig/node24](https://github.com/tsconfig/node24) by [@renovate\[bot\]](https://github.com/renovate\[bot]) in [#418](jdx/usage#418) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

Copilot AI review requested due to automatic review settings January 18, 2026 22:11

Copilot AI reviewed Jan 18, 2026

View reviewed changes

cursor Bot reviewed Jan 18, 2026

View reviewed changes

jdx and others added 2 commits January 18, 2026 16:25

fix: prevent command substitution and add PR/doc links to release notes

62b2570

- Use printf-based prompt construction to avoid backtick command substitution - Add instructions for PR links and documentation links in release notes Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

cursor Bot reviewed Jan 18, 2026

View reviewed changes

Comment thread tasks/release-plz Outdated

cursor Bot reviewed Jan 18, 2026

View reviewed changes

Comment thread .github/workflows/publish-cli.yml

cursor Bot reviewed Jan 19, 2026

View reviewed changes

Comment thread .github/workflows/publish-cli.yml

Comment thread scripts/gen-release-notes.sh

jdx and others added 3 commits January 18, 2026 19:10

fix: strip version header from PR body

ce72428

The PR title already contains the version, so strip the redundant version header (## [version] - date) from the PR body. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

jdx merged commit 3355a05 into main Jan 19, 2026
9 checks passed

jdx deleted the feat/llm-release-notes branch January 19, 2026 02:05

cursor Bot reviewed Jan 19, 2026

View reviewed changes

mise-en-dev mentioned this pull request Jan 19, 2026

chore: release v2.13.0 #420

Merged

mise-en-dev mentioned this pull request Jan 19, 2026

chore: release v2.13.1 #436

Merged

	if [[ "$(gh pr list --label release)" == "" ]]; then
	if [[ "$(gh pr list --label release --json number --jq 'length')" == "0" ]]; then

	if [[ -z $EDITORIALIZED_NOTES ]]; then
	if [[ -z "$EDITORIALIZED_NOTES" ]]; then

	if [[ $awk_exit_code -ne 0 ]]; then
	if [[ "$awk_exit_code" -ne 0 ]]; then

Uh oh!

Conversation

jdx commented Jan 18, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

How it works

Security

Example output format

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 18, 2026

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Jan 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cursor Bot Jan 18, 2026

Choose a reason for hiding this comment

PR check uses repo-wide label query instead of branch-specific

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot Jan 19, 2026

Choose a reason for hiding this comment

Stderr redirect may corrupt release notes output

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jdx commented Jan 18, 2026 •

edited by cursor Bot

Loading

codecov Bot commented Jan 18, 2026 •

edited

Loading