feat(cli): add AI changelog rollup with version_bump_reason field#13469
feat(cli): add AI changelog rollup with version_bump_reason field#13469Swimburger merged 12 commits intomainfrom
Conversation
Adds ConsolidateChangelog BAML function that deduplicates and formats multi-chunk changelog entries with markdown category headers and emojis. Called after chunk analysis when multiple entries exist, with fallback to raw entries on failure. Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
…tput format Update ConsolidateChangelog prompt to aggressively deduplicate repetitive entries and produce two outputs: CHANGELOG.md entry (Keep a Changelog format) and PR description (with Breaking Changes Before/After code fences and What's New sections grouped by theme). Add pr_description field to response model and pass through in LocalTaskHandler and sdkDiffCommand. Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
…tion responses Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
…matting Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
… prompt count Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
… changes Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
…_bump_reason Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
…tedFiles to GithubStep Previously, handleAutoVersioning computed prDescription and versionBumpReason but copyGeneratedFiles silently dropped them. Now these fields flow through the full chain: LocalTaskHandler -> runGenerator -> runLocalGenerationForWorkspace -> PostGenerationPipeline -> GithubStep -> parseCommitMessageForPR. PR body now uses prDescription (structured Before/After code fences) when available, and prepends versionBumpReason as a bold header. Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
| const rollup = await bamlClient.ConsolidateChangelog(rawEntries, bestBump, "unknown"); | ||
| changelogEntry = rollup.consolidated_changelog?.trim() || rawEntries; | ||
| versionBumpReason = rollup.version_bump_reason?.trim() || ""; |
There was a problem hiding this comment.
🟡 sdkDiffCommand discards pr_description from ConsolidateChangelog rollup, inconsistent with LocalTaskHandler
In sdkDiffCommand.ts, when the ConsolidateChangelog AI call succeeds (line 206), rollup.pr_description is computed but never captured or returned. In contrast, LocalTaskHandler.ts:352-353 correctly captures both consolidated_changelog and pr_description from the same rollup call. Since sdkDiffCommand returns AnalyzeCommitDiffResponse (which lacks a pr_description field), the structured PR description with Before/After code fences for breaking changes is silently lost in the CLI sdk-diff command path. This is an incomplete transformation — the same consolidation feature was added to both code paths, but only LocalTaskHandler captures the full result.
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
Acknowledged. The sdkDiffCommand returns AnalyzeCommitDiffResponse (Fiddle API type) which doesn't have a pr_description field — that field is only meaningful in the local generation path where it flows into GitHub PR creation via GithubStep. The sdk-diff CLI command analyzes diffs but doesn't create PRs, so pr_description isn't actionable there. If this command is later used to feed PR creation, we'd need to extend the response type.
| if (bestBump !== prevBest) { | ||
| bestMessage = chunkAnalysis.message; | ||
| bestVersionBumpReason = chunkAnalysis.version_bump_reason?.trim() || ""; |
There was a problem hiding this comment.
🟡 Multi-chunk merge only updates bestMessage/bestVersionBumpReason when bump level strictly increases, losing context when same-level chunks arrive
In both sdkDiffCommand.ts:172-174 and LocalTaskHandler.ts:316-318, bestMessage and bestVersionBumpReason are only updated when bestBump !== prevBest (i.e., the version bump level strictly increased). If the first chunk returns MAJOR with a vague message and a later chunk also returns MAJOR with a more specific breaking-change reason, the second chunk's message and reason are silently discarded because maxVersionBump(MAJOR, MAJOR) returns MAJOR which equals prevBest. This means the final commit message and version bump reason are determined solely by whichever chunk first reaches the highest bump level, regardless of whether later chunks contain more relevant information. This is a pre-existing pattern, but the new bestVersionBumpReason field inherits the same issue.
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
Acknowledged — this is a pre-existing pattern as noted. The bestVersionBumpReason inherits the same "first to reach highest bump wins" behavior as bestMessage. In practice, the AI consolidation rollup (which runs after all chunks) produces its own version_bump_reason that overrides the per-chunk one, so the final reason reflects the consolidated view of all changes rather than any single chunk's perspective.
Co-Authored-By: Niels Swimberghe <3382717+Swimburger@users.noreply.github.com>
Description
Refs: companion PRs needed in fiddle#656 and fern-platform#8454
When large SDK diffs are chunked for analysis, each chunk independently produces changelog entries. This leads to highly repetitive changelogs (e.g., 20+ identical "new
additionalProperty()methods" entries). This PR adds a post-chunk consolidation step that deduplicates and reformats the aggregated entries using an additional AI call.Additionally, a new
version_bump_reasonfield is added to both per-chunk analysis and consolidation responses, providing an explicit one-sentence justification for why a specific version bump (MAJOR/MINOR/PATCH) was chosen. This addresses customer feedback that the changelog doesn't clearly explain WHY a specific version was selected.Both
prDescriptionandversionBumpReasonare now fully propagated through the return type chain fromLocalTaskHandler→runGenerator→GenerationRunner→runLocalGenerationForWorkspace→PostGenerationPipeline→GithubStep→parseCommitMessageForPR(), so they appear in generated GitHub PRs.Link to Devin Session
Requested by: @Swimburger
Changes Made
Per-chunk
version_bump_reasonfielddiff_analyzer.baml: Addedversion_bump_reasonfield toAnalyzeCommitDiffResponse. Per-chunk prompt updated from "three fields" → "four fields" with guidelines per bump level. BAML client regenerated viabaml-cli generate.LocalTaskHandler.ts: TracksbestVersionBumpReasonthrough multi-chunk loop (updated when bump increases, same guard asbestMessage). Passes through toAutoVersionResult.sdkDiffCommand.ts: SamebestVersionBumpReasontracking for the CLIfern sdk-diffpath.Changelog consolidation (three-output format)
ConsolidateChangelogBAML function (diff_analyzer.baml): Takes raw aggregated entries + version bump + language, returns a three-output response:consolidated_changelog: CHANGELOG.md entry in Keep a Changelog format (### Breaking Changes, ### Added, ### Changed, ### Fixed). Bold symbol names, prose only, no code fences.pr_description: PR description with ## Breaking Changes (Before/After code fences + Migration line) and ## What's New (prose paragraphs grouped by theme). Aggressively deduplicates repetitive entries.version_bump_reason: One sentence explaining WHY the overall version bump was chosen.async_client.ts,sync_client.ts,types.ts,partial_types.ts,type_builder.ts, etc.): AddedConsolidateChangelogmethod andConsolidateChangelogResponsetype.LocalTaskHandler.ts: After multi-chunk aggregation, callsConsolidateChangelogwhen >1 changelog entry exists. Extracts all three fields. Falls back to raw-prefixed entries on failure.sdkDiffCommand.ts: Same rollup pattern for the CLIfern sdk-diffcommand path.AutoVersioningService.ts/AutoVersioningCache.ts: AddedprDescription?: stringandversionBumpReason?: stringfields toAutoVersionResultandCachedAnalysisinterfaces.Full propagation chain for PR fields
LocalTaskHandler.ts:copyGeneratedFiles()return type updated to includeautoVersioningPrDescriptionandautoVersioningVersionBumpReason. Both return statements pass through the computed fields.runGenerator.ts: Return type promise updated to include the two new fields.GenerationRunner.ts: Return type promise updated to include the two new fields.runLocalGenerationForWorkspace.ts: Destructures and passesautoVersioningPrDescriptionandautoVersioningVersionBumpReasonto PostGenerationPipeline github config.types.ts(generator-cli): AddedprDescriptionandversionBumpReasonfields toGithubStepConfiginterface.parseCommitMessage.ts: Updated to accept and useprDescription(takes priority overchangelogEntry) andversionBumpReason(prepended as bold "Version Bump:" header).GithubStep.ts: Passes new fields toparseCommitMessageForPR().Important Review Notes
prDescriptionprioritization:parseCommitMessageForPRnow usesprDescription(structured with Before/After code fences) when available, falling back tochangelogEntryonly ifprDescriptionis undefined. Verify this priority is correct.versionBumpReasonprepend format: The reason is prepended to PR body as**Version Bump:** {reason}\n\n{body}. Confirm this format is acceptable.sdkDiffCommand.tsdiscardspr_description: The CLIsdk-diffcommand path computespr_descriptionfrom the rollup but doesn't capture it, sinceAnalyzeCommitDiffResponse(the return type) doesn't have that field. This is intentional — the command analyzes diffs but doesn't create PRs. If this command is later used to feed PR creation, the response type would need extension.version_bump_reasonthat overrides the per-chunk one.sdkDiffCommand.tsalways passes"unknown"as language to the consolidation call, unlikeLocalTaskHandlerwhich uses the actual generator language.sdk-diffCLI paths. The remote generation path (Fiddle → FAI) is handled in the companion PRs.Testing
baml-cli generateHuman Review Checklist
Before merging, please verify:
prDescriptiontaking priority overchangelogEntryinparseCommitMessage.tsis the desired behaviorversionBumpReasonbold prepend format (**Version Bump:** {reason}) is acceptable for PR bodiesLocalTaskHandler→GithubSteplooks correctsdkDiffCommandintentionally not capturingpr_descriptionis acceptable (or needs follow-up)