Skip to content

feat(release): include a categorized PR changelog in orb-v release notes#3429

Merged
JSONbored merged 4 commits into
mainfrom
feat/orb-release-changelog
Jul 5, 2026
Merged

feat(release): include a categorized PR changelog in orb-v release notes#3429
JSONbored merged 4 commits into
mainfrom
feat/orb-release-changelog

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • Answers: why doesn't the orb-v0.2.0 release list all the changes the way metagraphed's releases do? Because .github/workflows/release-selfhost.yml's "GitHub Release" step deliberately dropped --generate-notes after the very first orb release (orb-v0.1.0-beta.1) hit GitHub's 125000-character release-body limit — with no prior orb-v tag to bound the diff, the auto-generated notes spanned the entire repo history.
  • Every release from orb-v0.1.0 onward now has a real previous tag, so this scopes the changelog explicitly to the range since the immediately preceding orb-v tag — found by actual tag-creation order (git tag --sort=-creatordate), not a semver sort (which would mis-order a prerelease against its own later stable release, e.g. 0.1.0-beta.2 vs 0.1.0) and not the Release Notes API's own "previous release" auto-detection (this repo's release list also carries unrelated mcp-v* releases on an independent cadence that could get picked up by mistake once the two schemes' timestamps interleave).
  • Adds .github/release.yml (mirroring metagraphed's own .github/release.yml) so the generated changelog groups PRs into 🚀 Features / 🐛 Fixes / 🧹 Other Changes using the gittensor:feature / gittensor:bug labels gittensory's own review engine already applies automatically to every merged PR (confirmed: 40/40 of the last 40 merged PRs already carry one) — no separate manual-labeling process needed, unlike metagraphed's interim label convention.
  • Keeps a safety net for the original failure mode: if the combined notes would still exceed GitHub's limit, or the generate-notes API call itself fails, falls back to the plain pull-command notes (with a compare-view link in the size-limit case) instead of ever blocking the release.
  • Verified locally against the real repo: orb-v0.1.0orb-v0.2.0 generates an ~11KB changelog (69 PRs), comfortably under the limit.

Scope

  • The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked an issue, or this is small enough that the summary explains why an issue is not needed. (Direct infra fix, prompted by a question about why orb-v0.2.0's release notes look sparse; no separate issue.)

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage locally; codecov/patch requires ≥97% coverage of the lines AND branches you changed (aim for 98%+ on your diff so CI variance does not fail near the threshold). Global coverage is a non-blocking trend with a loose 90% backstop, not the gate.
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries

If any required check was skipped, explain why:

  • This PR only touches .github/workflows/release-selfhost.yml, the new .github/release.yml, and a new backend-only test file — no src/**, UI, MCP, or migration changes, so Codecov's src/**-scoped patch gate and the UI/MCP/workers checks don't apply to this diff. Added test/unit/release-selfhost-notes.test.ts (5 tests) that extracts and executes the actual committed bash from the "GitHub Release" step against mocked git/gh binaries, covering: normal changelog inclusion, the no-prior-tag first-release case, the size-limit fallback, create-vs-edit routing, and API-failure fallback. Proved causality by reverting to the pre-fix step and confirming the 2 changelog-specific tests fail against it (the other 3 legitimately passed either way, since they cover create/edit routing and failure-safety behavior that was already correct). Also ran the 3 existing release-selfhost*/selfhost-sentry-release test files (18 tests) to confirm no regression, all green.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests. (N/A.)
  • API/OpenAPI/MCP behavior is updated and tested where needed. (N/A — CI/release tooling only, no product API surface.)
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks. (N/A.)
  • Visible UI changes include a UI Evidence section below with JPG/JPEG or PNG screenshots. (N/A — no UI change.)
  • Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs. (N/A — no CHANGELOG.md touched; this only affects future GitHub Release bodies.)

UI Evidence

N/A — no UI/frontend/docs/extension change.

Notes

  • This touches the release workflow (a guarded CI path), so it may be held for manual review/merge rather than auto-merging even though CI is green.
  • Once merged, I can backfill the already-published orb-v0.2.0 (and optionally orb-v0.1.0) release notes via gh release edit using the same generate-notes call, so the current releases get the changelog immediately instead of waiting for the next tag.

@superagent-security

Copy link
Copy Markdown

Superagent didn't find any vulnerabilities or security issues in this PR.

@gittensory-orb gittensory-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.5x multiplier. label Jul 5, 2026
@gittensory-orb

gittensory-orb Bot commented Jul 5, 2026

Copy link
Copy Markdown

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-05 07:28:03 UTC

3 files · 1 AI reviewer · no blockers · readiness 93/100 · CI pending · blocked

⏸️ Suggested Action - Manual Review

  • Touches a guarded path — held for manual review: This PR changes guardrail-protected path(s): .github/workflows/release-selfhost.yml (matched .github/workflows/**).

Review summary
This change restores orb release changelogs without reintroducing the original unbounded-history failure: the workflow explicitly selects the previous orb tag, calls GitHub's generate-notes API with that bounded range, and falls back to the existing operator notes on API failure or oversized output. The added release.yml categorization matches the stated label scheme, and the test harness drives the actual committed workflow step rather than a copied shell fragment. I do not see a reachable release-blocking defect in the visible diff.

Nits — 6 non-blocking
  • nit: .github/workflows/release-selfhost.yml:356 uses a hardcoded 120000-character cutoff; that is intentionally below GitHub's 125000 limit, but it would be clearer to name it once as `MAX_RELEASE_NOTES_CHARS=120000` near the fallback logic instead of embedding the threshold in the conditional and warning text.
  • nit: test/unit/release-selfhost-notes.test.ts:145 only asserts that the generated notes include the changelog, not that the plain notes and changelog are separated by the intended blank line; a regression to cramped release-body formatting would slip through.
  • nit: test/unit/release-selfhost-notes.test.ts:178 exercises the oversize fallback with an ASCII-only body, while the real release notes can include emoji and non-ASCII author/title text; if the API limit is byte-oriented rather than JS/bash character-oriented, this test does not cover that edge.
  • At .github/workflows/release-selfhost.yml:356, extract the release-body guard into `MAX_RELEASE_NOTES_CHARS=120000` and use it in both the comparison and warning text so future threshold changes cannot drift.
  • At test/unit/release-selfhost-notes.test.ts:145, add an assertion for the exact separator between `$NOTES` and `CHANGELOG`, for example checking that the pull-command block precedes `\n\n## What's Changed`.
  • Touches a guarded path — held for manual review — A maintainer must review and merge this change.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 56 registered-repo PR(s), 46 merged, 416 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 56 PR(s), 416 issue(s).
Gate result ⚠️ Not blocking Advisory; not blocking this PR.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository registration is not available in the local Gittensory cache.
  • Public profile languages: not available
  • Official Gittensor activity: 56 PR(s), 416 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • No action.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@codecov

codecov Bot commented Jul 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.99%. Comparing base (f6b9872) to head (142e8e7).
⚠️ Report is 5 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3429   +/-   ##
=======================================
  Coverage   92.99%   92.99%           
=======================================
  Files         295      295           
  Lines       30970    30970           
  Branches    11293    11293           
=======================================
  Hits        28801    28801           
  Misses       1514     1514           
  Partials      655      655           
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@gittensory-orb gittensory-orb Bot added the manual-review Gittensor contributor context label Jul 5, 2026
@JSONbored JSONbored force-pushed the feat/orb-release-changelog branch from 1832ff2 to e2f7bf6 Compare July 5, 2026 07:00
@JSONbored

Copy link
Copy Markdown
Owner Author

Addressed the review's nits: reworded the tag-order comment to not overclaim precision for lightweight tags sharing a commit (documented the actual, low-impact consequence instead), and added a run-log warning when the generate-notes API call itself fails so it's distinguishable from a genuinely empty PR range. Both changes covered by an updated/new assertion in release-selfhost-notes.test.ts, causality-proven by reverting and re-testing.

@JSONbored

Copy link
Copy Markdown
Owner Author

Addressed the second round of nits: .github/release.yml's exclude.authors used bare bot names (dependabot, github-actions), but I confirmed against this repo's own PR history that GitHub's actual login is [bot]-suffixed (dependabot[bot]) — the bare names never matched anything, so the exclusion was a silent no-op. Fixed, and added renovate[bot] (17 merged PRs here, same dependency-bump noise) while deliberately leaving sentry[bot]/gittensory-orb[bot] unexcluded since they produce real, correctly-labeled changes (e.g. #3305). Also strengthened the oversized-changelog fallback test to assert the pull command survives, not just that the oversized filler is gone.

JSONbored added 4 commits July 5, 2026 00:25
The GitHub Release step dropped `--generate-notes` outright after the very
first orb release hit GitHub's 125000-character release-body limit -- with
no prior orb-v tag to bound the diff, the generated notes spanned the
entire repo history. Every release from orb-v0.1.0 onward has a real
previous tag, so scope the changelog explicitly to the range since the
immediately preceding orb-v tag (found by tag-creation order, not a semver
sort that would mis-order a prerelease against its own later stable
release, and not the Release Notes API's own "previous release"
auto-detection, since this repo's release list also carries unrelated
mcp-v* releases on an independent cadence).

Add .github/release.yml so the generated changelog groups PRs into
Features/Fixes/Other Changes using the gittensor:feature/gittensor:bug
labels gittensory's own review engine already applies to every merged PR
-- no separate manual-labeling process needed.

Keep a safety net for the original failure mode: if the combined notes
would still exceed GitHub's limit, or the generate-notes call itself
fails, fall back to the plain pull-command notes (optionally with a
compare-view link) instead of ever blocking the release.
…ailure

Review nits on the changelog PR: the "actual tag-creation order" comment
overclaimed precision --sort=-creatordate can't guarantee for two
lightweight orb-v tags pointing at the same commit (e.g. promoting an -rc
straight to stable with no new commits); reword it as the best available
local tag-date ordering and document that the only consequence of a tie
is an emptier-than-expected changelog for that one release, never an
incorrect or blocked one. Also warn in the run log when the generate-notes
API call itself fails, so an operator can tell that apart from a
genuinely empty PR range between two tags.
The exclude.authors list used bare names (dependabot, github-actions),
but GitHub's changelog-generation API matches the literal API login,
which for a bot account includes the [bot] suffix -- confirmed against
this repo's own PR history, where dependabot's login is dependabot[bot].
The bare names never matched anything, so the exclusion was a no-op.

Also add renovate[bot] (17 merged PRs here, the same routine-dependency-
bump noise dependabot produces) to the list. Deliberately leave
sentry[bot] and gittensory-orb[bot] unexcluded: both produce real,
gittensor:bug/feature-labeled changes worth surfacing (e.g. #3305, a
Seer-authored fix for a real production error), not noise.

Also strengthen the oversized-changelog fallback test to assert the
operator-critical pull command survives alongside the compare-link
fallback, not just that the oversized filler text is gone.
The comment said "the two pure dependency-bump bots" but the list has
three entries since renovate[bot] was added alongside dependabot[bot];
reword to explain github-actions[bot]'s inclusion (no PRs here today, but
the same operator-facing noise if that changes) instead of miscounting.

Also tighten the changelog test to assert tag_name and previous_tag_name
appear on the SAME generate-notes call, not just as two substrings
present somewhere in the overall command log.
@JSONbored JSONbored force-pushed the feat/orb-release-changelog branch from 5146f0d to 142e8e7 Compare July 5, 2026 07:27
@JSONbored

Copy link
Copy Markdown
Owner Author

Third and hopefully final round: fixed the stale comment count (said "two" bots, list had three since renovate[bot] was added last round) and tightened the changelog test to assert tag_name/previous_tag_name are parameters of the SAME generate-notes call rather than just two substrings present somewhere in the log. Skipped the one explicitly-optional suggestion (a same-commit lightweight-tag tie test) since it's already documented as a low-impact non-goal and rc-to-stable-on-the-same-commit isn't a workflow this repo currently uses.

@JSONbored JSONbored merged commit be744ba into main Jul 5, 2026
10 checks passed
@JSONbored JSONbored deleted the feat/orb-release-changelog branch July 5, 2026 07:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.5x multiplier. manual-review Gittensor contributor context

Development

Successfully merging this pull request may close these issues.

1 participant