Add release process improvement

Changelog.md

@@ -25,6 +25,7 @@
 - Fixed filter Canvas Test Student from roster sync (#7926)
 
 ### 🔧 Internal changes
+- Add release automation scripts (#7914)
 - Added seed task to assign TAs to A1 groupings and criteria (#7867)
 - Updated autotest seed files to ensure settings follow tester JSON schema (#7775)
 - Refactored grade entry form helper logic into `GradeEntryFormsController` and removed the newly-unused helper file. (#7789)

release/RELEASING.md

@@ -0,0 +1,210 @@
+# Releasing MarkUs
+
+Step-by-step guide for cutting a MarkUs minor release. Helper scripts in this directory automate the tedious parts — each step shows the manual command and the script alternative.
+
+## Prerequisites
+
+- `gh` CLI authenticated (`gh auth status`)
+- Docker running (`docker compose up`)
+- A GitHub milestone exists for the target version with all relevant PRs merged and tagged
+- Clean working tree
+
+## Phase 1: Setup
+
+```bash
+git fetch origin
+git checkout release && git pull origin release
+git checkout -b v2.X.Y   # branch from release, not master
+```
+
+**Verify:** `git log --oneline -1` matches the latest release branch commit.
+
+## Phase 2: Recon — discover what to cherry-pick
+
+```bash
+RECON=$(ruby release/recon.rb v2.X.Y)
+echo "$RECON" | ruby release/recon-format.rb --summary
+echo "$RECON" | ruby release/recon-format.rb --plan
+```
+
+This queries the milestone, checks which PRs are already on the release branch, resolves file-overlap dependencies, and outputs a JSON plan. The `$RECON` variable is reused in later phases (release notes, PR body).
+
+Review the plan. Note any non-PR commits (direct pushes, fork merges) — decide whether to include or skip.
+
+## Phase 3: Cherry-pick
+
+**Automated (recommended):**
+
+```bash
+release/cherry-pick.sh v2.X.Y
+```
+
+This cherry-picks all milestone PRs in dependency order, auto-resolves Changelog conflicts, skips empty commits, and verifies each pick for contamination. It stops on code conflicts or contamination and tells you exactly what to do.
+
+After fixing a problem, resume from where it stopped:
+```bash
+release/cherry-pick.sh v2.X.Y --resume
+```
+
+At the end it prints the PR list and the `changelog.rb` command to run next.
+
+<details>
+<summary>Manual alternative</summary>
+
+For each PR in the order from recon:
+
+```bash
+git cherry-pick -m1 <merge_commit_hash>
+ruby release/verify.rb <PR_NUMBER>
+```
+
+Conflict handling:
+- **Changelog.md only:** `git checkout --ours Changelog.md && git add Changelog.md && GIT_EDITOR=true git cherry-pick --continue`
+- **Code files:** Stop. Resolve by comparing against `gh pr diff <N>`.
+- **Empty commit:** Already on release. `git cherry-pick --skip`.
+</details>
+
+## Phase 4: Rebuild the Changelog
+
+The Changelog is always corrupted after cherry-picks. Rebuild it:
+
+```bash
+ruby release/changelog.rb --mode=release --version=v2.X.Y --prs=7783,7851,7858
+```
+
+Pass the comma-separated list of cherry-picked PR numbers. The script reads `origin/release` and `origin/master`, filters master's unreleased entries to only the included PRs, and outputs a clean Changelog.
+
+```bash
+ruby release/changelog.rb --mode=release --version=v2.X.Y --prs=<PR_LIST> > Changelog.md
+```
+
+**Validate:**
+```bash
+bash release/validate_changelog.sh v2.X.Y
+```
+
+All 6 checks should pass: no conflict markers, empty unreleased, version section exists with entries, correct ordering, no duplicate PRs, older sections unchanged.
+
+## Phase 5: Version bump and commit
+
+```bash
+echo "VERSION=v2.X.Y,PATCH_LEVEL=DEV" > app/MARKUS_VERSION
+git add Changelog.md app/MARKUS_VERSION
+git commit -m "v2.X.Y"
+```
+
+`PATCH_LEVEL=DEV` is a legacy field — always keep it as-is.
+
+## Phase 6: Test
+
+```bash
+docker compose exec rails bundle exec rspec
+docker compose exec rails npx jest --no-coverage
+```
+
+Pre-existing failures on the release branch are expected. Verify no NEW failures were introduced by the cherry-picks.
+
+## Phase 7: Dependency and settings check
+
+```bash
+git diff origin/release -- Gemfile Gemfile.lock package.json package-lock.json
+git diff origin/release -- markus.control config/settings.yml
+git diff origin/release --name-only -- db/migrate/
+```
+
+If any of these show changes, notify sysadmins before deployment. They may need to `bundle install`, `npm install`, apply new settings to `settings.local.yml`, or run migrations.
+
+## Phase 8: Push and PR
+
+```bash
+git push -u origin v2.X.Y
+gh pr create --base release --title "v2.X.Y" --body "Release v2.X.Y"
+```
+
+Wait for CI. Get reviewer approval. **Merge with "Create a merge commit"** (never squash into release).
+
+## Phase 9: GitHub Release
+
+After the PR is merged:
+
+```bash
+# Re-run if your shell session expired since Phase 2
+RECON=$(ruby release/recon.rb v2.X.Y)
+gh release create v2.X.Y --repo MarkUsProject/Markus --target release --title "v2.X.Y" --notes "$(echo "$RECON" | ruby release/recon-format.rb --release-notes)"
+```
+
+Or create manually via GitHub UI: Releases > Create > tag `v2.X.Y`, target `release`.
+
+## Phase 10: Milestone management
+
+```bash
+# Close released milestone
+MILESTONE_ID=$(gh api repos/MarkUsProject/Markus/milestones --jq ".[] | select(.title==\"v2.X.Y\") | .number")
+gh api -X PATCH "repos/MarkUsProject/Markus/milestones/$MILESTONE_ID" -f state=closed
+
+# Create next milestone
+gh api repos/MarkUsProject/Markus/milestones -f title="v2.X.Z"
+```
+
+## Phase 11: Sync Changelog to master
+
+Move released entries from `[unreleased]` into a new version section on master:
+
+```bash
+git checkout master && git pull origin master
+git checkout -b v2.X.Y-changelog
+
+ruby release/changelog.rb --mode=master-sync --version=v2.X.Y --prs=<PR_LIST> > Changelog.md
+bash release/validate_changelog_master.sh v2.X.Y
+
+git add Changelog.md
+git commit -m "Update changelog with new release v2.X.Y [ci skip]"
+git push -u origin v2.X.Y-changelog
+gh pr create --base master --title "Update changelog for v2.X.Y" --body "Sync released entries."
+```
+
+Squash-merge is fine here (same branch lineage, `[ci skip]` skips CI).
+
+## Phase 12: Satellite repos (Wiki, Autotester)
+
+Check each repo's milestone for PRs. If any exist, follow the same cherry-pick + PR + release flow. If none, still create a GitHub release with the version tag.
+
+## Phase 13: Cleanup
+
+Delete version branches:
+```bash
+git push origin --delete v2.X.Y v2.X.Y-changelog
+git branch -d v2.X.Y v2.X.Y-changelog
+```
+
+---
+
+## Helper Scripts Reference
+
+| Script | What it does |
+|--------|-------------|
+| `cherry-pick.sh <version>` | Automated cherry-pick loop with verification. `--resume` to continue after fixing a conflict |
+| `recon.rb <version>` | Queries milestone, resolves cherry-pick order, outputs JSON |
+| `recon-format.rb --flag` | Formats recon JSON (pipe from stdin). Flags: `--summary`, `--plan`, `--order`, `--pr-list`, `--pr-body`, `--release-notes`, `--skipped` |
+| `verify.rb <pr_number>` | Compares last cherry-pick diff against original PR. Exit 0 = clean, 1 = contaminated |
+| `changelog.rb --mode=MODE --version=V --prs=N,N` | Rebuilds Changelog. Modes: `release` (for release branch), `master-sync` (for master) |
+| `validate_changelog.sh <version>` | 6-check validation for release branch changelog |
+| `validate_changelog_master.sh <version>` | 5-check validation for master changelog sync |
+
+All scripts accept `--help` for usage details.
+
+---
+
+## Pitfalls
+
+| Problem | Solution |
+|---------|----------|
+| Changelog corrupted after cherry-picks | Always rebuild with `changelog.rb`. Never trust the auto-merged result. |
+| Cherry-pick pulls in extra code | Git 3-way merge can import dependency PR code. Always run `verify.rb` after each pick. |
+| Empty cherry-pick commit | PR was in a prior release. Check changelog, `git cherry-pick --skip`. |
+| `PATCH_LEVEL=RELEASE` | Wrong. Always `PATCH_LEVEL=DEV`. Legacy field, unused at runtime. |
+| API returns 404 | Include `/csc108` prefix. Use `MarkUsAuth` not `Bearer`. |
+| Jest flag | `--testPathPatterns` (plural), not singular. |
+| Rails runner `!` escaping | Pipe via stdin: `echo '...' | docker compose exec -T rails bundle exec rails runner -` |
+| Squash-merge into release | Never. Use "Create a merge commit" to preserve commit history. |
+| Copying release Changelog to master | Never overwrite. Use `--mode=master-sync` to move entries from unreleased. |

release/changelog.rb

@@ -0,0 +1,157 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# Changelog Rebuild — Produces a clean Changelog.md after cherry-picks.
+#
+# Usage:
+#   ruby release/changelog.rb --mode=release --version=v2.9.6 --prs=7783,7851,7858
+#   ruby release/changelog.rb --mode=master-sync --version=v2.9.6 --prs=7783,7851,7858
+#   ruby release/changelog.rb --help
+#
+# Modes:
+#   release      — Empty [unreleased] + new version section + old sections from origin/release
+#   master-sync  — Move cherry-picked entries from [unreleased] to new version section on master
+
+require_relative 'common'
+
+CATEGORIES = [
+  "### \u{1F6E1}\u{FE0F} Security",
+  "### \u{1F6A8} Breaking changes",
+  "### \u{2728} New features and improvements",
+  "### \u{1F41B} Bug fixes",
+  "### \u{1F527} Internal changes"
+].freeze
+
+def parse_version_header(line)
+  return unless line =~ /^## \[(unreleased|v[\d.]+)\]/i
+
+  Regexp.last_match(1).downcase == 'unreleased' ? 'unreleased' : Regexp.last_match(1)
+end
+
+def process_changelog_line(line, result, current)
+  ver = parse_version_header(line)
+  return init_version_section(result, current, ver) if ver
+  return init_category(result, current, line) if line.start_with?('### ') && current[:ver]
+
+  append_entry(result, current, line) if line.start_with?('- ') && current[:ver] && current[:cat]
+end
+
+def init_version_section(result, current, ver)
+  current[:ver] = ver
+  result[:order] << ver
+  result[:sections][ver] = {}
+  current[:cat] = nil
+end
+
+def init_category(result, current, line)
+  current[:cat] = line
+  result[:sections][current[:ver]][line] ||= []
+end
+
+def append_entry(result, current, line)
+  result[:sections][current[:ver]][current[:cat]] << line
+end
+
+def warn_unknown_categories(sections)
+  unknown = sections.values.flat_map(&:keys).uniq - CATEGORIES
+  unknown.each { |c| warn "Warning: unknown category '#{c}' — entries may be dropped" } if unknown.any?
+end
+
+# Parses Changelog.md into { "sections" => { version => { category => [entries] } }, "version_order" => [...] }
+def parse_changelog(text)
+  result = { sections: {}, order: [] }
+  current = { ver: nil, cat: nil }
+  text.each_line { |line| process_changelog_line(line.rstrip, result, current) }
+  warn_unknown_categories(result[:sections])
+  { 'sections' => result[:sections], 'version_order' => result[:order] }
+end
+
+def emit_categories(out, entries_by_cat, skip_empty: false)
+  CATEGORIES.each do |cat|
+    entries = entries_by_cat[cat] || []
+    next if skip_empty && entries.empty?
+
+    out << cat
+    entries.each { |e| out << e }
+    out << ''
+  end
+end
+
+def emit_old_sections_verbatim(out, raw_text, skip:)
+  emitting = false
+  raw_text.each_line do |line|
+    line = line.chomp
+    if line =~ /^## \[(unreleased|v[\d.]+)\]/i
+      ver = Regexp.last_match(1).downcase == 'unreleased' ? 'unreleased' : Regexp.last_match(1)
+      emitting = skip.exclude?(ver)
+    end
+    out << line if emitting
+  end
+end
+
+def partition_entries(unreleased, pr_list)
+  matched = {}
+  unmatched = {}
+  unreleased.each do |cat, entries|
+    matched[cat], unmatched[cat] = entries.partition { |e| pr_list.any? { |n| e.match?(/\##{n}(?!\d)/) } }
+  end
+  [matched, unmatched]
+end
+
+def build_changelog_sections(mode, version, matched, unmatched)
+  out = ['# Changelog', '', '## [unreleased]', '']
+  emit_categories(out, mode == 'release' ? {} : unmatched, skip_empty: false)
+  out << "## [#{version}]"
+  out << ''
+  emit_categories(out, matched, skip_empty: true)
+  out
+end
+
+def build_changelog(mode, version, pr_list)
+  release_raw = ReleaseHelpers.run('git', 'show', 'origin/release:Changelog.md')
+  master_raw = ReleaseHelpers.run('git', 'show', 'origin/master:Changelog.md')
+  unreleased = parse_changelog(master_raw)['sections']['unreleased'] || {}
+  matched, unmatched = partition_entries(unreleased, pr_list)
+
+  out = build_changelog_sections(mode, version, matched, unmatched)
+  source_raw = mode == 'release' ? release_raw : master_raw
+  emit_old_sections_verbatim(out, source_raw, skip: ['unreleased', version])
+  "#{out.join("\n")}\n"
+end
+
+# --- Main ---
+
+if ARGV.empty? || ARGV.intersect?(['-h', '--help'])
+  warn <<~HELP
+    Usage: ruby release/changelog.rb --mode=MODE --version=VERSION --prs=N,N,N
+
+    Modes:
+      release      Build changelog for release branch (empty unreleased + new version)
+      master-sync  Build changelog for master (move entries from unreleased to version)
+
+    Options:
+      --mode=MODE        release or master-sync (required)
+      --version=VERSION  Target version, e.g. v2.9.6 (required)
+      --prs=N,N,N        Comma-separated PR numbers (required)
+  HELP
+  exit(ARGV.empty? ? 1 : 0)
+end
+
+args = {}
+ARGV.each { |a| args[Regexp.last_match(1)] = Regexp.last_match(2) if a =~ /^--(\w[\w-]*)=(.+)$/ }
+
+mode = args['mode']
+version = args['version']
+pr_list = (args['prs'] || '').split(',').map(&:strip).reject(&:empty?)
+
+unless mode && version && pr_list.any?
+  warn 'Error: --mode, --version, and --prs are all required. Run with --help.'
+  exit 1
+end
+unless %w[release master-sync].include?(mode)
+  warn "Error: --mode must be 'release' or 'master-sync', got '#{mode}'"
+  exit 1
+end
+ReleaseHelpers.validate_version!(version)
+
+puts build_changelog(mode, version, pr_list)