Skip to content

test(ci): Reproducible test harness for release notes pipeline #35016

@sfreudenthaler

Description

@sfreudenthaler

Summary

A checked-in bash script that creates a complete end-to-end test harness in any GitHub repo (primarily dotCMS/core-workflow-test) for validating the AI release notes pipeline. Includes cleanup support.

Acceptance Criteria

  • Creates 5 PRs with specific labels covering all signal types:
    • Feature (feature label)
    • Bug fix (bug label)
    • Rollback-unsafe (AI: Not Safe To Rollback label)
    • Skip (Changelog: Skip label)
    • Infrastructure (infrastructure label)
  • Creates 2 standard releases (v26.03.17-01 after PRs 1-3, v26.03.17-02 after PRs 4-5)
  • Creates filter test tags (v26.03.17_lts_v01, dotcms-cli-26.03.17-01)
  • Ensures required labels exist (idempotent label creation)
  • --cleanup flag tears down releases, tags, and branches
  • --repo flag for targeting any repo (defaults to current)
  • Prints expected behavior summary at the end

Technical Details

File: .github/scripts/gather-release-data/test/create-test-harness.sh

Usage:

# Create test harness
.github/scripts/gather-release-data/test/create-test-harness.sh --repo dotCMS/core-workflow-test

# Tear down
.github/scripts/gather-release-data/test/create-test-harness.sh --cleanup --repo dotCMS/core-workflow-test

Expected test results:

Test Expected
v26.03.17-01 → v26.03.17-02 2 commits, 1 change (infra), 1 skipped, 0 rollback-unsafe
v26.02.10-99 → v26.03.17-01 3+ PRs (feature, bug, rollback-unsafe), 1 rollback-unsafe
Single-tag v26.03.17-02 Auto-detects v26.03.17-01 as previous
Backfill with v26.03.17_lts_v01 Workflow skips (LTS tag filter)
Backfill with dotcms-cli-26.03.17-01 Workflow skips (CLI tag filter)
Re-trigger with notes present Skips (idempotency check)

Status: ✅ Implemented — test harness already created in dotCMS/core-workflow-test

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions