Skip to content

[Feature]: Automate regression baseline updates via GitHub Actions #320

@harryswift01

Description

@harryswift01

Feature Request

Problem / Motivation

Regression testing baselines currently have to be updated locally. This can introduce noisy PR diffs when baseline files include path-dependent changes from an individual developer’s machine or workspace layout.

Proposed Solution

Add a GitHub Action workflow that can update regression testing baselines in a consistent CI environment.

The workflow should:

  • Run on demand via workflow_dispatch
  • Check out the target branch or PR branch
  • Execute the regression baseline update command
  • Commit the updated baselines back to the branch, or optionally upload them as an artifact
  • Use a stable CI path/workspace so generated baseline output is consistent across developers

Alternatives Considered

  • Continue updating baselines locally, accepting occasional noisy path-related diffs
  • Document a standardized local workflow or containerized command for updating baselines
  • Provide a script that normalizes file paths before baselines are committed

Expected Impact

  • Reduces noisy PR diffs caused by developer-specific local paths
  • Makes baseline updates easier and more reproducible
  • Improves consistency of regression test maintenance
  • Lowers friction for contributors who need to update baselines

Additional Context

This would be especially useful for PRs where regression output changes are expected, but the baseline update itself should remain deterministic and easy to review.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions