diff --git a/.github/workflows/update_regression_baselines.yaml b/.github/workflows/update_regression_baselines.yaml new file mode 100644 index 0000000..fe83f16 --- /dev/null +++ b/.github/workflows/update_regression_baselines.yaml @@ -0,0 +1,44 @@ +name: Update Regression Baselines + +on: + workflow_dispatch: + +concurrency: + group: update-regression-baselines-${{ github.ref }} + cancel-in-progress: true + +permissions: + contents: write + +jobs: + update-baselines: + name: Update regression baselines + runs-on: ubuntu-24.04 + timeout-minutes: 30 + + steps: + - name: Checkout + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + with: + ref: ${{ github.ref }} + + - name: Set up Python + uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0 + with: + python-version: "3.14" + cache: pip + + - name: Install testing dependencies + run: | + python -m pip install --upgrade pip + python -m pip install -e .[testing] + + - name: Update regression baselines + run: pytest tests/regression --update-baselines + + - name: Commit updated baselines + uses: stefanzweifel/git-auto-commit-action@778341af668090896ca464160c2def5d1d1a3eb0 # v6 + with: + commit_message: "test(regression): update baselines" + commit_user_name: github-actions[bot] + commit_user_email: 41898282+github-actions[bot]@users.noreply.github.com diff --git a/docs/developer_guide.rst b/docs/developer_guide.rst index edf1b4b..640973a 100644 --- a/docs/developer_guide.rst +++ b/docs/developer_guide.rst @@ -60,13 +60,46 @@ Run tests with coverage:: pytest --cov CodeEntropy --cov-report=term-missing -Update regression baselines:: +Run a specific test:: + + pytest tests/unit/.../test_file.py::test_function + +Updating Regression Baselines +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Regression baselines should only be updated when a change intentionally alters +the expected numerical output of CodeEntropy. Do not update baselines simply +because a regression test fails. + +Before updating baselines, confirm that: + +- The code change is intentional and understood. +- The new output has been reviewed. +- The regression difference is expected. +- The pull request explains why the baselines changed. + +For local updates, run:: pytest tests/regression --update-baselines -Run a specific test:: +For pull requests, baselines can also be updated using the GitHub Actions +workflow **Update Regression Baselines**. - pytest tests/unit/.../test_file.py::test_function +To use it: + +1. Push your changes to your PR branch. +2. Open the **Actions** tab on GitHub. +3. Select **Update Regression Baselines**. +4. Click **Run workflow**. +5. Select your PR branch. +6. Run the workflow. + +The workflow runs the regression tests with ``--update-baselines`` and commits +any changed baseline files back to the selected branch. + +Only use this workflow when the baseline changes are intentional. If a regression +test fails unexpectedly, investigate the failure instead of updating the +baseline. Regression Test Data --------------------