feat(ci): run self-hosted upgrade test by aldy505 · Pull Request #113388 · getsentry/sentry

aldy505 · 2026-04-19T14:09:17Z

To prevent InconsistentMigrationHistory in the future.

Corresponding self-hosted PR: getsentry/self-hosted#4288

kenzoengineer

overall makes sense to me, would want some eyes from @hubertdeng123 though

just had one comment

kenzoengineer · 2026-04-27T17:36:55Z

+            vroom
+            uptime_checker
+          image_url: |-
+            ghcr.io/getsentry/sentry:${{ github.sha }}-amd64


we only build images if the PR author is an OWNER or MEMBER, does this mean this step fails when external contributors make a PR?

hubertdeng123

Do we have an idea how long this will take to run in CI? I'd suspect this will take quite a while, and we'd be running tests on migrations which doesn't seem necessary.

TBH I'm pretty certain we don't need to even bring up self-hosted here. Instead, we can just checkout sentry at the latest release tag and run migrations up to latest using the dev environment

aldy505 · 2026-04-28T23:00:12Z

Do we have an idea how long this will take to run in CI? I'd suspect this will take quite a while, and we'd be running tests on migrations which doesn't seem necessary.

I would assume 15-20 mins

TBH I'm pretty certain we don't need to even bring up self-hosted here. Instead, we can just checkout sentry at the latest release tag and run migrations up to latest using the dev environment

Sounds great to me. How do we do that?

hubertdeng123 · 2026-04-28T23:05:45Z

I'd imagine it'd look something like this at the beginning, we perform migrations up to the latest tag. Then, checkout nightly and perform migrations up to nightly

- name: Get latest release tag
    id: release
    run: |
      TAG=$(gh api repos/getsentry/sentry/releases/latest --jq '.tag_name')
      echo "tag=$TAG" >> "$GITHUB_OUTPUT"

- uses: actions/checkout@sha
    with:
      ref: ${{ steps.release.outputs.tag }}
 
- name: Setup sentry env
     uses: ./.github/actions/setup-sentry
       with:
         mode: migrations

- name: Apply migrations
    run: |
      sentry upgrade --noinput

sentry · 2026-04-30T14:31:22Z

+    runs-on: ubuntu-24.04
+    timeout-minutes: 30
+    needs: did-migration-change
+    if: ${{ needs.did-migration-change.outputs.modified == 'true' || needs.did-migration-change.outputs.added == 'true' }}


Bug: The upgrade-test CI job will not run for modified migration files because the migrations_modified filter is not defined in .github/file-filters.yml.
_{Severity: MEDIUM}

Suggested Fix

Define the migrations_modified filter in .github/file-filters.yml using the modified: event qualifier, similar to how migrations_added is defined. This will ensure the dorny/paths-filter action correctly detects modified migration files and populates the migrations_modified output, allowing the upgrade-test job to trigger as intended.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not valid. Location: .github/workflows/migrations.yml#L84 Potential issue: The `upgrade-test` job's conditional logic is intended to trigger for both added and modified migration files. However, the check for modified files, `needs.did-migration-change.outputs.modified == 'true'`, will never evaluate to true. This is because the underlying output `steps.changes.outputs.migrations_modified` from the `dorny/paths-filter` action is always empty, as the `migrations_modified` filter key is not defined in `.github/file-filters.yml`. Consequently, the `upgrade-test` job is silently skipped for all modified migration files, potentially allowing breaking changes to be merged without testing.

cursor · 2026-04-30T14:36:16Z

+    runs-on: ubuntu-24.04
+    timeout-minutes: 30
+    needs: did-migration-change
+    if: ${{ needs.did-migration-change.outputs.modified == 'true' || needs.did-migration-change.outputs.added == 'true' }}


Upgrade test never triggers for modified migrations

Medium Severity

The modified == 'true' part of this condition will never be true because the migrations_modified filter is not defined in .github/file-filters.yml — only migrations_added exists. The needs.did-migration-change.outputs.modified value will always be an empty string, meaning this upgrade test only runs for newly added migrations, not for modified ones. Since the stated goal is to prevent InconsistentMigrationHistory, and modifying an existing migration's dependencies is one way to cause that error, this is a coverage gap.

^{Reviewed by Cursor Bugbot for commit d12be93. Configure here.}

sentry · 2026-05-03T13:39:16Z

+        run: |
+          sentry upgrade --noinput


Bug: The upgrade-test job may fail with a ModuleNotFoundError because it runs new migrations against a stale Python environment that lacks new dependencies from the PR.
_{Severity: MEDIUM}

Suggested Fix

After checking out the PR's code with clean: false, re-synchronize the Python virtual environment using the PR's uv.lock file. Add a step to run uv sync --frozen --active to install any new dependencies before the sentry upgrade command is executed.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not valid. Location: .github/workflows/migrations.yml#L107-L108 Potential issue: In the `upgrade-test` job, the Python virtual environment is created based on an old release tag's dependencies. The workflow then checks out the pull request's code but preserves the old, stale environment. When `sentry upgrade --noinput` is executed, it runs new migration files from the PR. If a new migration imports a module from a newly added external Python dependency, the command will fail with a `ModuleNotFoundError` because the required dependency is not present in the virtual environment.

@hubertdeng123 is this true?

Yes, this is true since you're checking out the new branch but not installing new dependencies

hubertdeng123 · 2026-05-05T16:52:21Z

Do you have a successful run of this in CI? I'd like to see one to verify that this is working as intended

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit afac2a7. Configure here.}

cursor · 2026-05-15T02:02:03Z

+          clean: false
+      - name: Apply migrations again to test upgrade path
+        run: |
+          sentry upgrade --noinput


Missing dependency reinstall before second migration run

High Severity

After checking out the current ref (with clean: false), the upgrade-test job runs sentry upgrade --noinput without reinstalling Python dependencies. The setup-sentry action (which runs uv sync --frozen --active and fast_editable) is only executed against the release tag's uv.lock. If the current branch introduces any new dependencies, Django will fail to load apps/models with an ImportError, producing a false CI failure unrelated to migration consistency. Unlike the existing sql job which only runs git diff after checkout, this job needs the full sentry environment functional for the second sentry upgrade call.

^{Reviewed by Cursor Bugbot for commit afac2a7. Configure here.}

aldy505 · 2026-05-17T09:09:05Z

Do you have a successful run of this in CI? I'd like to see one to verify that this is working as intended

@hubertdeng123 Nope, do you have any idea on how to test this?

feat(ci): run self-hosted upgrade test

bd84e1c

aldy505 requested a review from a team as a code owner April 19, 2026 14:09

sentry Bot reviewed Apr 19, 2026

View reviewed changes

Comment thread .github/workflows/self-hosted.yml Outdated

cursor Bot reviewed Apr 19, 2026

View reviewed changes

Comment thread .github/workflows/self-hosted.yml Outdated

Comment thread .github/workflows/self-hosted.yml Outdated

aldy505 added 2 commits April 23, 2026 10:48

Merge branch 'master' into feat/self-hosted-upgrade-test

8e7b684

fix(ci): correct checkout order and needs value

16d82d4

sentry Bot reviewed Apr 23, 2026

View reviewed changes

Comment thread .github/workflows/self-hosted.yml Outdated

fix(ci): remove inconsistent criteria for checkout

019a423

aldy505 added the Trigger: getsentry tests Once code is reviewed: apply label to PR to trigger getsentry tests label Apr 23, 2026

cursor Bot reviewed Apr 23, 2026

View reviewed changes

Comment thread .github/workflows/self-hosted.yml Outdated

Dav1dde added Trigger: getsentry tests Once code is reviewed: apply label to PR to trigger getsentry tests and removed Trigger: getsentry tests Once code is reviewed: apply label to PR to trigger getsentry tests labels Apr 27, 2026

vercel Bot deployed to Preview April 27, 2026 07:53 View deployment

kenzoengineer reviewed Apr 27, 2026

View reviewed changes

hubertdeng123 requested changes Apr 28, 2026

View reviewed changes

aldy505 mentioned this pull request Apr 30, 2026

Release 26.5.0 getsentry/self-hosted#4287

Closed

18 tasks

feat(ci): run migration for sentry only

d12be93

aldy505 requested a review from hubertdeng123 April 30, 2026 14:28

github-actions Bot removed the Trigger: getsentry tests Once code is reviewed: apply label to PR to trigger getsentry tests label Apr 30, 2026

aldy505 added the Trigger: getsentry tests Once code is reviewed: apply label to PR to trigger getsentry tests label Apr 30, 2026

sentry Bot reviewed Apr 30, 2026

View reviewed changes

cursor Bot reviewed Apr 30, 2026

View reviewed changes

fix(ci): try to checkout dirty

6d302b7

github-actions Bot removed the Trigger: getsentry tests Once code is reviewed: apply label to PR to trigger getsentry tests label May 3, 2026

sentry Bot reviewed May 3, 2026

View reviewed changes

aldy505 added the Trigger: getsentry tests Once code is reviewed: apply label to PR to trigger getsentry tests label May 5, 2026

Merge branch 'master' into feat/self-hosted-upgrade-test

afac2a7

github-actions Bot removed the Trigger: getsentry tests Once code is reviewed: apply label to PR to trigger getsentry tests label May 15, 2026

cursor Bot reviewed May 15, 2026

View reviewed changes

Uh oh!

Conversation

aldy505 commented Apr 19, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kenzoengineer left a comment

Choose a reason for hiding this comment

Uh oh!

kenzoengineer Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

hubertdeng123 left a comment

Choose a reason for hiding this comment

Uh oh!

aldy505 commented Apr 28, 2026

Uh oh!

hubertdeng123 commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sentry Bot Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cursor Bot Apr 30, 2026

Choose a reason for hiding this comment

Upgrade test never triggers for modified migrations

Uh oh!

sentry Bot May 3, 2026

Choose a reason for hiding this comment

Uh oh!

aldy505 May 5, 2026

Choose a reason for hiding this comment

Uh oh!

hubertdeng123 May 5, 2026

Choose a reason for hiding this comment

Uh oh!

hubertdeng123 commented May 5, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot May 15, 2026

Choose a reason for hiding this comment

Missing dependency reinstall before second migration run

Uh oh!

aldy505 commented May 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

hubertdeng123 commented Apr 28, 2026 •

edited

Loading