Skip to content

Conversation

@srest2021
Copy link
Member

@srest2021 srest2021 commented Nov 4, 2025

fixes REPLAY-803

When detecting regressions via compare_version we are comparing the build codes. However, we are not ordering by build code or number in any other instances of semver ordering, which leads to inconsistent behavior.

For example, when determining the "next" release to resolve an issue by, we get the greatest semver version with an arbitrary build code--say the greatest semver in our project is 2.0.0 (10) and we pick 2.0.0 (8) arbitrarily. Say we later see that issue appearing in 2.0.0 (9). We would mark it as a regression because 2.0.0 (9) > 2.0.0 (8) according to compare_version, even though the greatest semver in our project was 2.0.0 (10) and it should not be marked as a regression.

In this PR we add build number and code to the semver ordering. We mimic the behavior of compare_version by ordering on build code and build number (aka numeric build code). This will make our semver ordering consistent when viewing sorted releases lists, resolving issues, detecting regressions, and sending alerts about regressions.

Build code ordering in compare_version, DESC (largest to smallest):

  1. alphanumeric builds (compared lexicographically)
  2. numeric builds (compared numerically)
  3. no build

For more information on the behavior of compare_version when detecting regressions, see this comment.

Notes:

  • Skipped updating the semver ordering in organization_trace_item_attributes.py as we later strip the build code anyway.
  • Skipped in tagstore/snuba/backend.py for the same reason.
  • Skipped for semver filters in the issues search. Filters are not generally concerned with ordering. We would only become a tiny bit concerned if the query returned more than 1000 releases and many of those releases have the same version and different build numbers, which is unlikely. And since we did not update the semver index with build code, we may even break index usage like here.
  • We also leave the tag values in issues search alone as described here, remaining consistent with the semver filters.

Before

Sorting by semver sorts the version but not the build number, which stays in the order in which it was created.

Screen.Recording.2025-11-11.at.10.55.50.AM.mov

Resolving in the next release, where the greatest semver release is 2.0.0 (10), instead arbitrarily picks 2.0.0 (8). You can also see that "The current semver release" in the dropdown has arbitrarily picked 2.0.0 (9).

Screen.Recording.2025-11-11.at.11.08.48.AM.mov

Searching for "release is latest" in Issues returns a group associated with 2.0.0 (9), not 2.0.0 (10).

Screen.Recording.2025-11-11.at.11.26.33.AM.mov

After

Sorting by semver now sorts the build numbers as well -- alphanumeric builds on top, then numeric builds, then no build.

Screen.Recording.2025-11-11.at.10.56.57.AM.mov

Resolving in the next release, where the greatest semver release is 2.0.0 (10), now correctly picks 2.0.0 (10). "The current semver release" in the dropdown also correctly references 2.0.0 (10).

Screen.Recording.2025-11-11.at.11.10.54.AM.mov

Searching for "release is latest" in Issues correctly returns a group associated with 2.0.0 (10).

Screen.Recording.2025-11-11.at.11.27.25.AM.mov

@linear
Copy link

linear bot commented Nov 4, 2025

@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Nov 4, 2025
@codecov
Copy link

codecov bot commented Nov 4, 2025

Codecov Report

❌ Patch coverage is 88.46154% with 6 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/sentry/api/endpoints/organization_releases.py 50.00% 5 Missing ⚠️
src/sentry/models/release.py 92.30% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           master   #102711      +/-   ##
===========================================
+ Coverage   80.02%    80.66%   +0.64%     
===========================================
  Files        9239      9241       +2     
  Lines      393532    393840     +308     
  Branches    25051     25051              
===========================================
+ Hits       314933    317703    +2770     
+ Misses      78149     75687    -2462     
  Partials      450       450              

@srest2021 srest2021 changed the title fix(releases): Sort by build number in semver fix(releases): Sort by build number & code in semver Nov 5, 2025
def test_release_list_order_by_semver(self) -> None:
self.login_as(user=self.user)
release_1 = self.create_release(version="test@2.2")
release_2 = self.create_release(version="test@10.0+122")
Copy link
Member Author

@srest2021 srest2021 Nov 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switch the order in which release_2 and release_7 are created so the build_number sorting doesn't coincidentally pass due to insertion order.

@srest2021 srest2021 force-pushed the srest2021/REPLAY-803 branch from 771af9f to 2ac040b Compare November 5, 2025 18:30
@srest2021 srest2021 changed the title fix(releases): Sort by build number & code in semver fix(releases): Consistently sort by build number & code in semver Nov 5, 2025
class ReleaseOrderBySemverTestCase(TestCase):
def test_compare_version_sorts_by_build_number(self):
"""
Test that compare_version correctly sorts releases by build number with various formats.
Copy link
Member Author

@srest2021 srest2021 Nov 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need to actually merge this but just thought I'd include it for a better picture of how compare_version actually works when detecting regressions. Here we can see that numeric builds are compared numerically and adding any letters makes the comparison lexicographic. We also see that alphanumeric builds are deemed "greater" than numeric builds, and any build is "greater" than no build.

release_1, # test@2.2
release_3, # test@2.2-alpha
release_9, # random_junk
release_8, # test@some_thing
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This order now matches what we see in test_compare_version_sorts_by_build_number below

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants