Skip to content

Conversation

@drazisil-codecov
Copy link
Contributor

@drazisil-codecov drazisil-codecov commented Jan 22, 2026

Problem

The lock acquisition error message was including the lock name, which contains:

  • Lock type (e.g., bundle_analysis_processing)
  • Repo ID (e.g., 19963992)
  • Commit ID (e.g., 347f87344b5dd1f29de2ddccbf43b98e45b62093)
  • Report type (e.g., bundle_analysis)

This creates a unique error message for every repo/commit combination, causing Sentry to create separate error groups for what is essentially the same issue. This floods Sentry with thousands of "unique" errors when it's really just multiple occurrences of the same lock acquisition failure.

Example before:

Lock acquisition failed after 11 retries (max: 11). Lock: bundle_analysis_processing_lock_19963992_347f87344b5dd1f29de2ddccbf43b98e45b62093_bundle_analysis, Repo: 19963992, Commit: 347f87344b5dd1f29de2ddccbf43b98e45b62093

Example after:

Lock acquisition failed after 11 retries (max: 11). Repo: 19963992, Commit: 347f87344b5dd1f29de2ddccbf43b98e45b62093

Solution

  • Removed the lock name from the error message string
  • Lock name is still available as an exception attribute (exc.lock_name)
  • Lock name is still captured in Sentry tags for searchability (tags.lock_name)
  • Lock name is still logged in structured logging (extra.lock_name)

This allows Sentry to properly group all "lock acquisition failed" errors together while maintaining full searchability and debuggability via tags and attributes.

Test Plan

  • Existing tests pass (lock_name is still available as an attribute)
  • Monitor Sentry to confirm errors are now properly grouped
  • Verify lock_name is still searchable in Sentry tags

Note

Simplifies lock acquisition failure messaging for better grouping.

  • Updates LockRetry error string to omit lock_name when max retries are exceeded, retaining repoid and commitid

Written by Cursor Bugbot for commit 56ba2a4. This will update automatically on new commits. Configure here.

…flooding

The lock name includes repo ID, commit ID, and lock type, creating unique
error messages for each combination. This causes Sentry to create separate
error groups instead of grouping them together, making it difficult to
track the actual issue.

Changes:
- Removed lock name from the error message string in LockRetry exception
- Lock name is still available as an exception attribute (lock_name)
- Lock name is still captured in Sentry tags for searchability
- Lock name is still logged in structured logging (extra fields)

This allows Sentry to properly group all "lock acquisition failed" errors
together while still maintaining full searchability via tags.
@sentry
Copy link

sentry bot commented Jan 22, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.38%. Comparing base (32fc1bd) to head (56ba2a4).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #668   +/-   ##
=======================================
  Coverage   92.38%   92.38%           
=======================================
  Files        1295     1295           
  Lines       47688    47688           
  Branches     1592     1592           
=======================================
  Hits        44056    44056           
  Misses       3323     3323           
  Partials      309      309           
Flag Coverage Δ
workerintegration 58.04% <ø> (ø)
workerunit 89.68% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@codecov-notifications
Copy link

codecov-notifications bot commented Jan 22, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

@drazisil-codecov drazisil-codecov requested a review from a team January 22, 2026 13:55
@drazisil-codecov drazisil-codecov added this pull request to the merge queue Jan 23, 2026
Merged via the queue into main with commit 482156a Jan 23, 2026
40 checks passed
@drazisil-codecov drazisil-codecov deleted the fix-lock-acquisition-sentry-flooding branch January 23, 2026 14:26
drazisil-codecov added a commit that referenced this pull request Jan 23, 2026
…try grouping

The previous fix (PR #668) removed the lock name but left repo and commit
in the error message. Since each repo/commit combination is unique, Sentry
was still creating separate issues for each occurrence.

Now the message is generic - all identifying info remains available in:
- Exception attributes (lock_name, repoid, commitid)
- Sentry context (lock_acquisition)
- Sentry tags (lock_name, lock_type)

Fixes CCMRG-2031
drazisil-codecov added a commit that referenced this pull request Jan 23, 2026
…try grouping

The previous fix (PR #668) removed the lock name but left repo and commit
in the error message. Since each repo/commit combination is unique, Sentry
was still creating separate issues for each occurrence.

Now the message is generic - all identifying info remains available in:
- Exception attributes (lock_name, repoid, commitid)
- Sentry context (lock_acquisition)
- Sentry tags (lock_name, lock_type)

Fixes CCMRG-2031
drazisil-codecov added a commit that referenced this pull request Jan 23, 2026
…try grouping

The previous fix (PR #668) removed the lock name but left repo and commit
in the error message. Since each repo/commit combination is unique, Sentry
was still creating separate issues for each occurrence.

Now the message is generic - all identifying info remains available in:
- Exception attributes (lock_name, repoid, commitid)
- Sentry context (lock_acquisition)
- Sentry tags (lock_name, lock_type)

Fixes CCMRG-2031
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants