New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(processing) Move redis operations in reprocessing to servicewrapper #66168
Conversation
We need to shift the reprocessing workload to a new redis cluster without downtime. In order to do that I would like to use our Service wrappers and ServiceDelegator to dual write and then shift reads via runtime options. Before I can move load with ServiceDelegator the existing logic needs to be moved into a service wrapper.
import redis | ||
from django.conf import settings | ||
|
||
from sentry.utils import json |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see that you're only using dumps
, explicit importing this would be better:
from sentry.utils import json | |
from sentry.utils.json import dumps |
for primary_hash in old_primary_hashes: | ||
key = _get_old_primary_hash_subset_key(project_id, group_id, primary_hash) | ||
event_count += client.llen(key) | ||
if options.get(use_store_option): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add a comment to here?
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #66168 +/- ##
==========================================
+ Coverage 84.24% 84.26% +0.01%
==========================================
Files 5299 5304 +5
Lines 236744 236906 +162
Branches 40981 40993 +12
==========================================
+ Hits 199442 199625 +183
+ Misses 37083 37062 -21
Partials 219 219
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i think it should be safe actually to skip the option entirely, and directly cut over to the service, for as long as the redis keys stay the same
…per (#66168) We need to shift the reprocessing workload to a new redis cluster without downtime. In order to do that I would like to use our Service wrappers and ServiceDelegator to dual write and then shift reads via runtime options. Before I can move load with ServiceDelegator the existing logic needs to be moved into a service wrapper.
We need to shift the reprocessing workload to a new redis cluster without downtime. In order to do that I would like to use our Service wrappers and ServiceDelegator to dual write and then shift reads via runtime options.
Before I can move load with ServiceDelegator the existing logic needs to be moved into a service wrapper.