feat: Add cron job to schedule indexer job#106377
Conversation
github-actions
bot
added
the
Scope: Backend
| }, | ||
| "seer-explorer-index": { | ||
| "task": "seer:sentry.tasks.seer_explorer_index.schedule_explorer_index", | ||
| "schedule": task_crontab("0", "*/1", "*", "*", "*"), |
There was a problem hiding this comment.
Running it every hour, just to make sure the task gets picked up in case a deploy or scheduler restart causes a missed window. Storing last_run in cache so it's only run every 24 hours.
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| if last_run and last_run > django_timezone.now() - EXPLORER_INDEX_RUN_FREQUENCY: | ||
| return | ||
|
|
||
| cache.set(LAST_RUN_CACHE_KEY, django_timezone.now(), LAST_RUN_CACHE_TIMEOUT) |
There was a problem hiding this comment.
Cache set before work completes blocks retries for 24 hours
Medium Severity
The cache.set() call at line 67 stores the last run timestamp before the actual work is performed (lines 75-76 where the generator is consumed). If any exception occurs during dispatch_explorer_index_projects() or get_seer_explorer_enabled_projects(), the cache entry is already set. Subsequent hourly runs will hit the early-return check at lines 63-65 and skip execution for ~24 hours, defeating the stated goal of running hourly "to make sure the task gets picked up in case a deploy or scheduler restart causes a missed window."
Additional Locations (1)
roaga
left a comment
roaga
left a comment
There was a problem hiding this comment.
makes sense to me. are we making sure that we're not running both this and the cron job defined in seer?
I've added an option to make sure this doesn't just start running when merged. I'll kill the seer job once I confirm this works! |
2 participants
Adds a periodic task to schedule Seer Explorer indexing
for projects in organizations with the
seer-explorer-indexfeature flag enabled. The cron tab is set for every hour,
just to make sure the task gets picked up in case a deploy or
scheduler restart causes a missed window.
Storing last_run in cache so it's only run every 24 hours.
This main task batches projects and for every batch of 100
spawns a task that calls seer's
/v1/automation/explorer/indexendpoint with computed delays (same logic as statistical detectors).