Cron monitors intermittently timing out as incomplete when the job runs successfully

[mlazar-endear]

How do you use Sentry?

Sentry Saas (sentry.io)

Version

2.50.0

Steps to Reproduce

I think I am seeing the same issue as these unresolved tickets

• Missing in_progress checkin with Cron Monitoring #3279
• Monitor intermitten failures on long running tasks #2422

I have a django project and I'm using huey to schedule and run nightly jobs using Huey.periodic_task. I have wrapped my task functions with sentry_sdk.monitor() and I'm also separately tracking the task status in my own database using huey's pre-execute and post-execute signals.

I have several dozen cron monitors instrumented this way. I am observing that the tasks that take a long-ish time to run are intermittently timing out as "incomplete" in sentry even though I can see the task has finished successfully on my end, and the runtime is within the max runtime configured for the monitor.

Here's an example of a monitor with a 120 minute runtime

versus my own logging

You can see on the 28th and 30th the task ran over the max runtime threshold and was correctly flagged as a timeout after many hours. But on days where the duration is missing in sentry, the task is incorrectly marked as a timeout.

Here's another example of a task running in a different environment. This monitor also has a max runtime of 120 minutes. In this case, the runtimes on the 26th and 27 were short (<15 minutes) but were still marked as a timeout in sentry.

Expected Result

The cron monitor status and duration should be consistently reported in Sentry.

Actual Result

Cron monitors are intermittently timing out as incomplete.

[linear]

PY-2068