Cron monitor shows "invalid duration" error when no duration is sent in check-in

[Kobby-Bawuah]

Environment

SaaS (https://sentry.io/)

Steps to Reproduce

1. Create a cron monitor with a schedule (e.g., every 10 minutes)
2. Send heartbeat check-ins using the simple URL format:
   GET https://{ORG}.ingest.us.sentry.io/api/{PROJECT_ID}/cron/{MONITOR_SLUG}/{DSN_KEY}/?status=ok
3. Do NOT include any duration parameter in the request
4. Observe the monitor details page after some time

See Linear thread for Zendesk ticket.

Expected Result

Check-ins should be recorded successfully without duration (since none was sent). No ingestion errors should appear.

Actual Result

The monitor shows an ingestion error:
"Errors were encountered while ingesting check-ins for this monitor"

• "Invalid duration"
• "A check-in was sent with an invalid duration of '6839405000'"

The duration column shows "—" for affected check-ins. User reports this has occurred on two separate monitors and was never seen before.

Product Area

Crons

Link

No response

DSN

No response

Version

No response

[linear]

ENG-6174

[getsantry]

Auto-routing to @getsentry/product-owners-crons for triage ⏲️

[evanpurkhiser]

If we're seeing the error A check-in was sent with an invalid duration that must mean we're trying to update an existing check-in.

Here's where that's happening

sentry/src/sentry/monitors/consumers/monitor_consumer.py

Lines 388 to 391 in ca2c094

	duration_error: CheckinInvalidDuration = {
	"type": ProcessingErrorType.CHECKIN_INVALID_DURATION,
	"duration": str(updated_duration),
	}

which is inside this function

sentry/src/sentry/monitors/consumers/monitor_consumer.py

Line 294 in ca2c094

def update_existing_check_in(

The way the logic works for this, when sending an OK without a explicit check-in ID, we'll try and lookup an existing in_progress check-in, and then compute the duration as the time of that in_progress subtracted from the timestamp of the processing ok chec-in.

Here's where that happens

sentry/src/sentry/monitors/consumers/monitor_consumer.py

Lines 376 to 385 in ca2c094

	# Only compute a new duration if there isn't already an existing duration
	# set, and a duration hasn't been explicitly provided in this update.
	if updated_duration is None and existing_check_in.duration is None:
	# We use abs here because in some cases we might end up having checkins arrive
	# slightly out of order due to race conditions in relay. In cases like this,
	# we're happy to just assume that the duration is the absolute difference between
	# the two dates.
	updated_duration = abs(
	int((start_time - existing_check_in.date_added).total_seconds() * 1000)
	)

6839405000 ms would be about 80 days ago.. this does seems strange though if this is happening out of nowhere.

[mbrunelli]

I saw the issue again this morning. I'm basically just trying to utilize this as a heartbeat monitor. I don't care about duration or making sure it completes after being marked as started; only that it checked in.

Is there a different way I should be handling this, rather than sending status=ok each time the job completes?

[mbrunelli]

I could always send status=in_progress, wait a second, and then send status=ok, but that feels a little bit hacky.

[evanpurkhiser]

No sending status=ok should absolutely be fine. Could you email me your org slug my.fullname@sentry.io (with period between first and last). I'd like to take a closer look at your check-ins.

Thanks @mbrunelli