fix(data-imports): use dateReceived as the incremental field for Sentry event endpoints

[Gilbert09]

Problem

Every incremental sync of a Sentry issue_events or project_events table fails with IncrementalFieldMissingFromDataError, permanently pausing the schema. Confirmed in error tracking.

Both endpoints fetch full event bodies (full=true), which carry a dateReceived timestamp. dateCreated was the only incremental field ever offered for these two endpoints, so the configured cursor never matched a real column and every sync failed at cursor extraction.

Changes

• Swap the issue_events and project_events endpoint configs to use dateReceived as the incremental field (and matching partition_key), the field Sentry's full event serializer actually returns.
• Add a data migration correcting already-configured schemas whose stored incremental_field is the broken "dateCreated" value, so existing paused schemas pick up the fix without manual reconfiguration.

How did you test this code?

• Added test_events_endpoints_default_to_date_received, parameterized over both endpoints: guards against the default incremental field regressing to dateCreated.
• Added a migration test (mirroring the existing test_migration_0075 pattern) covering the new data migration: fixes both affected schemas, is idempotent, and leaves unrelated schema names and source types untouched.
• Ran the full test_sentry.py suite (133 tests, all pass).
• Applied the new migration against a local dev database; makemigrations --check reports no model drift.
• Not run: a live sync against the real Sentry API (no test credentials in this environment).

Automatic notifications

• Publish to changelog?

Docs update

None — internal source config only, no documented behavior changes.

🤖 Agent context

Autonomy: Fully autonomous

Investigated via PostHog error tracking (issue 019fd673-7755-7962-9f4c-0a8f8ab9070b), reading the stack trace down to get_incremental_field_value in pipelines/common/load.py, then tracing the schema/pipeline-version properties on the exception event back to the Sentry issue_events source config. Confirmed the actual Sentry API field name via docs before changing the default. Skills invoked: /django-migrations, /writing-tests, /writing-pr-descriptions.

[trunk-io]

😎 Merged successfully - details.

[github-actions]

Hey @Gilbert09! 👋

It looks like your git author email on this PR isn't your @posthog.com address (owerstom@gmail.com). Since you're on the PostHog team, it's worth pointing your local git author email at your @posthog.com address. Why it matters:

• Consistent work identity in git history — internal tooling that attributes commits to team members keys off your @posthog.com address.
• Keeps team contributions easy to tell apart from external community ones when scanning history.

You can fix it for this repo with:

git config user.email "you@posthog.com"

Or set it globally with git config --global user.email "you@posthog.com". No need to redo this PR — just a nudge for next time. 🙂

[github-actions]

🤖 CI report

✅ Backend coverage — all changed backend lines covered

🧪 Backend test coverage

Patch coverage — changed backend lines (products + core): ████████████████████ 100.0% (39 / 39)

All changed backend lines are covered ✅

Per-product line coverage (touched products)

Product	Coverage	Lines
demo	███████████░░░░░░░░░ 56.3%	1,497 / 2,661
tasks	██████████████░░░░░░ 70.7%	34,687 / 49,088
signals	████████████████░░░░ 82.5%	27,025 / 32,777
cdp	█████████████████░░░ 84.2%	3,914 / 4,646
data_modeling	█████████████████░░░ 85.9%	7,959 / 9,263
notebooks	█████████████████░░░ 86.4%	8,026 / 9,293
actions	█████████████████░░░ 86.6%	717 / 828
managed_warehouse	█████████████████░░░ 87.1%	6,219 / 7,141
cohorts	██████████████████░░ 87.6%	6,482 / 7,400
data_warehouse	██████████████████░░ 87.8%	11,281 / 12,843
product_tours	██████████████████░░ 87.9%	1,303 / 1,482
exports	██████████████████░░ 89.2%	7,429 / 8,328
engineering_analytics	██████████████████░░ 89.2%	6,705 / 7,516
dashboards	██████████████████░░ 89.4%	5,983 / 6,693
alerts	██████████████████░░ 90.3%	4,482 / 4,966
conversations	██████████████████░░ 90.4%	18,026 / 19,932
streamlit_apps	██████████████████░░ 90.7%	2,630 / 2,901
error_tracking	██████████████████░░ 91.1%	11,153 / 12,243
canvas	██████████████████░░ 91.2%	2,043 / 2,241
stamphog	██████████████████░░ 91.3%	4,505 / 4,936
slack_app	██████████████████░░ 91.6%	10,519 / 11,487
marketing_analytics	██████████████████░░ 92.1%	13,388 / 14,539
product_analytics	███████████████████░ 92.5%	5,849 / 6,321
mcp_analytics	███████████████████░ 92.7%	3,810 / 4,112
early_access_features	███████████████████░ 92.9%	1,340 / 1,443
ai_observability	███████████████████░ 92.9%	17,086 / 18,399
web_analytics	███████████████████░ 93.1%	15,963 / 17,154
surveys	███████████████████░ 93.2%	5,858 / 6,286
posthog_ai	███████████████████░ 93.3%	1,327 / 1,423
reminders	███████████████████░ 93.4%	468 / 501
approvals	███████████████████░ 93.5%	3,491 / 3,734
workflows	███████████████████░ 94.2%	7,807 / 8,287
endpoints	███████████████████░ 94.3%	8,771 / 9,306
review_hog	███████████████████░ 94.6%	8,246 / 8,715
skills	███████████████████░ 94.8%	3,478 / 3,669
logs	███████████████████░ 95.6%	10,759 / 11,259
experiments	███████████████████░ 95.7%	27,503 / 28,753
annotations	███████████████████░ 96.2%	732 / 761
revenue_analytics	███████████████████░ 96.3%	1,887 / 1,960
feature_flags	███████████████████░ 96.4%	17,539 / 18,195
replay_vision	███████████████████░ 96.5%	17,135 / 17,762
user_interviews	███████████████████░ 96.5%	2,638 / 2,734
customer_analytics	███████████████████░ 97.1%	10,748 / 11,070
warehouse_sources	███████████████████░ 97.4%	381,099 / 391,263
data_catalog	████████████████████ 97.9%	2,650 / 2,708
pulse	████████████████████ 98.4%	2,017 / 2,049

Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.

⚠️ Django migration SQL — 1 new migration to review

We've detected new migrations on this PR. Review the SQL output for each migration:

products/warehouse_sources/backend/migrations/0120_fix_sentry_events_incremental_field.py

BEGIN;
--
-- Raw Python operation
--
-- THIS OPERATION CANNOT BE WRITTEN AS SQL
COMMIT;

Last updated: 2026-08-06 12:40 UTC (9171605)

✅ Django migration risk — migration analysis complete

We've analyzed your migrations for potential risks.

Summary: 0 Safe | 1 Needs Review | 0 Blocked

⚠️ Needs Review

May have performance impact

warehouse_sources.0120_fix_sentry_events_incremental_field
  └─ #1 ⚠️ RunPython: RunPython data migration needs review for performance

📚 How to Deploy These Changes Safely

RunPython:

Use batching for large data migrations:

• Use .iterator() to avoid loading all rows into memory
• Use .bulk_update() instead of saving individual objects
• Batch size: 1,000-10,000 rows per batch
• Add pauses between batches
• Consider background jobs for very large updates (millions of rows)

See the migration safety guide

Last updated: 2026-08-06 12:41 UTC (9171605)

[trunk-io]

   

Failed Test	Failure Summary	Logs
property-definition-update tool	The API request was denied because the action does not support personal API key access.	Logs ↗︎

View Full Report ↗︎ ⋅ Docs

[stamphog]

Note

🤖 stamphog reviewed 9171605d359da389ae4ca639821b9cd6dfae8153 — verdict: REFUSED

Gates denied this PR (data migration touches the deny-list and it's classified T2-never), so it requires human sign-off regardless of code quality; the change also includes a data migration correcting existing records, which is risky territory needing explicit review.

• 👍 on the PR from hex-security-app[bot].
• Gates denied: PR includes a data migration (deny-listed) and is classified T2-never, both of which require human review before merge.
• Data migrations that rewrite existing schema config values are risky territory — needs a human (e.g. @PostHog/team-warehouse-sources) to confirm the migration is safe and idempotent in production.

Gate mechanics and policy version

Gate		Result
prerequisites	✓	all clear
deny-list	✗	matches: migrations
size	✓	62L, 2F substantive, 130L/5F incl. docs/generated/snapshots — within ceiling
tier	✗	classified as T2-never: T2-never (130L, 5F, single-area, fix)
stamphog 2.0.0b4		.stamphog/policy.yml @ 64bcdbd · reviewed head 9171605

Updated in place — this replaces 2 earlier stamphog review(s) on this PR.

[fuziontech]

The endpoint configuration and targeted data migration are narrowly scoped, idempotent, and covered by focused tests. CI reports the Django and Python checks passing.

Non-blocking follow-up: consider updating the issue_events and project_events canonical column descriptions to include dateReceived, so the corrected cursor column receives the expected catalog description.

This review was generated by an automated review agent on behalf of @fuziontech.

[talyn-app]

/trunk merge

[deployment-status-posthog]

Deploy status

Environment	Status	Deployed At	Workflow
dev	✅ Deployed	2026-08-07 11:45 UTC	Run
prod-us	✅ Deployed	2026-08-07 12:01 UTC	Run
prod-eu	✅ Deployed	2026-08-07 12:07 UTC	Run