feat(data-warehouse): implement drata import source

[Gilbert09]

Problem

Drata was a scaffolded warehouse source with no sync logic. Teams using Drata for compliance automation (SOC 2, ISO 27001, HIPAA) want their controls, monitoring-test results, evidence, personnel and device compliance, vendors, policies, risks, and audit-trail events in the warehouse so they can trend audit readiness and blend GRC posture with the rest of their data.

Why: part of implementing the eng/support source batch scaffolded in #71137 — this fills in the Drata stub so the connector actually syncs and is visible to users.

Changes

Implements the Drata source end to end as a ResumableSource, following the standard source.py / settings.py / drata.py split:

• Auth and regions: Bearer API key plus a region select (North America / Europe / Asia-Pacific) mapped to Drata's regional API hosts. All outbound HTTP goes through make_tracked_session() with the key redacted.
• 14 tables, declared in settings.py: account-level inventories (workspaces, users, personnel, devices, assets, vendors, policies), the events audit trail, workspace-scoped fan-out tables (controls, monitoring_tests, evidence_library, frameworks, each carrying an injected workspaceId in a composite primary key), and risk tables (risk_registers, risks) which are deselected by default because they 403 without Drata's Risk Management Pro feature.
• Pagination: v2 cursor pagination (cursor/size, explicit sort=createdAt&sortDir=ASC), with a guard that terminates if the cursor stops advancing. Resume state (cursor + fan-out parent bookmark) is saved after each yielded batch, so Temporal can resume mid-sync without re-walking earlier pages or parents.
• Incremental sync on events only — the one endpoint with a documented server-side timestamp filter (createdAtStartDate). Everything else has no since-filter in the v2 API, so those tables ship full refresh rather than a fake client-side "incremental".
• Partitioning on the stable createdAt field (monthly), sort_mode="desc" on the incremental endpoint so the watermark only commits after a complete sync.
• Non-retryable errors for invalid/expired keys, missing scopes, and unaccepted API terms (412); 403 at source-create is accepted since custom-scoped keys may legitimately lack the probe endpoint's scope. Curated canonical_descriptions.py from the official API docs; static table catalog exposed for public docs; SOURCES.md updated.

The source ships visible with releaseStatus=ALPHA (the scaffold's unreleasedSource=True is removed). The icon already existed at frontend/public/services/drata.png.

Note

Endpoint paths, params, pagination, and response shapes were verified against Drata's current public API v2 reference (developers.drata.com). Live curl verification wasn't possible without an API key — Drata returns 401 on every path before route resolution — so two behaviors are declared conservatively and noted in code comments: the createdAtStartDate filter on events is documented but not live-tested, and response ordering couldn't be confirmed, so the incremental endpoint uses sort_mode="desc" (watermark commits only at successful job end, correct regardless of arrival order).

How did you test this code?

89 unit tests across the two standard modules, all passing, plus the registry-wide source suites (1816 passed; the two pre-existing test_stripe_config failures also fail on the base branch with this change stashed). ruff check / ruff format and hogli ci:preflight --fix are clean. No live-credential testing was possible in this environment.

• tests/test_drata.py (transport): catches regressions in cursor termination (including a non-advancing cursor, which would otherwise loop forever), resume-from-saved-cursor without re-fetching page one, fan-out parent bookmarks (resume skips completed workspaces, saved cursor applies only to the bookmarked parent, deleted parents fall back to a full walk), workspaceId/riskRegisterId injection that keeps composite keys unique table-wide, the events server-side filter on every page, region-host routing, retryable-vs-permanent status mapping, URL query scrubbing in error messages, and credential-validation mapping (401 invalid, 403 accepted at create, 412 terms).
• tests/test_drata_source.py (source class): catches drift in the schema catalog and incremental flags, the risk tables' default-off state, the placeholder-config docs path, per-region non-retryable error matching, config-to-transport plumbing (including nulling the watermark when incremental is off), and the resume-manager binding.

Automatic notifications

• Publish to changelog?
• Alert Sales and Marketing teams?

Docs update

The user-facing doc below follows the documenting-warehouse-sources template and needs to land in the posthog.com repo as contents/docs/cdp/sources/drata.md (no posthog.com checkout was available here, so audit_source_docs couldn't be run). docsUrl already points at https://posthog.com/docs/cdp/sources/drata.

posthog.com doc: contents/docs/cdp/sources/drata.md

---
title: Linking Drata as a source
sidebar: Docs
showTitle: true
availability: { free: full, selfServe: full, enterprise: full }
sourceId: Drata
beta: true
---

import SourceSetupIntro from "../_snippets/source-setup-intro.mdx"
import SyncModes from "../_snippets/sync-modes.mdx"
import TroubleshootingLink from "../_snippets/dw-troubleshooting-link.mdx"
import AlphaRelease from "../_snippets/alpha-release.mdx"

<AlphaRelease />

The Drata connector syncs your [Drata](https://drata.com/) compliance data, including controls, monitoring tests, evidence, personnel and device compliance, vendors, policies, risks, and audit-trail events, into the PostHog Data warehouse. Use it to trend audit readiness and analyze your compliance posture alongside the rest of your data.

## Prerequisites

- A Drata account with permission to create API keys.
- Accepted Drata API terms and conditions (Drata prompts for this the first time the API is used).
- The `risk_registers` and `risks` tables require Drata's Risk Management Pro feature.

## Adding a data source

<SourceSetupIntro />

The connector authenticates with a Drata API key:

1. In Drata, go to **Settings** and open **API keys**, then create a new API key.
2. Grant the key **read** scopes for the data you want to sync (or the "All read" scope). Write scopes are not needed.
3. Copy the key when it is shown. Drata only displays it once.
4. Pick the **region** that matches your Drata account: North America, Europe, or Asia-Pacific.

If your key uses IP allowlisting, create a key without an allowlist for this connection.

## Sync modes

<SyncModes />

The `events` table supports incremental sync on `createdAt`. The other tables are compliance inventories with no server-side change filter, so they sync as full refresh.

## Configuration

<SourceParameters />

## Supported tables

<SourceTables />

Tables scoped to a workspace in Drata (`controls`, `monitoring_tests`, `evidence_library`, `frameworks`) sync across every workspace, with a `workspaceId` column identifying the workspace each row belongs to. Rows in `risks` carry a `riskRegisterId` column identifying their risk register.

## Troubleshooting

- **Invalid Drata API key**: the key is wrong, expired, or was revoked. Create a new key under Settings and API keys in Drata, then update the source.
- **Permission errors (403)**: grant the API key the matching read scope for the table you're syncing. The `risk_registers` and `risks` tables also require the Risk Management Pro feature.
- **"You must accept the Drata terms and conditions"**: open Drata and accept the API terms, then retry the sync.

<TroubleshootingLink />

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

• Built with PostHog Code (Claude Code). Skills invoked: /implementing-warehouse-sources, /writing-tests, /documenting-warehouse-sources.
• Endpoint selection cross-referenced the Drata v2 API reference against the streams other connectors ship. Workspace-scoped resources use a fan-out over /workspaces with a stable parent-id resume bookmark (klaviyo's pattern) rather than asking the user for a workspace id, since most accounts have more than one and the list is cheap.
• Incremental sync is limited to events on purpose: the v2 API has no updated-since filter, and a sorted-DESC client-side walk was rejected in favor of honest full refresh for the inventory tables, which are small.
• Regenerating generated_configs.py produced unrelated formatting churn in other sources' configs, so the file was re-formatted with ruff and only the Drata hunk kept.

---

Created with PostHog Code

[github-actions]

Hey @Gilbert09! 👋

It looks like your git author email on this PR isn't your @posthog.com address (owerstom@gmail.com). Since you're on the PostHog team, it's worth pointing your local git author email at your @posthog.com address. Why it matters:

• Consistent work identity in git history — internal tooling that attributes commits to team members keys off your @posthog.com address.
• Keeps team contributions easy to tell apart from external community ones when scanning history.

You can fix it for this repo with:

git config user.email "you@posthog.com"

Or set it globally with git config --global user.email "you@posthog.com". No need to redo this PR — just a nudge for next time. 🙂

[github-actions]

🤖 CI report

✅ Bundle size — no change

Uncompressed size of every built .js bundle, compared against the base branch.

Total: 64.77 MiB · no change

No file changed by more than 1000 B.

_{Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report}

✅ Eager graph — within budget

How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.

Root	Eager (shipped)	Δ vs base	Budget
entry (logged-out pages, app bootstrap) src/index.tsx	1.22 MiB · 22 files	no change	███░░░░░░░ 28.4% of 4.29 MiB
authenticated shell (every logged-in page) src/scenes/AuthenticatedShell.tsx	8.13 MiB · 2,978 files	no change	█████████░ 87.9% of 9.25 MiB

🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx

Largest files eagerly shipped from src/index.tsx

Size	File
126.8 KiB	../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
24.6 KiB	../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js
6.3 KiB	../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js
4.5 KiB	../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js
3.9 KiB	../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js
1.4 KiB	../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js
1.3 KiB	src/RootErrorBoundary.tsx
912 B	../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js
789 B	src/scenes/ChunkLoadErrorBoundary.tsx
762 B	src/index.tsx

Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx

Size	File
281.3 KiB	../node_modules/.pnpm/posthog-js@1.402.2/node_modules/posthog-js/dist/rrweb.js
267.7 KiB	../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
235.5 KiB	src/taxonomy/core-filter-definitions-by-group.json
222.9 KiB	../node_modules/.pnpm/posthog-js@1.402.2/node_modules/posthog-js/dist/module.js
164.0 KiB	src/queries/validators.js
154.3 KiB	../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
126.8 KiB	../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
105.8 KiB	src/lib/api.ts
93.3 KiB	../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
92.7 KiB	../packages/quill/packages/quill/dist/index.js

_{Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479}

✅ Dist folder size — no change

Total size of the built frontend/dist folder (all assets), compared against the base branch.

Total: 1312.40 MiB · no change

ℹ️ MCP UI apps size — 31 app(s), 16561.5 KB JS

Built size of each MCP UI app (main.js + styles.css).

App	JS	CSS
debug	598.2 KB	179.2 KB
action	456.5 KB	179.2 KB
action-list	563.0 KB	179.2 KB
cohort	455.4 KB	179.2 KB
cohort-list	562.0 KB	179.2 KB
email-template	455.3 KB	179.2 KB
error-details	471.1 KB	179.2 KB
error-issue	456.1 KB	179.2 KB
error-issue-list	562.9 KB	179.2 KB
experiment	560.1 KB	179.2 KB
experiment-list	563.8 KB	179.2 KB
experiment-results	561.8 KB	179.2 KB
feature-flag	565.8 KB	179.2 KB
feature-flag-list	569.5 KB	179.2 KB
feature-flag-testing	459.4 KB	179.2 KB
insight-actors	560.8 KB	179.2 KB
invite-email-preview	454.7 KB	179.2 KB
llm-costs	558.1 KB	179.2 KB
session-recording	457.2 KB	179.2 KB
session-summary	462.5 KB	179.2 KB
survey	457.0 KB	179.2 KB
survey-global-stats	560.9 KB	179.2 KB
survey-list	563.7 KB	179.2 KB
survey-stats	560.8 KB	179.2 KB
trace-span	455.8 KB	179.2 KB
trace-span-list	562.9 KB	179.2 KB
workflow	455.8 KB	179.2 KB
workflow-list	562.4 KB	179.2 KB
query-results	743.7 KB	179.2 KB
render-ui	824.0 KB	179.2 KB
visual-review-snapshots	460.3 KB	179.2 KB

⚠️ Playwright — 1 failed

🎭 Playwright report · View test results →

❌ 1 failed test:

• view results, pause, resume, and ship (chromium)

These issues are not necessarily caused by your changes.
Annoyed by this section? Help fix flakies and failures and it will go green!

⚠️ Backend coverage — 98.0% of changed backend lines covered — 8 uncovered

🧪 Backend test coverage

Patch coverage — changed backend lines (products + core): ████████████████████ 98.0% (551 / 559)

File	Patch	Uncovered changed lines
products/warehouse_sources/backend/temporal/data_imports/sources/drata/drata.py	96.8%	56, 68, 82, 129, 135
products/warehouse_sources/backend/temporal/data_imports/sources/drata/tests/test_drata.py	98.8%	81, 200, 274

🤖 Agents: add a test covering the lines above, or note why under "How did you test this code?". Machine-readable gap list: the patch-coverage artifact on this run (gh run download 29450830846 -n patch-coverage), or the coverage-data block at the end of this comment.

Per-product line coverage (touched products)

Product	Coverage	Lines
platform_features	██░░░░░░░░░░░░░░░░░░ 12.1%	7 / 58
batch_exports	████████░░░░░░░░░░░░ 39.7%	8,416 / 21,220
demo	███████████░░░░░░░░░ 56.2%	1,497 / 2,663
warehouse_sources_queue	████████████░░░░░░░░ 59.2%	148 / 250
tasks	█████████████░░░░░░░ 67.4%	25,560 / 37,895
data_tools	██████████████░░░░░░ 70.0%	63 / 90
ai_gateway	███████████████░░░░░ 75.0%	9 / 12
signals	████████████████░░░░ 79.1%	19,033 / 24,073
data_modeling	████████████████░░░░ 80.0%	4,834 / 6,045
cdp	████████████████░░░░ 80.7%	3,118 / 3,864
wizard	████████████████░░░░ 82.5%	772 / 936
notebooks	█████████████████░░░ 84.3%	6,343 / 7,520
agent_platform	█████████████████░░░ 84.7%	3,273 / 3,862
cohorts	█████████████████░░░ 86.1%	4,022 / 4,671
actions	█████████████████░░░ 86.6%	717 / 828
product_tours	█████████████████░░░ 87.5%	1,266 / 1,447
exports	██████████████████░░ 88.3%	6,891 / 7,800
visual_review	██████████████████░░ 88.5%	5,565 / 6,287
business_knowledge	██████████████████░░ 88.5%	4,400 / 4,969
conversations	██████████████████░░ 88.9%	16,129 / 18,133
dashboards	██████████████████░░ 89.1%	5,719 / 6,418
mcp_analytics	██████████████████░░ 89.1%	2,502 / 2,807
error_tracking	██████████████████░░ 89.6%	9,718 / 10,852
alerts	██████████████████░░ 89.9%	3,638 / 4,046
engineering_analytics	██████████████████░░ 90.1%	5,105 / 5,665
streamlit_apps	██████████████████░░ 90.4%	2,499 / 2,764
slack_app	██████████████████░░ 90.6%	9,511 / 10,503
links	██████████████████░░ 90.6%	183 / 202
marketing_analytics	██████████████████░░ 90.8%	11,514 / 12,684
product_analytics	██████████████████░░ 91.1%	5,599 / 6,143
mcp_store	██████████████████░░ 91.8%	3,668 / 3,995
managed_migrations	██████████████████░░ 91.9%	908 / 988
data_warehouse	██████████████████░░ 92.1%	17,946 / 19,489
workflows	██████████████████░░ 92.4%	5,148 / 5,574
notifications	███████████████████░ 92.7%	1,026 / 1,107
web_analytics	███████████████████░ 92.7%	13,624 / 14,691
ai_observability	███████████████████░ 92.8%	14,868 / 16,019
surveys	███████████████████░ 92.9%	5,660 / 6,094
posthog_ai	███████████████████░ 93.2%	1,322 / 1,418
tracing	███████████████████░ 93.3%	2,435 / 2,611
approvals	███████████████████░ 93.3%	3,395 / 3,640
reminders	███████████████████░ 93.4%	468 / 501
early_access_features	███████████████████░ 93.8%	848 / 904
legal_documents	███████████████████░ 94.1%	1,568 / 1,667
endpoints	███████████████████░ 94.1%	8,606 / 9,143
skills	███████████████████░ 94.4%	2,827 / 2,995
revenue_analytics	███████████████████░ 94.5%	3,598 / 3,809
messaging	███████████████████░ 94.5%	2,530 / 2,677
review_hog	███████████████████░ 94.6%	6,529 / 6,902
logs	███████████████████░ 95.3%	9,528 / 9,994
experiments	███████████████████░ 95.6%	24,139 / 25,256
replay_vision	███████████████████░ 95.7%	13,300 / 13,896
growth	███████████████████░ 95.8%	2,836 / 2,959
warehouse_sources	███████████████████░ 96.2%	224,496 / 233,396
annotations	███████████████████░ 96.2%	732 / 761
feature_flags	███████████████████░ 96.3%	16,002 / 16,625
user_interviews	███████████████████░ 96.4%	2,242 / 2,325
access_control	███████████████████░ 96.8%	849 / 877
data_catalog	███████████████████░ 97.1%	2,034 / 2,095
customer_analytics	███████████████████░ 97.2%	7,480 / 7,698
analytics_platform	████████████████████ 98.0%	2,102 / 2,145
metrics	████████████████████ 98.2%	2,404 / 2,448
live_debugger	████████████████████ 99.2%	613 / 618
field_notes	████████████████████ 99.4%	158 / 159

Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.

✅ Django migration risk — no migrations to analyze

No Django migrations need risk analysis.

ℹ️ Docs preview — preview build triggered

Docs from this PR will be published at posthog.com.

Project	Preview	Updated (UTC)
posthog.com	Open preview	Jul 15, 2026, 9:14 PM

The preview should be ready in about 10 minutes. Open the preview at /handbook/engineering/.

[trunk-io]

     

View Full Report ↗︎ ⋅ Docs

[danielcarletti]

LGTM

[github-actions]

🦔 Hogbox preview · ✅ ready

▶ Open the preview

🔑 Login	test@posthog.com / 12345678 (demo data)
🧩 Running	this PR's backend and frontend, on the PostHog :master base
🔗 Link	stable across rebuilds — a re-push swaps the box underneath, the URL stays
🔒 Access	tailnet only (PostHog VPN)
🛠️ Admin	inspect & debug state in hogland
💤 Idle	sleeps after ~30 min idle (snapshot to S3, zero node cost) and wakes on your next visit in ~30s, behind a brief "waking up" screen

_{commit 4c83bc2 · box box-2dc64ceb877e · ready in 2151s (push → usable) · build log · rebuilds on every push, torn down on close}

[deployment-status-posthog]

Deploy status

Environment	Status	Deployed At	Workflow
dev	✅ Deployed	2026-07-17 06:19 UTC	Run
prod-us	✅ Deployed	2026-07-17 06:33 UTC	Run
prod-eu	✅ Deployed	2026-07-17 06:31 UTC	Run