feat(data-warehouse): implement rapid7_insightvm import source

[Gilbert09]

Problem

Rapid7 InsightVM is a cloud vulnerability management product. Data teams warehouse its asset inventory and vulnerability findings to build remediation-SLA, risk-score, and fleet-coverage dashboards. The source existed only as a hidden scaffold (unreleasedSource=True, no fields, no sync logic). This implements it end to end so users can actually connect it.

Changes

Fills in the scaffolded rapid7_insightvm source. It talks to the Insight Platform Cloud Integrations API (v4), which isn't a good fit for the generic REST client, so it uses a small bespoke transport:

• Auth + region: API key sent in the X-Api-Key header. Region is a required config field (US, EU, Canada, Australia, Asia Pacific, Japan) that selects a fixed regional base URL (https://{region}.api.insight.rapid7.com). The host set is fixed, so there's no SSRF surface.
• Endpoints: assets (host inventory with embedded vulnerability findings) and vulnerabilities (unique vulnerability definitions with CVE/CVSS data) — the two purpose-built bulk-export search endpoints. Cross-referenced against third-party connector docs for the canonical stream list.
• Transport: POST search endpoints with an empty body, size=1000 paging, and cursor-based pagination that resumes from the metadata.cursor token. Termination is defensive — it stops on a missing cursor, a repeated cursor (some deployments echo the last token), or an empty page. All outbound HTTP goes through make_tracked_session().
• Resumable: ResumableSource persisting the cursor after each yielded batch, so a Temporal heartbeat timeout resumes mid-stream instead of restarting.
• Credential validation, get_non_retryable_errors (401/403), a static schema catalog (lists_tables_without_credentials = True), and canonical_descriptions.py from the API docs.

Ships released with releaseStatus=ReleaseStatus.ALPHA — unreleasedSource is removed.

Note

Incremental sync is intentionally off. The v4 search body accepts filter expressions, but I couldn't verify (no live credentials) that a timestamp filter actually constrains results server-side rather than being silently ignored. Per the implementing-warehouse-sources guidance, both endpoints ship full refresh until that's confirmed with a curl smoke test. Partitioning is also left off — the v4 objects have no field confirmed stable across scans, and partitioning on a revised field would rewrite partitions every sync. These are conservative choices documented in code comments; they can be tightened once the API is verified against a real account.

Docs

I authored the user-facing doc following the documenting-warehouse-sources template (docsUrl → https://posthog.com/docs/cdp/sources/rapid7-insightvm). No posthog.com checkout was available in this environment, so the doc still needs to land in the posthog.com repo at contents/docs/cdp/sources/rapid7-insightvm.md and audit_source_docs run there. The content is ready to drop in:

rapid7-insightvm.md

---
title: Linking Rapid7 InsightVM as a source
sidebar: Docs
showTitle: true
availability: { free: full, selfServe: full, enterprise: full }
sourceId: Rapid7Insightvm
---

import SourceSetupIntro from "../_snippets/source-setup-intro.mdx"
import SyncModes from "../_snippets/sync-modes.mdx"
import TroubleshootingLink from "../_snippets/dw-troubleshooting-link.mdx"
import AlphaRelease from "../_snippets/alpha-release.mdx"

<AlphaRelease />

Sync your Rapid7 InsightVM asset inventory and vulnerability findings into PostHog to build remediation-SLA, risk-score, and fleet-coverage dashboards alongside the rest of your data.

## Prerequisites

Before you connect, you need:

- A paid InsightVM Cloud (Insight Platform) subscription.
- An Insight Platform API key. Log in at [insight.rapid7.com](https://insight.rapid7.com), go to **Settings → API Keys**, and create a new user or organization key.
- The region your Insight Platform account lives in. It's the first part of your product URL — for example `us` in `us.idr.insight.rapid7.com`. Supported regions are US, EU, Canada, Australia, Asia Pacific, and Japan.

## Adding a data source

<SourceSetupIntro />

Enter your **API key** and select your **region**. PostHog sends the key in the `X-Api-Key` header to the regional Cloud Integrations API (for example `https://us.api.insight.rapid7.com`).

## Sync modes

<SyncModes />

Rapid7 InsightVM syncs with full refresh. Each sync pulls the current asset inventory and vulnerability definitions through the Cloud Integrations API's cursor-paginated search endpoints.

## Configuration

<SourceParameters />

## Supported tables

<SourceTables />

## Troubleshooting

- **Authentication errors**: If the sync fails with a 401 or 403, the API key is invalid, expired, or missing permissions, or the selected region doesn't match your account. Generate a fresh key under **Settings → API Keys**, confirm the region, and reconnect.

<TroubleshootingLink />

How did you test this code?

Automated tests only — I (Claude) had no live Rapid7 credentials, so the transport was verified against the documented request/response shape, not a real account.

• tests/test_rapid7_insightvm.py (transport): cursor pagination walks pages and terminates on missing / repeated cursor and empty page; state saved after each batch; resume-from-saved-cursor uses the token as the starting request; retryable (429/5xx) vs terminal (4xx) status handling; credential-validation status mapping; SourceResponse shape.
• tests/test_rapid7_insightvm_source.py (source): source type, config fields and region options, the unreleasedSource guard (asserts it's falsy so the scaffold flag can't creep back in), full-refresh schemas, credential plumbing, resumable-manager binding, source_for_pipeline argument plumbing.

Ran the targeted suite (27 passed), plus test_source_config_generator and test_source_categories (1700 passed) to confirm the generated config and category wiring hold. ruff check and ruff format clean.

Automatic notifications

• Publish to changelog?
• Alert Sales and Marketing teams?

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Authored by Claude (Fable) via PostHog Code. Invoked the /implementing-warehouse-sources, /documenting-warehouse-sources, and /writing-tests skills.

Key decisions: chose a bespoke requests-based transport over rest_source.RESTClient because the v4 API uses POST search bodies with cursor tokens and mandatory regional hosts. Went with ResumableSource since the cursor gives a clean resumption point. Kept both endpoints on full refresh and left partitioning off rather than guessing at unverified server-side timestamp filters / stable partition fields — noted inline and above so it's an explicit, reversible call. Config fields were defined in source.py, then generate:source-configs regenerated generated_configs.py (schema:build was a no-op here since the enum entry already exists from the scaffold and I didn't touch schema-general.ts; frontend node_modules also wasn't installed).

---

Created with PostHog Code

[github-actions]

Hey @Gilbert09! 👋

It looks like your git author email on this PR isn't your @posthog.com address (owerstom@gmail.com). Since you're on the PostHog team, it's worth pointing your local git author email at your @posthog.com address. Why it matters:

• Consistent work identity in git history — internal tooling that attributes commits to team members keys off your @posthog.com address.
• Keeps team contributions easy to tell apart from external community ones when scanning history.

You can fix it for this repo with:

git config user.email "you@posthog.com"

Or set it globally with git config --global user.email "you@posthog.com". No need to redo this PR — just a nudge for next time. 🙂

[github-actions]

🤖 CI report

⚠️ Backend coverage — 99.0% of changed backend lines covered — 2 uncovered

🧪 Backend test coverage

Patch coverage — changed backend lines (products + core): ████████████████████ 99.0% (293 / 295)

File	Patch	Uncovered changed lines
products/warehouse_sources/backend/temporal/data_imports/sources/rapid7_insightvm/source.py	92.9%	49, 53

🤖 Agents: add a test covering the lines above, or note why under "How did you test this code?". Machine-readable gap list: the patch-coverage artifact on this run (gh run download 29471176884 -n patch-coverage), or the coverage-data block at the end of this comment.

Per-product line coverage (touched products)

Product	Coverage	Lines
demo	███████████░░░░░░░░░ 56.2%	1,497 / 2,663
tasks	█████████████░░░░░░░ 67.4%	25,560 / 37,895
warehouse_sources	████████████████░░░░ 78.7%	192,021 / 243,961
signals	████████████████░░░░ 79.1%	19,037 / 24,073
data_modeling	████████████████░░░░ 80.0%	4,834 / 6,045
cdp	████████████████░░░░ 80.7%	3,118 / 3,864
notebooks	█████████████████░░░ 84.3%	6,343 / 7,520
agent_platform	█████████████████░░░ 84.7%	3,273 / 3,862
cohorts	█████████████████░░░ 86.1%	4,022 / 4,671
actions	█████████████████░░░ 86.6%	717 / 828
product_tours	█████████████████░░░ 87.5%	1,266 / 1,447
exports	██████████████████░░ 88.3%	6,891 / 7,800
conversations	██████████████████░░ 88.9%	16,129 / 18,133
dashboards	██████████████████░░ 89.1%	5,719 / 6,418
mcp_analytics	██████████████████░░ 89.1%	2,502 / 2,807
error_tracking	██████████████████░░ 89.6%	9,718 / 10,852
alerts	██████████████████░░ 89.9%	3,638 / 4,046
engineering_analytics	██████████████████░░ 90.1%	5,105 / 5,665
streamlit_apps	██████████████████░░ 90.4%	2,499 / 2,764
slack_app	██████████████████░░ 90.6%	9,511 / 10,503
marketing_analytics	██████████████████░░ 90.8%	11,514 / 12,684
product_analytics	██████████████████░░ 91.1%	5,599 / 6,143
data_warehouse	██████████████████░░ 92.1%	18,133 / 19,683
workflows	██████████████████░░ 92.4%	5,148 / 5,574
web_analytics	███████████████████░ 92.7%	13,624 / 14,691
ai_observability	███████████████████░ 92.8%	14,868 / 16,019
surveys	███████████████████░ 92.9%	5,687 / 6,120
posthog_ai	███████████████████░ 93.2%	1,322 / 1,418
approvals	███████████████████░ 93.3%	3,395 / 3,640
reminders	███████████████████░ 93.4%	468 / 501
early_access_features	███████████████████░ 93.8%	848 / 904
endpoints	███████████████████░ 94.1%	8,606 / 9,143
skills	███████████████████░ 94.4%	2,827 / 2,995
revenue_analytics	███████████████████░ 94.5%	3,598 / 3,809
review_hog	███████████████████░ 94.6%	6,532 / 6,905
logs	███████████████████░ 95.3%	9,528 / 9,994
experiments	███████████████████░ 95.6%	24,171 / 25,288
replay_vision	███████████████████░ 95.7%	13,354 / 13,952
annotations	███████████████████░ 96.2%	732 / 761
feature_flags	███████████████████░ 96.3%	16,038 / 16,661
user_interviews	███████████████████░ 96.4%	2,242 / 2,325
data_catalog	███████████████████░ 97.1%	2,035 / 2,095
customer_analytics	███████████████████░ 97.2%	7,480 / 7,698

Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.

✅ Django migration risk — no migrations to analyze

No Django migrations need risk analysis.

ℹ️ Docs preview — preview build triggered

Docs from this PR will be published at posthog.com.

Project	Preview	Updated (UTC)
posthog.com	Open preview	Jul 15, 2026, 8:57 PM

The preview should be ready in about 10 minutes. Open the preview at /handbook/engineering/.

[trunk-io]

     

View Full Report ↗︎ ⋅ Docs

[danielcarletti]

LGTM

[deployment-status-posthog]

Deploy status

Environment	Status	Deployed At	Workflow
dev	✅ Deployed	2026-07-16 06:31 UTC	Run
prod-us	✅ Deployed	2026-07-16 06:46 UTC	Run
prod-eu	✅ Deployed	2026-07-16 06:44 UTC	Run