feat(data-warehouse): implement helicone import source - #70714
Conversation
|
Hey @Gilbert09! 👋 It looks like your git author email on this PR isn't your
You can fix it for this repo with: git config user.email "you@posthog.com"Or set it globally with |
🦔 Hogbox preview · ✅ ready▶ Open the preview
commit |
🤖 CI report✅ Bundle size — no changeUncompressed size of every built Total: 64.77 MiB · no change No file changed by more than 1000 B. Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report ✅ Eager graph — within budgetHow much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy
🟢 Largest files eagerly shipped from
|
| Size | File |
|---|---|
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 24.6 KiB | ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js |
| 6.3 KiB | ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js |
| 4.5 KiB | ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js |
| 3.9 KiB | ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js |
| 1.4 KiB | ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js |
| 1.3 KiB | src/RootErrorBoundary.tsx |
| 912 B | ../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js |
| 789 B | src/scenes/ChunkLoadErrorBoundary.tsx |
| 762 B | src/index.tsx |
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
| Size | File |
|---|---|
| 281.3 KiB | ../node_modules/.pnpm/posthog-js@1.402.3/node_modules/posthog-js/dist/rrweb.js |
| 267.7 KiB | ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js |
| 235.5 KiB | src/taxonomy/core-filter-definitions-by-group.json |
| 222.9 KiB | ../node_modules/.pnpm/posthog-js@1.402.3/node_modules/posthog-js/dist/module.js |
| 164.0 KiB | src/queries/validators.js |
| 154.3 KiB | ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js |
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 105.8 KiB | src/lib/api.ts |
| 93.3 KiB | ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js |
| 92.7 KiB | ../packages/quill/packages/quill/dist/index.js |
Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479
✅ Dist folder size — 🔺 +1.1 KiB (+0.0%)
Total size of the built frontend/dist folder (all assets), compared against the base branch.
Total: 1312.48 MiB · 🔺 +1.1 KiB (+0.0%)
⚠️ Backend coverage — 95.0% of changed backend lines covered — 21 uncovered
🧪 Backend test coverage
Patch coverage — changed backend lines (products + core): ███████████████████░ 95.0% (450 / 471)
| File | Patch | Uncovered changed lines |
|---|---|---|
products/warehouse_sources/backend/temporal/data_imports/sources/helicone/helicone.py |
86.7% | 62, 83, 98, 101–102, 230, 277, 297–298, 303, 305–306, 316–321, 323 |
products/warehouse_sources/backend/temporal/data_imports/sources/helicone/source.py |
94.3% | 95, 99 |
🤖 Agents: add a test covering the lines above, or note why under "How did you test this code?". Machine-readable gap list: the patch-coverage artifact on this run (gh run download 29458579227 -n patch-coverage), or the coverage-data block at the end of this comment.
Per-product line coverage (touched products)
| Product | Coverage | Lines |
|---|---|---|
demo |
███████████░░░░░░░░░ 56.2% |
1,497 / 2,663 |
tasks |
█████████████░░░░░░░ 67.4% |
25,560 / 37,895 |
signals |
████████████████░░░░ 79.1% |
19,033 / 24,073 |
data_modeling |
████████████████░░░░ 80.0% |
4,834 / 6,045 |
cdp |
████████████████░░░░ 80.7% |
3,118 / 3,864 |
notebooks |
█████████████████░░░ 84.3% |
6,343 / 7,520 |
agent_platform |
█████████████████░░░ 84.7% |
3,273 / 3,862 |
cohorts |
█████████████████░░░ 86.1% |
4,022 / 4,671 |
actions |
█████████████████░░░ 86.6% |
717 / 828 |
product_tours |
█████████████████░░░ 87.5% |
1,266 / 1,447 |
exports |
██████████████████░░ 88.3% |
6,891 / 7,800 |
conversations |
██████████████████░░ 88.9% |
16,129 / 18,133 |
dashboards |
██████████████████░░ 89.1% |
5,719 / 6,418 |
mcp_analytics |
██████████████████░░ 89.1% |
2,502 / 2,807 |
error_tracking |
██████████████████░░ 89.6% |
9,718 / 10,852 |
alerts |
██████████████████░░ 89.9% |
3,638 / 4,046 |
engineering_analytics |
██████████████████░░ 90.1% |
5,105 / 5,665 |
streamlit_apps |
██████████████████░░ 90.4% |
2,499 / 2,764 |
slack_app |
██████████████████░░ 90.6% |
9,511 / 10,503 |
marketing_analytics |
██████████████████░░ 90.8% |
11,514 / 12,684 |
product_analytics |
██████████████████░░ 91.1% |
5,599 / 6,143 |
data_warehouse |
██████████████████░░ 92.1% |
18,133 / 19,683 |
workflows |
██████████████████░░ 92.4% |
5,148 / 5,574 |
web_analytics |
███████████████████░ 92.7% |
13,624 / 14,691 |
ai_observability |
███████████████████░ 92.8% |
14,868 / 16,019 |
surveys |
███████████████████░ 92.9% |
5,687 / 6,120 |
posthog_ai |
███████████████████░ 93.2% |
1,322 / 1,418 |
approvals |
███████████████████░ 93.3% |
3,395 / 3,640 |
reminders |
███████████████████░ 93.4% |
468 / 501 |
early_access_features |
███████████████████░ 93.8% |
848 / 904 |
endpoints |
███████████████████░ 94.1% |
8,606 / 9,143 |
skills |
███████████████████░ 94.4% |
2,827 / 2,995 |
revenue_analytics |
███████████████████░ 94.5% |
3,598 / 3,809 |
review_hog |
███████████████████░ 94.6% |
6,532 / 6,905 |
logs |
███████████████████░ 95.3% |
9,528 / 9,994 |
experiments |
███████████████████░ 95.6% |
24,171 / 25,288 |
replay_vision |
███████████████████░ 95.7% |
13,354 / 13,952 |
annotations |
███████████████████░ 96.2% |
732 / 761 |
warehouse_sources |
███████████████████░ 96.2% |
230,698 / 239,749 |
feature_flags |
███████████████████░ 96.3% |
16,002 / 16,625 |
user_interviews |
███████████████████░ 96.4% |
2,242 / 2,325 |
data_catalog |
███████████████████░ 97.1% |
2,034 / 2,095 |
customer_analytics |
███████████████████░ 97.2% |
7,480 / 7,698 |
Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.
![veria-ai[bot]](https://avatars.githubusercontent.com/in/1723776?s=60&v=4){kind=small}
PR overviewAll previously flagged issues have been addressed. No open security concerns remain on this pull request. Security reviewNo open security issues remain on this pull request. Fixed/addressed: 2 · PR risk: 0/10 |
Implements the Helicone warehouse source as a ResumableSource pulling requests (incremental on request_created_at via the filter AST), sessions, users, and prompts, with regional US/EU hosts, tracked HTTP transport, canonical table descriptions, and offset-based resume state. Generated-By: PostHog Code Task-Id: b085672b-ea5a-42d3-8a00-d49175254f10
…ting Declare region as a connection host field so editing it forces re-entering the API key instead of reusing the stored secret against the other regional host (matches the Datadog/AssemblyAI pattern). Generated-By: PostHog Code Task-Id: b085672b-ea5a-42d3-8a00-d49175254f10
Helicone request pages carry request_body/response_body containing customer prompts and model output. Pass capture=False on both the sync and credential-validation tracked sessions so those bodies never land in diagnostic HTTP samples, and redact the API key from logged URLs and samples for defense in depth. Generated-By: PostHog Code Task-Id: 7a31032b-1697-4380-89d2-c521e498c9dc
Remove unreleasedSource=True so the finished source is visible in the connector catalog, and drop any test asserting the hidden state.
9693f25 to
57bc622
Compare
![graphite-app[bot]](https://avatars.githubusercontent.com/in/158384?s=60&v=4){kind=small}
The prompts endpoint is page-based, but resume state stored the cumulative row count and derived the page via offset // pageSize. That drifts once any page comes back short, re-requesting an already-paged page on resume. Store the next page index directly in the resume state instead. Generated-By: PostHog Code Task-Id: 00dc30e7-8768-485d-937f-5be541b5b0a5
2 participants
Problem
Helicone is a scaffolded warehouse source (registered with
unreleasedSource=Trueand an empty stub) with no sync logic. Teams using Helicone for LLM observability can't join their LLM request, cost, and usage data with their product data in PostHog.Why
Fills in the scaffolded Helicone connector so LLM observability data (requests, sessions, per-user usage, prompts) can be imported into the Data warehouse.
Changes
Implements the Helicone source end-to-end as a
ResumableSource, following the standardsource.py/settings.py/helicone.pysplit:requests(the logged LLM request stream via the bulkPOST /v1/request/query-clickhouseendpoint),sessions(POST /v1/session/query),users(per-user usage aggregates viaPOST /v1/user/query), andprompts(POST /v1/prompt-2025/query).requestsonly, using a server-siderequest_created_atgte condition in Helicone's filter AST with ascendingcreated_atsort and limit/offset paging. The filter rides in every page's body, so pagination stays bounded by the watermark. First incremental sync is capped at 365 days.sessions/users/promptsare aggregates or metadata that change in place with no changed-since filter, so they're full refresh only.regionselect (USapi.helicone.ai/ EUeu.api.helicone.ai) alongside the API key, mirroring the Amplitude source's pattern.make_tracked_session(); retries via tenacity on 429/5xx/transient transport errors; 401/403 on both regional hosts registered as non-retryable errors.canonical_descriptions.pyfor all four tables (from the official API docs),lists_tables_without_credentials = Trueso the public docs render the table catalog, and the official Helicone icon mark (SVG, from their Apache-2.0 repo).HeliconeSourceConfig, moved helicone into the Implemented table inSOURCES.md.Per the rollout plan the source stays behind
unreleasedSource=TruewithreleaseStatus=ReleaseStatus.ALPHAfor now.Note
Webhook support was deliberately left out despite Helicone having a manageable webhook API. Helicone's webhook payload (verified against their open-source sender) carries only
request_id, truncated bodies, and optional model/provider/token metadata, with no creation timestamp and a different shape than the pull rows, so merging webhook deliveries into therequest_created_at-partitioned requests table would produce partition-key-less, schema-divergent rows. The append-only request log makes incremental pull a safe substitute for deltas.The user-facing doc is in PostHog/posthog.com#18459;
docsUrlmatches its slug andaudit_source_docspasses for helicone against that checkout.How did you test this code?
pytest products/.../sources/helicone/tests/- 48 tests pass. Transport tests catch: wrong filter-AST shape (which Helicone answers with silently empty data), pagination not terminating or not resuming from saved offset, resume recomputing the time window and shifting rows across page boundaries, sync-mode/primary-key/partition metadata regressions, and non-retryable error keys drifting from the messageraise_for_statusactually produces. Source tests cover schema advertising, credential-validation plumbing, and full-refresh runs dropping the watermark.test_source_categories.pyandtest_source_config_generator.py(1537 tests) pass.ruff check/ruff formatclean.hogliisn't available in this environment, soci:preflightcouldn't run.Automatic notifications
Docs update
Doc added in PostHog/posthog.com#18459.
🤖 Agent context
Autonomy: Fully autonomous
/implementing-warehouse-sources,/documenting-warehouse-sources,/writing-tests.ResumableSource). Chose pull-only over the suggestedResumableSource+WebhookSourcecombination after verifying from Helicone's OSS webhook sender that payloads lack the partition timestamp needed for safe merges (details in the note above).generate_source_configsproduced unrelated drift ingenerated_configs.py(including a semantic change toStripeAuthMethodConfigoptionality that would break Stripe config parsing), so only the Helicone hunk was kept, exactly as the generator emitted it. The generator/committed-file drift may be worth a separate look.Created with PostHog Code