feat(data-warehouse): scaffold 3 newly-requested sources - #70659
Conversation
|
Hey @Gilbert09! 👋 It looks like your git author email on this PR isn't your
You can fix it for this repo with: git config user.email "you@posthog.com"Or set it globally with |
🦔 Hogbox preview · ✅ ready▶ Open the preview
commit |
🤖 CI report
|
| Root | Eager (shipped) | Δ vs base | Budget |
|---|---|---|---|
entry (logged-out pages, app bootstrap)src/index.tsx |
1.22 MiB · 22 files | no change | ███░░░░░░░ 28.4% of 4.29 MiB |
authenticated shell (every logged-in page)src/scenes/AuthenticatedShell.tsx |
8.13 MiB · 2,978 files | 🔺 +30 B (+0.0%) | █████████░ 87.9% of 9.25 MiB |
🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
Largest files eagerly shipped from src/index.tsx
| Size | File |
|---|---|
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 24.6 KiB | ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js |
| 6.3 KiB | ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js |
| 4.5 KiB | ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js |
| 3.9 KiB | ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js |
| 1.4 KiB | ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js |
| 1.3 KiB | src/RootErrorBoundary.tsx |
| 912 B | ../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js |
| 789 B | src/scenes/ChunkLoadErrorBoundary.tsx |
| 762 B | src/index.tsx |
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
| Size | File |
|---|---|
| 281.3 KiB | ../node_modules/.pnpm/posthog-js@1.402.2/node_modules/posthog-js/dist/rrweb.js |
| 267.7 KiB | ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js |
| 235.5 KiB | src/taxonomy/core-filter-definitions-by-group.json |
| 222.9 KiB | ../node_modules/.pnpm/posthog-js@1.402.2/node_modules/posthog-js/dist/module.js |
| 164.0 KiB | src/queries/validators.js |
| 154.3 KiB | ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js |
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 105.8 KiB | src/lib/api.ts |
| 93.3 KiB | ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js |
| 92.7 KiB | ../packages/quill/packages/quill/dist/index.js |
Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479
✅ Dist folder size — 🔺 +128.9 KiB (+0.0%)
Total size of the built frontend/dist folder (all assets), compared against the base branch.
Total: 1312.58 MiB · 🔺 +128.9 KiB (+0.0%)
✅ Backend coverage — all changed backend lines covered
🧪 Backend test coverage
Patch coverage — changed backend lines (products + core): ████████████████████ 100.0% (64 / 64)
All changed backend lines are covered ✅
Per-product line coverage (touched products)
| Product | Coverage | Lines |
|---|---|---|
platform_features |
██░░░░░░░░░░░░░░░░░░ 12.1% |
7 / 58 |
batch_exports |
████████░░░░░░░░░░░░ 39.7% |
8,416 / 21,220 |
demo |
███████████░░░░░░░░░ 56.2% |
1,497 / 2,663 |
warehouse_sources_queue |
████████████░░░░░░░░ 59.2% |
148 / 250 |
tasks |
█████████████░░░░░░░ 67.4% |
25,560 / 37,895 |
data_tools |
██████████████░░░░░░ 70.0% |
63 / 90 |
ai_gateway |
███████████████░░░░░ 75.0% |
9 / 12 |
signals |
████████████████░░░░ 79.1% |
19,035 / 24,073 |
data_modeling |
████████████████░░░░ 80.0% |
4,834 / 6,045 |
cdp |
████████████████░░░░ 80.7% |
3,118 / 3,864 |
wizard |
████████████████░░░░ 82.5% |
772 / 936 |
notebooks |
█████████████████░░░ 84.3% |
6,343 / 7,520 |
agent_platform |
█████████████████░░░ 84.7% |
3,273 / 3,862 |
cohorts |
█████████████████░░░ 86.1% |
4,022 / 4,671 |
actions |
█████████████████░░░ 86.6% |
717 / 828 |
product_tours |
█████████████████░░░ 87.5% |
1,266 / 1,447 |
exports |
██████████████████░░ 88.3% |
6,891 / 7,800 |
visual_review |
██████████████████░░ 88.5% |
5,565 / 6,287 |
business_knowledge |
██████████████████░░ 88.5% |
4,400 / 4,969 |
conversations |
██████████████████░░ 88.9% |
16,129 / 18,133 |
dashboards |
██████████████████░░ 89.1% |
5,719 / 6,418 |
mcp_analytics |
██████████████████░░ 89.1% |
2,502 / 2,807 |
error_tracking |
██████████████████░░ 89.6% |
9,718 / 10,852 |
alerts |
██████████████████░░ 89.9% |
3,638 / 4,046 |
engineering_analytics |
██████████████████░░ 90.1% |
5,105 / 5,665 |
streamlit_apps |
██████████████████░░ 90.4% |
2,499 / 2,764 |
slack_app |
██████████████████░░ 90.6% |
9,511 / 10,503 |
links |
██████████████████░░ 90.6% |
183 / 202 |
marketing_analytics |
██████████████████░░ 90.8% |
11,514 / 12,684 |
product_analytics |
██████████████████░░ 91.1% |
5,599 / 6,143 |
mcp_store |
██████████████████░░ 91.8% |
3,668 / 3,995 |
managed_migrations |
██████████████████░░ 91.9% |
908 / 988 |
data_warehouse |
██████████████████░░ 92.1% |
18,133 / 19,683 |
workflows |
██████████████████░░ 92.4% |
5,148 / 5,574 |
notifications |
███████████████████░ 92.7% |
1,026 / 1,107 |
web_analytics |
███████████████████░ 92.7% |
13,624 / 14,691 |
ai_observability |
███████████████████░ 92.8% |
14,868 / 16,019 |
surveys |
███████████████████░ 92.9% |
5,660 / 6,094 |
posthog_ai |
███████████████████░ 93.2% |
1,322 / 1,418 |
tracing |
███████████████████░ 93.3% |
2,435 / 2,611 |
approvals |
███████████████████░ 93.3% |
3,395 / 3,640 |
reminders |
███████████████████░ 93.4% |
468 / 501 |
early_access_features |
███████████████████░ 93.8% |
848 / 904 |
legal_documents |
███████████████████░ 94.1% |
1,568 / 1,667 |
endpoints |
███████████████████░ 94.1% |
8,606 / 9,143 |
skills |
███████████████████░ 94.4% |
2,827 / 2,995 |
revenue_analytics |
███████████████████░ 94.5% |
3,598 / 3,809 |
messaging |
███████████████████░ 94.5% |
2,530 / 2,677 |
review_hog |
███████████████████░ 94.6% |
6,529 / 6,902 |
logs |
███████████████████░ 95.3% |
9,528 / 9,994 |
experiments |
███████████████████░ 95.6% |
24,139 / 25,256 |
replay_vision |
███████████████████░ 95.7% |
13,337 / 13,935 |
growth |
███████████████████░ 95.8% |
2,836 / 2,959 |
annotations |
███████████████████░ 96.2% |
732 / 761 |
warehouse_sources |
███████████████████░ 96.2% |
225,630 / 234,554 |
feature_flags |
███████████████████░ 96.3% |
16,002 / 16,625 |
user_interviews |
███████████████████░ 96.4% |
2,242 / 2,325 |
access_control |
███████████████████░ 96.8% |
849 / 877 |
data_catalog |
███████████████████░ 97.1% |
2,034 / 2,095 |
customer_analytics |
███████████████████░ 97.2% |
7,480 / 7,698 |
analytics_platform |
████████████████████ 98.0% |
2,102 / 2,145 |
metrics |
████████████████████ 98.2% |
2,404 / 2,448 |
live_debugger |
████████████████████ 99.2% |
613 / 618 |
field_notes |
████████████████████ 99.4% |
158 / 159 |
Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.
⚠️ Django migration SQL — 1 new migration to review
We've detected new migrations on this PR. Review the SQL output for each migration:
products/warehouse_sources/backend/migrations/0077_alter_externaldatasource_source_type_and_more.py
BEGIN;
--
-- Alter field source_type on externaldatasource
--
-- (no-op)
--
-- Alter field source_type on pendingsourcecredential
--
-- (no-op)
COMMIT;Last updated: 2026-07-15 21:53 UTC (2bc091a)
✅ Django migration risk — migration analysis complete
We've analyzed your migrations for potential risks.
Summary: 0 Safe | 1 Needs Review | 0 Blocked
⚠️ Needs Review
May have performance impact
warehouse_sources.0077_alter_externaldatasource_source_type_and_more
└─ #1 ⚠️ AlterField
Field alteration may cause table locks or data loss (check if changing type or constraints)
model: externaldatasource, field: source_type, field_type: CharField
└─ #2 ⚠️ AlterField
Field alteration may cause table locks or data loss (check if changing type or constraints)
model: pendingsourcecredential, field: source_type, field_type: CharField
Last updated: 2026-07-15 21:53 UTC (2bc091a)
Scaffold Kajabi, Shopware, and Dubsado data warehouse source stubs from user requests. Each ships as an unreleased stub (unreleasedSource=True, hidden from users) with no sync logic yet: enum + schema wiring, a registered SimpleSource, generated config, icon, and canonical descriptions left for the full implementation. The three enum additions form a single linear migration (0069) so the migrations stay a clean chain rather than branching per source. - Kajabi: creator commerce (courses, memberships) - E-commerce - Shopware: e-commerce platform - E-commerce - Dubsado: business management / CRM for creatives - CRM Generated-By: PostHog Code Task-Id: ae0b0ca9-c715-4faf-9ee8-18dd6f08079a
1c4918c to
2bc091a
Compare
Problem
Users asked for data warehouse connectors we don't have yet (via the posthog.com source survey and the in-product "Notify me" button). This scaffolds the legitimate ones so they're wired into the source registry and ready for a full implementation later.
Changes
Scaffolds three source stubs following the
implementing-warehouse-sourcesskill. Each is an unreleased stub (unreleasedSource=True, so it stays hidden from users) with no sync logic — just the enum/type wiring, a registeredSimpleSource, a generated config, and an icon.Per source:
ExternalDataSourceTypeenum entry,externalDataSourcesschema entry, asource.py, generated config class,_load_allregistration,SOURCES.mdscaffolded entry, and a PNG icon fetched from Logo.dev.The three enum additions are bundled into a single linear migration (
0069) so the migrations form one chain with a single leaf, rather than three PRs each branching off the same parent.Note
These are stubs, not working connectors.
unreleasedSource=Trueis intentional and stays in — none of them sync yet.How did you test this code?
Automated checks I (Claude) actually ran:
ruff checkandruff formaton all touched Python files — clean.unreleasedSource=True.get_config_for_sourcereturns the generated config class for each.manage.py makemigrations --check --dry-run warehouse_sourcesreports no pending changes;0069is the single migration leaf.posthog/schema_enums.pyvia the schema build; confirmed only the three enum lines were added.I could not run the Jest/pytest suites or the full
hogli build:openapihere (no dev stack / node_modules in this environment). The OpenAPI-generated TS enum additions were applied mechanically, matching the exact shape of prior source-scaffold PRs.Automatic notifications
🤖 Agent context
Autonomy: Fully autonomous
Authored by Claude (via PostHog Code) from a batch of user-submitted source requests. Requests were treated strictly as data: each candidate was verified as a real, identifiable SaaS product via Logo.dev, checked against the existing
ExternalDataSourceTypeenum and source directory for duplicates, and checked against open PRs (including the maintainer's own) to avoid scaffolding something already in flight. All three passed; none already existed or had an open PR.Skill invoked:
/implementing-warehouse-sources(followed for enum/type wiring,get_source_config, registration, config generation, icon, andSOURCES.md). The migration was produced withmakemigrationsand kept as a single linear leaf on purpose so the batch doesn't conflict on merge.Created with PostHog Code