Skip to content

feat(data-warehouse): implement unleash import source - #70002

Merged
Gilbert09 merged 2 commits into
masterfrom
posthog-code/implement-unleash-source
Jul 14, 2026
Merged

feat(data-warehouse): implement unleash import source#70002
Gilbert09 merged 2 commits into
masterfrom
posthog-code/implement-unleash-source

Conversation

@Gilbert09

Copy link
Copy Markdown
Member

Problem

PostHog can't yet pull Unleash feature-management configuration into the data warehouse. Teams that run Unleash want their flags, projects, environments, strategies, segments, and related config available alongside product analytics and experiments so they can join flag state against behavioral data.

Changes

This completes the unleash data warehouse import source and ships it behind unreleasedSource=True at releaseStatus=alpha.

The connector talks to the Unleash Admin REST API (/api/admin) and follows the standard source split:

  • settings.py — declarative endpoint catalog: features (flag search, paginated), projects, environments, strategies, segments, context_fields, tags, tag_types, feature_types, addons, users. Each declares its response selector and a table-wide-unique primary key (e.g. tags keys on (type, value); flags on name).
  • unleash.py — tracked-session transport (make_tracked_session), tenacity retry/backoff, SSRF guards on the customer-supplied instance URL (redirects refused, private hosts blocked on cloud, https enforced on cloud, parser-differential userinfo/backslash URLs rejected), credential validation, and per-endpoint permission probing.
  • source.py — source config/form fields (instance URL + Admin API token sent as the raw Authorization header, no Bearer prefix), schema listing, non-retryable auth errors, and resumable-source wiring.
  • canonical_descriptions.py — curated table/column descriptions from the official API docs for the AI agent and public docs.

All endpoints sync in full refresh mode: the Admin API exposes no server-side updated_after-style filter on these configuration list endpoints, so there is no reliable cursor to drive an incremental sync. Collections are configuration-sized, so a full pull each sync is cheap. The one paginated endpoint (features) is resumable via a persisted offset.

Note

The transport, settings, credential validation, canonical descriptions, and both test modules were already present on master. This change finalizes the connector: it replaces the placeholder service icon with the official Unleash logo (unleash.svg, points iconPath at it, drops the unused PNG) and removes the stale duplicate unleash entry from the scaffolded list in SOURCES.md (it already appears in the implemented table).

How did you test this code?

Automated tests run locally (all passing):

  • products/warehouse_sources/backend/temporal/data_imports/sources/unleash/tests/ — 74 tests covering URL normalization, SSRF/redirect guards, pagination + resume, row extraction, primary-key mapping, credential validation status mapping, and per-endpoint permission probing.
  • products/warehouse_sources/backend/temporal/data_imports/sources/tests/test_source_categories.py — 1506 tests (confirms unleash has a valid category).
  • ruff check and ruff format --check clean on the changed files.

I could not exercise the live Admin API (no Unleash credentials in this environment); endpoint behavior follows the current public API docs, and the pagination/sort assumptions are noted in code comments.

Docs update

The user-facing doc belongs in the separate posthog.com repo at contents/docs/cdp/sources/unleash.md (matching docsUrl). No posthog.com checkout was available here, so audit_source_docs could not be run. The doc has been drafted (canonical template, <SourceParameters /> + <SourceTables />, alpha-release callout) and still needs to land in posthog.com. The source sets lists_tables_without_credentials = True, so the Supported tables section will render.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Authored with Claude Code (PostHog Code cloud task). Skills used: /implementing-warehouse-sources and /documenting-warehouse-sources.

On starting, the full source implementation was already merged on master and verified passing, so no re-implementation was possible or needed. The remaining gaps were the service icon (a red/blue placeholder glyph, not the Unleash brand) and a SOURCES.md inventory inconsistency. The official Unleash logo was pulled from the open-source Unleash/unleash repo (logoPlain.svg, the icon mark) rather than a placeholder or a hardcoded Logo.dev key. unreleasedSource=True / releaseStatus=alpha were left in place per the connector's alpha state.


Created with PostHog Code

Fill in the scaffolded Unleash data warehouse source end to end: Admin API
transport, endpoint catalog, credential validation, per-endpoint permission
probing, canonical column descriptions, an SVG icon, and tests.

Ships behind unreleasedSource=True at releaseStatus=alpha.

Generated-By: PostHog Code
Task-Id: e41299d2-ac3e-4079-bb38-d75add0ac8e4
The source now references the official unleash.svg logo, so the placeholder PNG is unused.

Generated-By: PostHog Code
Task-Id: e41299d2-ac3e-4079-bb38-d75add0ac8e4
@Gilbert09 Gilbert09 added the skip-agent-review Save $$$, skip auto agent reviews (Greptile) — use for trivial or chore PRs label Jul 10, 2026
@Gilbert09 Gilbert09 self-assigned this Jul 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Hey @Gilbert09! 👋

It looks like your git author email on this PR isn't your @posthog.com address (owerstom@gmail.com). Since you're on the PostHog team, it's worth pointing your local git author email at your @posthog.com address. Why it matters:

  • Consistent work identity in git history — internal tooling that attributes commits to team members keys off your @posthog.com address.
  • Keeps team contributions easy to tell apart from external community ones when scanning history.

You can fix it for this repo with:

git config user.email "you@posthog.com"

Or set it globally with git config --global user.email "you@posthog.com". No need to redo this PR — just a nudge for next time. 🙂

@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

🦔 Hogbox preview · ✅ ready

▶ Open the preview

🔑 Login test@posthog.com / 12345678 (demo data)
🧩 Running this PR's backend and frontend, on the PostHog :master base
🔗 Link stable across rebuilds — a re-push swaps the box underneath, the URL stays
🔒 Access tailnet only (PostHog VPN)
💤 Idle sleeps after ~30 min idle (snapshot to S3, zero node cost) and wakes on your next visit in ~30s, behind a brief "waking up" screen

commit 1ff5e65 · box box-fcaa4373d095 · ready in 752s (push → usable) · build log · rebuilds on every push, torn down on close

@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested a review from a team July 10, 2026 12:59
@greptile-apps

greptile-apps Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Reviews (1): Last reviewed commit: "chore(data-warehouse): remove placeholde..." | Re-trigger Greptile

@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

🤖 CI report

Bundle size — no change

Uncompressed size of every built .js bundle, compared against the base branch.

Total: 70.08 MiB · no change

No file changed by more than 1000 B.

Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report

Eager graph — within budget

How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.

Root Eager (shipped) Δ vs base Budget
entry (logged-out pages, app bootstrap)
src/index.tsx
1.20 MiB · 22 files no change ███░░░░░░░ 28.1% of 4.29 MiB
authenticated shell (every logged-in page)
src/scenes/AuthenticatedShell.tsx
8.08 MiB · 2,958 files no change █████████░ 87.3% of 9.25 MiB

🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx

Largest files eagerly shipped from src/index.tsx
Size File
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
24.6 KiB ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js
6.3 KiB ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js
4.5 KiB ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js
3.9 KiB ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js
1.4 KiB ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js
1.3 KiB src/RootErrorBoundary.tsx
912 B ../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js
789 B src/scenes/ChunkLoadErrorBoundary.tsx
668 B src/index.tsx
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
Size File
277.3 KiB ../node_modules/.pnpm/posthog-js@1.399.1/node_modules/posthog-js/dist/rrweb.js
266.9 KiB ../node_modules/.pnpm/@posthog+icons@0.37.4_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
224.6 KiB src/taxonomy/core-filter-definitions-by-group.json
221.5 KiB ../node_modules/.pnpm/posthog-js@1.399.1/node_modules/posthog-js/dist/module.js
164.0 KiB src/queries/validators.js
154.0 KiB ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
105.9 KiB src/lib/api.ts
93.3 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
90.6 KiB ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js

Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479

Dist folder size — 🟢 -41.8 KiB (-0.0%)

Total size of the built frontend/dist folder (all assets), compared against the base branch.

Total: 1435.40 MiB · 🟢 -41.8 KiB (-0.0%)

Playwright — all passed

All tests passed.

View test results →

@trunk-io

trunk-io Bot commented Jul 10, 2026

Copy link
Copy Markdown

Static BadgeStatic BadgeStatic BadgeStatic Badge

View Full Report ↗︎Docs

@estefaniarabadan estefaniarabadan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small, isolated change to a single existing source — low-risk and self-contained, CI green. LGTM 👍

@Gilbert09
Gilbert09 merged commit acd491e into master Jul 14, 2026
222 of 267 checks passed
@Gilbert09
Gilbert09 deleted the posthog-code/implement-unleash-source branch July 14, 2026 10:34
@inkeep

inkeep Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

👋 Hey @Gilbert09!

I analyzed this PR for documentation updates but noticed the Unleash connector is currently gated behind unreleasedSource=True with releaseStatus=alpha.

No documentation PR will be created at this time since the feature is not yet publicly available.

Once the source is ready for general availability and these flags are removed, please ping me again and I'll create the documentation at contents/docs/cdp/sources/unleash.md in the posthog.com repo.


This analysis was performed by the Inkeep Content Writer agent, which is maintained by the Docs and Wizard team. Please reach out on Slack for help if needed.

@deployment-status-posthog

deployment-status-posthog Bot commented Jul 14, 2026

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-07-14 11:12 UTC Run
prod-us ✅ Deployed 2026-07-14 11:23 UTC Run
prod-eu ✅ Deployed 2026-07-14 11:32 UTC Run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-agent-review Save $$$, skip auto agent reviews (Greptile) — use for trivial or chore PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants