Skip to content

feat(data-warehouse): implement workable import source#65657

Merged
talyn-app[bot] merged 5 commits into
masterfrom
posthog-code/implement-workable-source
Jun 24, 2026
Merged

feat(data-warehouse): implement workable import source#65657
talyn-app[bot] merged 5 commits into
masterfrom
posthog-code/implement-workable-source

Conversation

@Gilbert09

Copy link
Copy Markdown
Member

Problem

Workable (a cloud applicant tracking / recruiting platform) was registered as a scaffolded data warehouse source — visible in the registry but with no sync logic. Customers running recruiting on Workable couldn't pull their jobs and candidates into PostHog for analysis.

Changes

Implements the Workable source end to end against the SPI v3 REST API, following the architecture contract from the implementing-warehouse-sources skill (source.py / settings.py / workable.py split):

  • Streams: jobs, candidates, members, recruiters, stages — the canonical Workable connector stream set.
  • Base class: ResumableSource. Pagination follows the paging.next cursor URL; resume state (the next URL) is persisted after each page so a heartbeat-timeout can pick up where it left off rather than restarting.
  • Incremental: jobs and candidates expose genuine server-side time filters (updated_after / created_after), so they support incremental + append on updated_at (default) or created_at. members, recruiters and stages have no update timestamp (and stages/recruiters aren't even paginated) so they ship full refresh.
  • Auth: Bearer SPI access token plus the per-account subdomain. The subdomain is validated against a strict DNS-label pattern before building the URL, and declared in connection_host_fields so retargeting it re-requires the token (prevents credential exfiltration to an attacker-controlled host).
  • Transport: all outbound HTTP goes through make_tracked_session(). Tenacity owns retries (the adapter's own status retries are disabled to avoid double-retrying) and backs off on 429 / 5xx to stay clear of Workable's ~10 req/10s limit. 401/403/invalid-subdomain are mapped as non-retryable.
  • Curated canonical_descriptions.py for all five tables from the official docs.
  • Kept behind unreleasedSource=True with releaseStatus=ALPHA.

A note on sort order

Workable paginates by since_id (ascending id, which tracks ascending created_at) and has no way to sort by updated_at. So sort_mode is chosen per run: asc when the cursor field is created_at (arrival order matches the cursor, watermark advances per batch), desc when it's updated_at (arrival order is unrelated to the cursor, so the watermark commit is deferred to sync completion — avoiding advancing past unsynced rows on a partial failure). Merge dedupes on the primary key in all cases.

How did you test this code?

I'm an agent. I verified the live API shape with curl and the public docs (response wrappers, paging.next, the updated_after/created_after params, candidate/job field presence) before finalizing endpoint logic — an unauthenticated probe confirmed the https://<subdomain>.workable.com/spi/v3/ host and a 401 on missing auth. I could not curl the authenticated endpoints (no token), so the incremental filter behavior is taken from the documented params rather than a future-date smoke test; this is called out in code comments.

Automated tests I actually ran (all passing):

  • posthog/temporal/data_imports/sources/workable/tests/test_workable.py — transport: subdomain validation (incl. host-injection attempts), datetime formatting, URL/filter construction, sort-mode selection, cursor pagination + resume + state-save-after-yield, retryable vs non-retryable status handling, SourceResponse shape per endpoint, credential status mapping.
  • posthog/temporal/data_imports/sources/workable/tests/test_workable_source.py — source class: config/fields, per-endpoint incremental support, schema filtering, credential 401/403 (create vs schema) mapping, canonical descriptions, argument plumbing.
  • posthog/temporal/data_imports/sources/tests/test_source_categories.py (CI-enforced category coverage).
  • ruff check and ruff format on the changed files.

Docs update

A docs page at posthog.com/docs/cdp/sources/workable (referenced by docsUrl) does not exist yet and would need to be added separately.

🤖 Agent context

Autonomy: Fully autonomous

  • Built with Claude Code. Invoked the implementing-warehouse-sources skill and followed its end-to-end workflow and conventions; used the repo's generate_source_configs management command to regenerate WorkableSourceConfig.
  • Modeled closely on the existing klaviyo source (also a cursor-paginated ResumableSource).
  • Decisions worth flagging for the reviewer: (1) shipped only the 5 streams with reliable, well-documented shapes rather than guessing at events/activities; (2) members relies on paging.next + limit=100 and would truncate accounts with >100 members (the docs show no paging object for it) — acceptable for alpha, noted in code; (3) the sort_mode reasoning above; (4) webhooks are out of scope for this PR despite Workable supporting subscription management — the pull source stands on its own and webhooks can be layered on later.

Fills in the scaffolded Workable (ATS / recruiting) data warehouse source end to end.

Implements a ResumableSource over Workable's SPI v3 REST API: jobs, candidates,
members, recruiters and stages. Jobs and candidates sync incrementally via the
server-side updated_after / created_after filters; the rest are full refresh.
Pagination follows the paging.next cursor, with resume state saved after each page.

Generated-By: PostHog Code
Task-Id: 31740fe2-f367-4984-8fa8-a913aa1d6c13
@github-actions

Copy link
Copy Markdown
Contributor

Hey @Gilbert09! 👋

It looks like your git author email on this PR isn't your @posthog.com address (owerstom@gmail.com). Since you're on the PostHog team, it's worth pointing your local git author email at your @posthog.com address. Why it matters:

  • Consistent work identity in git history — internal tooling that attributes commits to team members keys off your @posthog.com address.
  • Keeps team contributions easy to tell apart from external community ones when scanning history.

You can fix it for this repo with:

git config user.email "you@posthog.com"

Or set it globally with git config --global user.email "you@posthog.com". No need to redo this PR — just a nudge for next time. 🙂

@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested a review from a team June 23, 2026 21:33
Comment thread posthog/temporal/data_imports/sources/workable/workable.py Outdated
Comment thread posthog/temporal/data_imports/sources/workable/workable.py Outdated
@greptile-apps

greptile-apps Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Reviews (1): Last reviewed commit: "feat(data-warehouse): implement workable..." | Re-trigger Greptile

Comment thread posthog/temporal/data_imports/sources/workable/workable.py Outdated
Fix the mypy errors flagged by CI (all in the test modules: loosen the page-dict
type, annotate an empty dict, pass response= to HTTPError, narrow the field type
before reading .secret, type the patched return as Any).

Also tighten the subdomain validation regex to reject a trailing hyphen, which is
not a valid DNS label and would otherwise fail at the HTTP layer with a confusing
network error rather than a clear validation message.

Generated-By: PostHog Code
Task-Id: 31740fe2-f367-4984-8fa8-a913aa1d6c13
Comment thread posthog/temporal/data_imports/sources/workable/workable.py Outdated
Comment thread posthog/temporal/data_imports/sources/workable/workable.py Outdated
Pass redact_values=(api_token,) to the tracked sessions in get_rows and
validate_credentials so the Workable bearer token is scrubbed from any
captured HTTP request/response samples, consistent with other connectors.

Generated-By: PostHog Code
Task-Id: 42bb41c5-6cc6-4fd1-9654-af539b030e73
Generated-By: PostHog Code
Task-Id: 42bb41c5-6cc6-4fd1-9654-af539b030e73

Copy link
Copy Markdown
Member Author

CI status update

The mypy / Python code-quality checks that were failing are now green, and the two security review threads (Workable token redaction in tracked HTTP sessions) have been addressed in redact_values=(api_token,) and resolved.

The branch is up to date with master.

Two checks remain red, but neither is caused by this PR:

  • Validate migrations fails during manage.py migrate with psycopg.errors.DuplicateTable: relation "django_session" already exists. This PR contains no Django migrations or model changes. The same failure reproduces on unrelated open PRs (e.g. fix(max): stop reporting benign streaming reconnect races as errors #65667 hit the identical django_session already exists error), so this is a transient/CI-infra issue in the migration job rather than anything in this diff. I re-ran the job once and it reproduced.
  • Django Tests Pass is the aggregator gate — it fails only because the migration check above failed. All 12 Django/Temporal test shards passed.

No further code changes are warranted here; the migration-job failure needs to clear at the infra level (or a fresh re-run once master CI is healthy).

Generated-By: PostHog Code
Task-Id: 42bb41c5-6cc6-4fd1-9654-af539b030e73
@talyn-app
talyn-app Bot merged commit 585a749 into master Jun 24, 2026
258 of 260 checks passed
@talyn-app
talyn-app Bot deleted the posthog-code/implement-workable-source branch June 24, 2026 10:11
@deployment-status-posthog

deployment-status-posthog Bot commented Jun 24, 2026

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-06-24 10:39 UTC Run
prod-us ✅ Deployed 2026-06-24 10:58 UTC Run
prod-eu ✅ Deployed 2026-06-24 11:02 UTC Run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants