feat(warehouse): sync Gong call participants and CRM associations - #71046
Conversation
Add a `calls_extensive` table to the Gong data-warehouse source, backed by `POST /v2/calls/extensive`. Unlike the basic `GET /v2/calls` list that the existing `calls` table uses, the extensive endpoint returns each call's `parties` (participant name/email/affiliation) and CRM `context` (linked Salesforce/HubSpot objects and their fields), so this data now lands in the warehouse. The new table reuses the existing date-windowed, cursor-paginated, incremental-on-`started` machinery. The extensive endpoint is POST-based with its filter, content selector, and pagination cursor in a JSON body, and wraps call fields under `metaData`; those rows are flattened so `id`/`started` stay top-level for merge and partitioning while `parties`/`context` ride along as nested columns. Kept as a separate table so enabling it never changes the `calls` schema. Requires the broader `api:calls:read:extensive` scope, noted in the connection caption. Generated-By: PostHog Code Task-Id: 629c7cae-fbb5-4716-8f26-41ceb444e81e
|
Hey @simfish85! 👋 It looks like your git author email on this PR isn't your
You can fix it for this repo with: git config user.email "you@posthog.com"Or set it globally with |
PR overviewAll previously flagged issues have been addressed. No open security concerns remain on this pull request. Security reviewNo open security issues remain on this pull request. Fixed/addressed: 1 · PR risk: 0/10 |
Generated-By: PostHog Code Task-Id: 629c7cae-fbb5-4716-8f26-41ceb444e81e
Generated-By: PostHog Code Task-Id: 629c7cae-fbb5-4716-8f26-41ceb444e81e
…ture The `/v2/calls/extensive` responses carry participant names and free-form CRM field values that the name-based sample scrubbers can't recognise. Default tracked-session sampling would copy those bodies into HTTP troubleshooting storage. Add a per-endpoint `capture_http_samples` flag (default True) and set it False for `calls_extensive`, so its requests stay metered and logged but their bodies are excluded from sample capture. Basic list endpoints are unaffected. Generated-By: PostHog Code Task-Id: 629c7cae-fbb5-4716-8f26-41ceb444e81e
|
hey @Gilbert09, You have a ton of PRs waiting. writing to you here since i noticed that if you run |
Problem
PostHog's alpha Gong connector syncs four tables —
calls,users,scorecards,workspaces— none of which carry call participants or CRM associations. That's because thecallstable is fed by the basicGET /v2/callslist endpoint, which returns only flat call metadata. Gong exposes participant identities (parties) and CRM object links (context) exclusively through the separatePOST /v2/calls/extensiveendpoint, which the connector didn't call — so even with CRM association configured perfectly in Gong, there was no field to carry it into the warehouse.Why: A user wanted their Gong calls joined to CRM/account data (and participant emails) inside PostHog, and found the synced
callsrows had no association fields at all. This makes that data available.Changes
calls_extensivedata-warehouse table backed byPOST /v2/calls/extensive, returning each call'sparties(participant name/email/affiliation) and CRMcontext(linked Salesforce/HubSpot objects and fields).startedsync path. Added POST support to the fetcher (filter + content selector + cursor in the JSON body) and flattening of the extensive response'smetaDatawrapper soid/startedstay top-level for merge and partitioning whileparties/contextbecome nested columns.callsschema.api:calls:read:extensivescope in the connection caption, and adds canonical column descriptions for the new table.How did you test this code?
Added unit tests in
test_gong.pycovering the new path: thatcalls_extensiveissues a POST to/v2/calls/extensivewith the correctcontentSelector(Extended context + parties), flattensmetaDataup while preservingparties/context, and passes the pagination cursor in the body rather than the query string. Updated the endpoint-set assertion to include the new table.I (the agent) was not able to run the repo's pytest suite in this environment (dev dependencies aren't installed). I verified all edited modules byte-compile and exercised the new body-building and flattening logic in a standalone harness. The tests follow the existing
_FakeSessionpatterns in the file and should be run in CI.🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Authored by the PostHog Slack app from a support thread. The root cause was diagnosed by inspecting the connector source (basic
/v2/callsvs/v2/calls/extensive) and confirming the synced tables carry no association data. Chose to add a dedicatedcalls_extensivetable rather than switchcallsto the extensive endpoint, to avoid a breaking schema change to an existing table. No repo skills were invoked.Created with PostHog from a Slack thread