Skip to content

feat: migrate crustdata to live person enrich API (CM-1354) - #4447

Merged
skwowet merged 4 commits into
mainfrom
CM-1354/crustdata-live-enrich-migration
Aug 6, 2026
Merged

feat: migrate crustdata to live person enrich API (CM-1354)#4447
skwowet merged 4 commits into
mainfrom
CM-1354/crustdata-live-enrich-migration

Conversation

@skwowet

@skwowet skwowet commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Migrate the Crustdata member enrichment source from the legacy screener realtime endpoint to POST /person/professional_network/enrich/live.
  • Update auth (Bearer + x-api-version), credits check, response parsing, and normalizer field paths for 1:1 parity with the fields we use today (emails and org description intentionally deferred).

Changes

  • Switch enrich call to live person enrich with explicit fields (basic_profile, social_handles, professional_network, experience, education, skills).
  • Switch credits to GET /account/credits and gate on account.credits > 7.
  • Retype cached payload around nested person_data (+ CDP metadata) and type the enrich response envelope (matched_on, match_type, matches[].confidence_score).
  • Remap normalize for attributes, LinkedIn/Twitter/GitHub identities, reach, and employment (professional_network_id → LinkedIn company identity).

Copilot AI balanced review requested due to automatic review settings August 6, 2026 11:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Migrates Crustdata enrichment to the live person API and adapts normalization to its nested response schema.

Changes:

  • Updates authentication, credit checks, and enrichment requests.
  • Remaps profile, social, education, and employment data.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
types.ts Defines the new API response types.
service.ts Implements requests and normalization.
Suppressed comments (2)

services/apps/members_enrichment_worker/src/sources/crustdata/service.ts:214

  • This documents Crustdata's no-match response contract but provides no source. Link the external reference as required by the project's comment convention.
    // No match returns 200 with empty matches[].

services/apps/members_enrichment_worker/src/sources/crustdata/service.ts:326

  • These lines restate the mappings already expressed by linkedinUrl and githubUrl below. Remove them to keep the code self-explanatory without redundant comments.
    // Crustdata social_handles use generic identifiers:
    // professional_network = LinkedIn, dev_platform = GitHub.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread services/apps/members_enrichment_worker/src/sources/crustdata/service.ts Outdated
Comment thread services/apps/members_enrichment_worker/src/sources/crustdata/service.ts Outdated
Copilot AI review requested due to automatic review settings August 6, 2026 11:49
skwowet added 3 commits August 6, 2026 17:21
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
@skwowet
skwowet force-pushed the CM-1354/crustdata-live-enrich-migration branch from 1a22a8e to 30681a1 Compare August 6, 2026 11:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Copilot AI review requested due to automatic review settings August 6, 2026 11:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (2)

services/apps/members_enrichment_worker/src/sources/crustdata/types.ts:28

  • The live payload keeps dev_platform_identifier present with profile_url: null when no developer profile is found. Model that nullable value explicitly so the declared response/cache shape matches what callers actually receive.
      profile_url?: string

services/apps/members_enrichment_worker/src/sources/crustdata/types.ts:8

  • Live enrich responses use null for an ongoing role's end_date, not just an omitted value. Excluding null makes the API/cache contract inaccurate and lets callers assume a present value is always a string; include it explicitly.

This issue also appears on line 28 of the same file.

  end_date?: string

Copilot AI review requested due to automatic review settings August 6, 2026 14:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

@skwowet
skwowet merged commit 858d121 into main Aug 6, 2026
14 checks passed
@skwowet
skwowet deleted the CM-1354/crustdata-live-enrich-migration branch August 6, 2026 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants