Skip to content

chore: refactor sync insights project to common layer and use it in nango worker as well (CM-786)#3628

Merged
themarolt merged 3 commits into
mainfrom
insights-github-sync-CM-786
Nov 24, 2025
Merged

chore: refactor sync insights project to common layer and use it in nango worker as well (CM-786)#3628
themarolt merged 3 commits into
mainfrom
insights-github-sync-CM-786

Conversation

@themarolt
Copy link
Copy Markdown
Contributor

@themarolt themarolt commented Nov 21, 2025

Note

Moves repository-to-insights syncing and segment repo queries into common services and the data-access layer, updates backend to use them, and adds a nango worker activity to trigger sync after GitHub processing.

  • Common Services:
    • Add CommonIntegrationService.syncGithubRepositoriesToInsights(...) to upsert segmentRepositories, prune stale mappings, and update insightsProject.repositories.
    • Enhance settings decryption; add Redis dependency.
  • Data Access Layer (DAL):
    • Integrations: add findRepositoriesForSegment(...), findNangoRepositoriesToBeRemoved(...), plain repo unmap helpers, and include segmentId in fetchIntegrationById.
    • Segments: add getGithubMappedRepos(...), getGitlabMappedRepos(...), and cross-segment URL checks.
    • Utils: standardize updatedAt to now().
  • Backend:
    • API GET /segments/{id}/repositories: switch to DAL findRepositoriesForSegment.
    • IntegrationService: replace bespoke repo sync with CommonIntegrationService.syncGithubRepositoriesToInsights(...); use DAL for mapped repo queries; remove legacy segment-repo sync logic.
    • Remove segment repo query methods from SegmentRepository now provided by DAL.
  • Nango Worker:
    • Add syncGithubReposToInsights activity and call it post GitHub connection processing to persist repositories to insights.

Written by Cursor Bugbot for commit 41808d4. This will update automatically on new commits. Configure here.

…ango worker as well

Signed-off-by: Uroš Marolt <uros@marolt.me>
@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

1 similar comment
@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

@themarolt themarolt changed the title chore: refactor sync insights project to common layer and use it in nango worker as well chore: refactor sync insights project to common layer and use it in nango worker as well (CM-786) Nov 21, 2025
@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is being reviewed by Cursor Bugbot

Details

Your team is on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle for each member of your team.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

Comment thread services/libs/data-access-layer/src/segments/index.ts
Comment thread services/libs/data-access-layer/src/segments/index.ts
Comment thread services/libs/data-access-layer/src/integrations/index.ts
@themarolt themarolt requested a review from mbani01 November 21, 2025 18:38
Copy link
Copy Markdown
Contributor

@mbani01 mbani01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Perhaps @ulemons can double-check since he got more context on segmentRepositories just in case

Comment thread services/libs/data-access-layer/src/integrations/index.ts Outdated
@ulemons
Copy link
Copy Markdown
Contributor

ulemons commented Nov 24, 2025

LGTM ✅

@themarolt themarolt merged commit 96ba7de into main Nov 24, 2025
16 checks passed
@themarolt themarolt deleted the insights-github-sync-CM-786 branch November 24, 2025 10:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants