Skip to content

chore: deprecate insightsProject.repos from remaining pipes [CM-915]#3819

Merged
mbani01 merged 1 commit intomainfrom
chore/deprecate_insightsProjectRepos_from_tb_pipes
Feb 3, 2026
Merged

chore: deprecate insightsProject.repos from remaining pipes [CM-915]#3819
mbani01 merged 1 commit intomainfrom
chore/deprecate_insightsProjectRepos_from_tb_pipes

Conversation

@mbani01
Copy link
Copy Markdown
Contributor

@mbani01 mbani01 commented Feb 3, 2026

This pull request updates several SQL pipes in the services/libs/tinybird/pipes directory to improve data integrity and filtering for project repositories. The main changes ensure that only enabled and non-deleted repositories are included, and they clarify joins and field references for better consistency across pipes.

Repository Filtering and Join Improvements:

  • Updated all relevant pipes to join insightsProjects and repositories using explicit ON r.insightsProjectId = i.id syntax, ensuring correct project-repository associations. [1] [2] [3]
  • Added filters to exclude deleted projects and repositories (isNull(i.deletedAt) and isNull(r.deletedAt)) and to include only enabled repositories (r.enabled = true). [1] [2] [3]

Field Reference Updates:

  • Changed repository field references from repo to r.url for clarity and consistency. Also updated output field names from repo to repository where applicable. [1] [2] [3]

Parameter Filtering Consistency:

  • Updated parameter filters to use the new field references (r.url instead of repo) in conditional logic for project and repository selection.

These changes help ensure that queries only return relevant, active repositories and projects, and that joins are performed correctly for more reliable data.


Note

Medium Risk
Query semantics change to exclude disabled/deleted repositories and to use a different join path, which may change counts and results in downstream metrics/copies if data is inconsistent or expectations relied on the old array field.

Overview
Updates several Tinybird pipes to stop reading repository URLs via the deprecated insightsProjects.repos array and instead join insightsProjects to repositories via r.insightsProjectId = i.id.

Tightens repository selection by filtering out deleted projects/repos and excluding disabled repos (isNull(i.deletedAt), isNull(r.deletedAt), r.enabled = true), and updates downstream filters/fields to use r.url consistently.

Written by Cursor Bugbot for commit d831a09. This will update automatically on new commits. Configure here.

@mbani01 mbani01 requested a review from gaspergrom February 3, 2026 11:47
@mbani01 mbani01 self-assigned this Feb 3, 2026
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

isNull (deletedAt)
isNull(i.deletedAt)
AND isNull(r.deletedAt)
AND r.enabled = true
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing excluded filter when joining repositories table

Medium Severity

The new queries joining the repositories table are missing the r.excluded = false filter. The repositories table has both excluded and enabled fields with distinct meanings. The established pattern in segments_filtered.pipe (the core project context provider) and activityRelations_bucket_clean_enrich_copy_pipe_*.pipe files includes all three filters: isNull(r.deletedAt), r.excluded = false, and r.enabled = true. Without the excluded filter, these queries may return repositories that are explicitly marked as excluded from processing.

Additional Locations (2)

Fix in Cursor Fix in Web

@mbani01 mbani01 merged commit cd2abde into main Feb 3, 2026
18 checks passed
@mbani01 mbani01 deleted the chore/deprecate_insightsProjectRepos_from_tb_pipes branch February 3, 2026 11:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants