Skip to content

fix: deterministic package repo tie-break - #4263

Merged
epipav merged 6 commits into
mainfrom
fix/package-repo-tiebreak
Aug 6, 2026
Merged

fix: deterministic package repo tie-break#4263
epipav merged 6 commits into
mainfrom
fix/package-repo-tiebreak

Conversation

@epipav

@epipav epipav commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Note

Low Risk
Narrow query-ordering change; may shift which repo is shown when ties exist, but does not alter auth, writes, or broader business logic.

Overview
When a package maps to multiple repos with the same confidence, which repo wins was undefined, so scorecard-driven health, filters, and detail views could flip between runs.

Postgres (osspckgs/api.ts): Every lateral that picks one package_repos row for scorecard (status counts, package list, scatter) now sorts by confidence DESC then repo_id DESC. Package detail keeps its existing declared preference and adds repo_id DESC after that.

Tinybird (ossPackages_enriched.pipe): The per-package argMax(repoId, …) tie-break tuple now includes repoId after confidence and verifiedAt, matching the same deterministic rule for enriched health scores.

Reviewed by Cursor Bugbot for commit ecab2d3. Bugbot is set up for automated code reviews on this repo. Configure here.

Signed-off-by: anilb <epipav@gmail.com>
@epipav
epipav requested review from Copilot and ulemons and removed request for Copilot June 25, 2026 08:19
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

Copilot AI balanced review requested due to automatic review settings June 25, 2026 08:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to make “winner repo” selection deterministic when a package maps to multiple repositories with tied confidence, so Postgres-derived API responses and Tinybird-enriched health/scorecard data don’t fluctuate run-to-run.

Changes:

  • Postgres: adds repo_id DESC as a deterministic tie-breaker to several LIMIT 1 repo-selection laterals in services/libs/data-access-layer/src/osspckgs/api.ts.
  • Tinybird: adds repoId as a final component in the argMax tuple for per-package repo selection in ossPackages_enriched.pipe.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
services/libs/data-access-layer/src/osspckgs/api.ts Adds deterministic ordering for selecting a single repo per package in multiple Postgres queries.
services/libs/tinybird/pipes/ossPackages_enriched.pipe Adds a final tie-break key to Tinybird argMax selection to make repo choice deterministic under ties.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread services/libs/data-access-layer/src/osspckgs/api.ts Outdated
Comment thread services/libs/data-access-layer/src/osspckgs/api.ts Outdated
Comment thread services/libs/data-access-layer/src/osspckgs/api.ts Outdated
Comment thread services/libs/data-access-layer/src/osspckgs/api.ts Outdated
ulemons
ulemons previously approved these changes Jun 25, 2026
@joanagmaia

Copy link
Copy Markdown
Contributor

@epipav if this is already in production can we merge it? Some Tinybird PRs are becoming stale, and might lead to confusion on what's in production or not. Thanks!

…reak

Signed-off-by: anilb <epipav@gmail.com>

# Conflicts:
#	services/libs/data-access-layer/src/osspckgs/api.ts
Copilot AI review requested due to automatic review settings August 6, 2026 08:35
@cursor

cursor Bot commented Aug 6, 2026

Copy link
Copy Markdown

PR Summary

Low Risk
Query-only ordering changes; primary ranking (confidence, declared source) is unchanged, with a stable final tie-break that may shift a small set of ambiguous mappings.

Overview
When several package_repos rows tie on confidence (and sometimes source), the chosen repo could vary between queries or runs. This PR standardizes that pick everywhere the “best repo” is resolved.

Postgres: ORDER BY for best-repo laterals now ends with repo_id DESC. Shared BEST_REPO_LINK_JOIN in sqlFragments.ts is updated so package detail, contacts, and related APIs stay consistent. Several scorecard-only laterals that previously ordered only by confidence DESC now also prefer declared source, then repo_id, matching the canonical rule. The security-contacts on-demand ingest path uses the same ordering so ingest targets the repo the API shows.

Tinybird: ossPackages_enriched_pkg_repo extends the argMax sort key with repoId so enriched analytics pick the same repo on ties as Postgres.

Reviewed by Cursor Bugbot for commit dc22458. Bugbot is set up for automated code reviews on this repo. Configure here.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (3)

services/libs/data-access-layer/src/osspckgs/api.ts:524

  • The equivalent scorecard lateral in stewardships.ts:626-634 still orders only by confidence. listMyPackages uses that selected scorecard for its health value, filter, and sort, so equal-confidence mappings can still flip between runs despite the PR's stated scope. Please add the same repo_id DESC tie-break there.
      ORDER BY pr.confidence DESC, pr.repo_id DESC

services/libs/data-access-layer/src/osspckgs/api.ts:343

  • No DAL test currently calls these scorecard-based status/list/scatter functions with equal-confidence repo mappings, so the regression this change targets is unverified. Please add coverage that creates two tied mappings and asserts the higher repo_id supplies the selected scorecard.

This issue also appears on line 524 of the same file.

      ORDER BY pr.confidence DESC, pr.repo_id DESC

services/libs/data-access-layer/src/osspckgs/sqlFragments.ts:27

  • The existing getPackageDetailsByPurls integration test inserts only one repo mapping, so it does not exercise either tie-break added here. Please add tied mappings that verify a declared source wins first and the higher repo_id wins when source and confidence are equal.
      ORDER BY pr2.confidence DESC, (pr2.source = 'declared') DESC, pr2.repo_id DESC

@epipav

epipav commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread services/libs/data-access-layer/src/osspckgs/api.ts Outdated
Copilot AI review requested due to automatic review settings August 6, 2026 10:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (4)

services/libs/data-access-layer/src/osspckgs/api.ts:524

  • This package-list scorecard selector now prefers a declared link over a higher repo_id on equal confidence, contrary to the stated confidence DESC, repo_id DESC rule. That is a behavioral change beyond resolving nondeterminism and can alter displayed health and filtering.
      ORDER BY pr.confidence DESC, (pr.source = 'declared') DESC, pr.repo_id DESC

services/libs/data-access-layer/src/osspckgs/api.ts:979

  • The scatter scorecard lookup should add only the deterministic repo_id tie-break described by the PR. The inserted declared-source preference can select a lower-ID repository and therefore changes scatter health data rather than merely stabilizing it.
      ORDER BY pr.confidence DESC, (pr.source = 'declared') DESC, pr.repo_id DESC

services/libs/data-access-layer/src/osspckgs/api.ts:343

  • The PR says scorecard selectors should keep confidence as the only preference and use repo_id to break ties. Placing source = 'declared' first changes status counts whenever equally confident links have different sources, instead of only making the existing choice deterministic.

This issue also appears in the following locations of the same file:

  • line 524
  • line 979
      ORDER BY pr.confidence DESC, (pr.source = 'declared') DESC, pr.repo_id DESC

services/libs/data-access-layer/src/osspckgs/stewardships.ts:632

  • This scorecard lookup previously ranked links only by confidence, so inserting a declared-source preference changes which repository supplies health data when confidence ties. Keep the existing semantics and add repo_id directly as the deterministic tie-break.
      ORDER BY pr.confidence DESC, (pr.source = 'declared') DESC, pr.repo_id DESC

@epipav

epipav commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit dc22458. Configure here.

@epipav
epipav merged commit 1a92550 into main Aug 6, 2026
16 checks passed
@epipav
epipav deleted the fix/package-repo-tiebreak branch August 6, 2026 10:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants