feat(miners): per-repo since windows on pulls/issues endpoints#109
Open
anderdc wants to merge 1 commit into
Open
feat(miners): per-repo since windows on pulls/issues endpoints#109anderdc wants to merge 1 commit into
anderdc wants to merge 1 commit into
Conversation
Adds POST /api/v1/miners/:githubId/pulls and .../issues that accept a
{ since_by_repo: { "<owner/repo>": "<ISO timestamp>" } } body and
window each repository to its own since. The existing scalar-since GET
endpoints are unchanged — NestJS routes GET and POST separately.
Server-side: a windows CTE unnests the per-repo (repo, since) pairs and
INNER-JOINs the PR/issue query, reusing the existing per-state column
logic (OPEN/CLOSED -> created_at/closed_at, MERGED -> merged_at). The
JOIN restricts results to the named repos. The SELECT column lists are
extracted to shared consts so the GET and POST queries cannot drift.
No DB schema change.
anderdc
pushed a commit
to entrius/gittensor
that referenced
this pull request
May 17, 2026
The validator sends each repo's pr_lookback_days as a since_by_repo map to the mirror's new POST endpoints, which window each repo server-side. This replaces the client-side workaround that fetched the widest window and re-filtered every PR/issue locally. - MirrorClient.get_miner_pulls/get_miner_issues take since_by_repo; a non-empty map POSTs the per-repo windows, an omitted/empty map keeps the plain GET (the unbounded open-issue-count call). _get and _post share one retry loop. - load.py / scan.py build the map from master_repositories and drop the downstream re-filter (_maybe_add_pr window checks, _issue_within_window, the max() widest-window logic). Requires the das-github-mirror POST endpoints (entrius/das-github-mirror#109) deployed first.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds per-repo time-windowing to the miner PR/issue endpoints. New
POSTvariants of/api/v1/miners/:githubId/pullsand.../issuesaccept a{ "since_by_repo": { "<owner/repo>": "<ISO timestamp>" } }body and window each repository to its ownsince.Motivation: the gittensor validator is gaining per-repository
pr_lookback_days. One mirror query per miner spans all repos, so a single scalarsincecan't express per-repo windows. ThePOSTvariant lets the validator send one request per miner with each repo's own cutoff, instead of over-fetching the widest window and re-filtering client-side.Backwards compatibility
The existing scalar-
sinceGETendpoints are untouched — same path; NestJS routesGETandPOSTseparately. Safe to deploy ahead of the gittensor change.Implementation
miners.service.ts:getPullRequestsByRepo/getIssuesByRepo— awindowsCTEunnests the parallel(repo, since)arrays and INNER-JOINs the query, reusing the GET's per-state column logic (OPEN/CLOSED →created_at/closed_at, MERGED →merged_at). The SELECT column lists are extracted to shared consts so GET and POST cannot drift. POST responses setsince: null.miners.controller.ts: two@Posthandlers + aparseSinceByRepovalidator (rejects empty/oversized maps, bad repo keys, unparseable dates, post-lowercase duplicate keys).No DB schema / migration change — pure read path;
unnestis core Postgres.Verification
nest build,eslint,prettierclean. Manual smoke recommended: POST with a 2-repo map at differingsincereturns only those repos, each windowed; malformed bodies → 400; the GET path is unchanged.