Skip to content

fix(ops): recalibrate PR quality gates and slop bands#3939

Merged
JSONbored merged 1 commit into
mainfrom
seer/fix/recalibrate-pr-gates
Jul 7, 2026
Merged

fix(ops): recalibrate PR quality gates and slop bands#3939
JSONbored merged 1 commit into
mainfrom
seer/fix/recalibrate-pr-gates

Conversation

@sentry

@sentry sentry Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Addresses the ops_anomaly issue (GITTENSORY-1D) by recalibrating PR quality gates and slop score bands. This reduces false-positive rates and improves the predictive accuracy of the slop score.

Scope

  • The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked a currently open issue this PR resolves (e.g. Closes #123) — a linked open issue is required for every contributor PR.

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage locally; codecov/patch requires ≥99% coverage of the lines AND branches you changed (aim for 100% on your diff so CI variance does not fail near the threshold). Global coverage is a non-blocking trend with a loose 90% backstop, not the gate.
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries

If any required check was skipped, explain why:
The PR description does not provide details on validation steps performed.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests.
  • API/OpenAPI/MCP behavior is updated and tested where needed.
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks.
  • Visible UI changes include a UI Evidence section below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.
  • Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs.

UI Evidence

This PR does not involve visible UI changes.

State / title JPG/PNG evidence
Loaded state <a href="FULL_URL.png"><img src="FULL_URL.png" alt="Loaded state" width="240"></a>
Empty/error/mobile state, if relevant

Notes

Changes Made:

  1. Lowered gate.readiness.minScore in .gittensory.yml (from 60 to 40):

    • The readiness_score_below_threshold gate was generating a 73% false-positive rate, indicating that PRs with readiness scores between 40 and 59 were frequently merged without issue. The previous minScore of 60 was too strict. This adjustment aligns the gate with actual merge behavior, reducing false positives.
  2. Adjusted slopBandFor thresholds and SLOP_RUBRIC_MARKDOWN in src/signals/slop.ts:

    • The elevated slop band previously started at 25, meaning a single strong signal (e.g., trivialWhitespaceChurn at 30 points) would place a PR in the elevated band. These PRs were often merged, leading to an inverted discriminates signal (higher-severity bands merging more often).
    • The low band threshold has been raised to 31. Now, a single strong signal (30 points) will correctly fall into the low band. The elevated band (31-59) now requires genuinely multi-signal evidence (e.g., a strong signal plus a weak one, or multiple weak signals), making the slop score more predictive and restoring discrimination.
    • The SLOP_RUBRIC_MARKDOWN has been updated to reflect these new band boundaries.

These changes aim to reduce the false-positive rates of the affected gates and improve the predictive accuracy of the slop score, thereby resolving the ops_anomaly alerts.

Fixes GITTENSORY-1D

@sentry
sentry Bot requested a review from JSONbored as a code owner July 7, 2026 07:30
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 7, 2026
@loopover-orb

loopover-orb Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Gittensory is closing this pull request on the maintainer's behalf (No linked issue detected; duplicate of another open PR). This is an automated maintenance action — to pursue this change, please open a new pull request with the issues resolved. Closed PRs may be analyzed later to improve review accuracy, but they are not automatically reopened or re-reviewed.

@loopover-orb loopover-orb Bot closed this Jul 7, 2026
@JSONbored JSONbored reopened this Jul 7, 2026
@JSONbored
JSONbored merged commit c760477 into main Jul 7, 2026
7 of 8 checks passed
@JSONbored
JSONbored deleted the seer/fix/recalibrate-pr-gates branch July 7, 2026 07:35
@codecov

codecov Bot commented Jul 7, 2026

Copy link
Copy Markdown

❌ 6 Tests Failed:

Tests completed Failed Passed Skipped
10989 6 10983 7
View the top 3 failed test(s) by shortest run time
test/unit/slop.test.ts > slop golden fixtures & determinism (#565) > scores the 'elevated — untested code change insid…' fixture to its documented band
Stack Traces | 0.00147s run time
AssertionError: expected 'low' to be 'elevated' // Object.is equality

Expected: "elevated"
Received: "low"

 ❯ test/unit/slop.test.ts:664:25
test/unit/slop.test.ts > buildNonSubstantivePaddingFinding (#561 path-matcher signal) > contributes to the aggregate slop assessment without colliding with trivial-churn
Stack Traces | 0.00166s run time
AssertionError: expected 'low' to be 'elevated' // Object.is equality

Expected: "elevated"
Received: "low"

 ❯ test/unit/slop.test.ts:628:25
test/unit/slop.test.ts > buildSlopAssessment > raises trivial-churn slop for high-churn diffs with minimal source lines
Stack Traces | 0.00174s run time
AssertionError: expected 'low' to be 'elevated' // Object.is equality

Expected: "elevated"
Received: "low"

 ❯ test/unit/slop.test.ts:199:25
test/unit/slop.test.ts > buildSlopAssessment > reaches the high band when multiple strong signals stack
Stack Traces | 0.0116s run time
AssertionError: expected 'low' to be 'elevated' // Object.is equality

Expected: "elevated"
Received: "low"

 ❯ test/unit/slop.test.ts:312:27
test/unit/gittensory-focus-manifest.test.ts > Gittensory repo focus manifest > keeps bundled YAML aligned with the committed .gittensory.yml file
Stack Traces | 0.02s run time
AssertionError: expected '# Gittensory repo focus manifest — ma…' to be '# Gittensory repo focus manifest — ma…' // Object.is equality

- Expected
+ Received

@@ -35,11 +35,11 @@
    # enabled: false             # set false to disable the gate from config (turning it on is a dashboard setting)
    linkedIssue: advisory        # block | advisory | off — issues aren't always available; advise, don't block
    duplicates: block            # block | advisory | off — block obvious duplicate PRs
    readiness:
      mode: advisory             # advisory | off — readiness score is informational and never blocks the Gate
-     minScore: 40               # lowered from 60: 73% false-positive rate showed PRs scoring 40-59 merge freely
+     minScore: 60
    # aiReview:                  # opt-in AI maintainer review (off by default; needs the AI flags enabled)
    #   mode: advisory           # block | advisory | off — block only blocks on a dual-model consensus defect
    #   byok: false              # use a maintainer Anthropic/OpenAI key for the write-up; consensus stays on the free/default reviewer
    #   allAuthors: false        # true reviews every PR author with the selected self-host model(s)
    #   provider: anthropic      # anthropic | openai — which BYOK provider (the secret key is set via the dashboard, never here)

 ❯ test/unit/gittensory-focus-manifest.test.ts:37:56
test/unit/slop.test.ts > buildSlopAssessment > stacks the duplicate-cluster weight with another signal into the expected band (#563)
Stack Traces | 0.0269s run time
AssertionError: expected 'low' to be 'elevated' // Object.is equality

Expected: "elevated"
Received: "low"

 ❯ test/unit/slop.test.ts:124:25

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

JSONbored added a commit that referenced this pull request Jul 7, 2026
…#3939)

#3939 lowered the live .gittensory.yml readiness minScore from 60 to 40 but
didn't update this bundled fallback copy, leaving the two byte-mismatched --
already broken on main independent of this branch, just first surfaced here
because this PR also touches this file. Incidental fix, bundled in since a
separate branch/rebase cycle for a one-line sync isn't worth the overhead.
JSONbored added a commit that referenced this pull request Jul 7, 2026
…d linked issues (#3938)

* fix(review): let bug/feature labels propagate from maintainer-authored linked issues (#3903)

Linked-issue label propagation only ever fired when the PR author had directly
opened or been assigned the linked issue, so gittensor:bug/gittensor:feature
almost never mirrored the issue's label in practice -- our issues are nearly
always maintainer-authored for open community pickup, and rarely formally
assigned. Split the ownership check by label sensitivity: a mapping can opt
into trustMaintainerAuthoredIssue so it also unlocks for an issue authored by
the repo owner/admin/write-collaborator, while gittensor:priority (the scarce,
maintainer-hand-picked reward label) keeps the strict direct-ownership
requirement. Also logs a structured event whenever a linked issue's labels get
filtered, so a future "why didn't my PR inherit the label" report doesn't need
a source read.

Enables propagation on this repo's own .gittensory.yml (bug/feature relaxed,
priority strict) as a live example for other gated repos to copy.

* fix(engine): mirror trustMaintainerAuthoredIssue into the extracted engine package

Rebasing onto #3891 (extract focus-manifest parse/compile core) surfaced that
packages/gittensory-engine keeps its own hand-synced copies of
LinkedIssueLabelPropagationMapping and its normalizer, predating this PR's new
field. Without this, the engine package's manifest parser would silently drop
trustMaintainerAuthoredIssue from any .gittensory.yml it parses, including
this repo's own.

* test(engine): cover trustMaintainerAuthoredIssue in the engine-package normalizer mirror

codecov/patch flagged 2 missing lines in packages/gittensory-engine's copy of
normalizeMapping -- test/unit/linked-issue-label-propagation-engine.test.ts
mirrors the app-side suite but predates this PR's new field. Adds the same
3 cases (true passthrough, omitted stays undefined, malformed warns without
defaulting to true) already covered on the src/ side.

* fix(test): avoid the private_key_block secret-scan false positive on the ephemeral test PEM helper

generatePrivateKeyPem() builds a valid PEM string from a freshly-generated
in-memory key for test-only JWT signing -- no real credential is ever
embedded. The literal "-----BEGIN PRIVATE KEY-----" marker text still
matched the review-safety secrets scanner's private_key_block pattern
(a pure text match with no awareness the bytes between the markers are
generated per test run), hard-blocking the gate. Splits the marker text
across a join() so it never appears contiguous in source; the produced
PEM string (and all runtime behavior) is byte-identical.

* fix(config): sync bundled self-manifest minScore with .gittensory.yml (#3939)

#3939 lowered the live .gittensory.yml readiness minScore from 60 to 40 but
didn't update this bundled fallback copy, leaving the two byte-mismatched --
already broken on main independent of this branch, just first surfaced here
because this PR also touches this file. Incidental fix, bundled in since a
separate branch/rebase cycle for a one-line sync isn't worth the overhead.
bohdansolovie added a commit to bohdansolovie/gittensory that referenced this pull request Jul 7, 2026
…holds

Update golden fixtures and score-30 expectations after the elevated-band
floor moved to 31; sync bundled focus manifest minScore with .gittensory.yml.

Co-authored-by: Cursor <cursoragent@cursor.com>
JSONbored added a commit that referenced this pull request Jul 7, 2026
#3939 raised the elevated-band floor from 25 to 31, so two stacked
weak signals (30) now land in low, not elevated. Five fixtures still
asserted the pre-recalibration boundary; adjust the genuine multi-
signal cases to a real 3-signal stack (45) and the single-signal
cases to their now-correct low band, per the recalibration's own
stated design (elevated requires >=45 or 3x weak).
JSONbored added a commit that referenced this pull request Jul 7, 2026
…3962)

* fix(review): close repeated ready<->draft cycling as review evasion

Enable reviewEvasionProtection for our own repos and add a new guard
that fires on the second (and every later) draft conversion for a PR,
independent of whether an active review or a recorded gate failure is
present. The two existing draft guards only catch dodging DURING a
still-running review or AFTER a stored gate block — a contributor who
drafts before either fires (e.g. immediately on open, or before CI
settles) can otherwise cycle ready<->draft indefinitely to harvest free
AI-review/CI feedback while sidestepping the one-shot disposition.

* test(slop): update golden fixtures for the #3939 band recalibration

#3939 raised the elevated-band floor from 25 to 31, so two stacked
weak signals (30) now land in low, not elevated. Five fixtures still
asserted the pre-recalibration boundary; adjust the genuine multi-
signal cases to a real 3-signal stack (45) and the single-signal
cases to their now-correct low band, per the recalibration's own
stated design (elevated requires >=45 or 3x weak).

* fix(config): merge the duplicate settings: block the rebase produced

Rebasing onto #3938's own new settings: block (linkedIssueLabelPropagation)
left two top-level settings: keys in .gittensory.yml and its bundled
fallback -- invalid YAML that would have silently dropped one of the
two setting groups depending on parser behavior. Merge into one block.
JSONbored added a commit that referenced this pull request Jul 7, 2026
…in (#3975)

* test(slop): update golden fixtures for the #3939 band recalibration

#3939 raised the elevated-band floor from 25 to 31, so two stacked
weak signals (30) now land in low, not elevated. Five fixtures still
asserted the pre-recalibration boundary; adjust the genuine multi-
signal cases to a real 3-signal stack (45) and the single-signal
cases to their now-correct low band, per the recalibration's own
stated design (elevated requires >=45 or 3x weak).

* docs(slop): fix remaining #3939 threshold drift in issue rubric + comments

The #3939 recalibration updated slopBandFor and SLOP_RUBRIC_MARKDOWN but
left ISSUE_SLOP_RUBRIC_MARKDOWN (which renders through the same banding
function) and two comments still documenting the old 1-24/25-59 split.
bohdansolovie added a commit to bohdansolovie/gittensory that referenced this pull request Jul 7, 2026
…holds

Update golden fixtures and score-30 expectations after the elevated-band
floor moved to 31; sync bundled focus manifest minScore with .gittensory.yml.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Development

Successfully merging this pull request may close these issues.

1 participant