[BUG] closed-then-reopened PRs bypass OSS credibility penalty within lookback window

[sudoHayato]

Context

Discovered during active mining research on entrius/gittensor subnet mechanics. I considered holding this as a reserve finding for strategic timing, but decided that transparent disclosure matches the contribution ethics this subnet is built to reward. Filing openly.

Summary

try_add_open_or_closed_pr in gittensor/utils/github_api_tools.py routes PRs into closed_pull_requests based on the current GraphQL snapshot (state, closedAt). When a PR transitions OPEN → CLOSED → OPEN within the scoring lookback window, GitHub clears closedAt to null on reopen. The validator performs no historical event lookup, so the snapshot reports state='OPEN' with no indication the PR was ever closed. The PR never contributes to closed_pull_requests, even when it was closed within the lookback window.

Combined with snapshot-only credibility calculation in calculate_credibility, this creates a direct bypass of the MIN_CREDIBILITY=0.80 eligibility gate.

Attack vector

A rational miner whose credibility is borderline (near 0.80) can reopen any PRs they previously closed within the 35-day lookback. Each reopen removes one entry from closed_count, pushing credibility toward 1.0. The action is a single GitHub UI click, stable across validator polling intervals, and deterministic in outcome.

Cost to the miner: each reopened PR now contributes open-PR collateral (~20% of earned-score potential per current OPEN_PR_COLLATERAL_PERCENT). This is a small fraction of what a failed eligibility gate costs (zeroes the entire OSS score).

Net: rational bypass is always economically favorable when credibility is the binding constraint. This is not a theoretical edge case — it's an attack that any miner hitting the credibility floor will discover and execute.

Reproduction

The following script reproduces the bypass deterministically on HEAD 0ecafea of the test branch:

"""Cross-file reopen bypass repro: close-and-reopen erases credibility penalty.

Pipeline under test:
1. Fetcher (github_api_tools.try_add_open_or_closed_pr) reads PR state from GraphQL
2. Routes CLOSED → miner_eval.closed_pull_requests, OPEN → miner_eval.open_pull_requests
3. credibility.calculate_credibility reads list lengths — snapshot-based, no historic tracking
4. check_eligibility gates on credibility >= 0.80

Bug: GitHub's closedAt becomes null on reopen. Fetcher has no timelineItems query
for [CLOSED_EVENT, REOPENED_EVENT] on PRs. A miner's PR that toggled
OPEN→CLOSED→OPEN loses its closed-history permanently from the validator's view.
"""
from datetime import datetime, timezone, timedelta
import os

# Disable DEV_MODE so MAINTAINER_ASSOCIATIONS filter fires normally
os.environ.pop('DEV_MODE', None)

from gittensor.classes import MinerEvaluation, PRState, PullRequest
from gittensor.utils.github_api_tools import try_add_open_or_closed_pr
from gittensor.validator.oss_contributions.credibility import (
    calculate_credibility, check_eligibility,
)

LOOKBACK_DAYS = 35
now = datetime(2026, 4, 24, 12, 0, 0, tzinfo=timezone.utc)
lookback_date_filter = now - timedelta(days=LOOKBACK_DAYS)


def mk_pr_raw(number, state, created_offset_days, closed_at, merged_at=None, author='miner_alice'):
    """Build a GraphQL-shaped raw PR dict matching fetcher expectations."""
    return {
        'number': number, 'title': f'PR #{number}',
        'state': state, 'merged': state == 'MERGED',
        'mergedAt': merged_at,
        'createdAt': (now + timedelta(days=created_offset_days)).isoformat().replace('+00:00', 'Z'),
        'closedAt': closed_at,
        'lastEditedAt': None, 'bodyText': '',
        'additions': 50, 'deletions': 5, 'commits': {'totalCount': 1},
        'repository': {'name': 'gittensor', 'owner': {'login': 'entrius'},
                       'defaultBranchRef': {'name': 'main'}},
        'headRepository': {'name': 'gittensor', 'owner': {'login': 'miner_alice'}},
        'baseRefName': 'main', 'baseRefOid': 'abc',
        'headRefName': 'feat', 'headRefOid': 'def',
        'author': {'login': author}, 'authorAssociation': 'CONTRIBUTOR',
        'mergedBy': None,
        'closingIssuesReferences': {'nodes': []}, 'reviews': {'nodes': []},
    }


def build_pr_object(pr_raw):
    """Build a minimal PR with token_score set high enough to count."""
    merged_at = None
    if pr_raw['mergedAt']:
        merged_at = datetime.fromisoformat(pr_raw['mergedAt'].replace('Z', '+00:00'))
    pr = PullRequest(
        number=pr_raw['number'], repository_full_name='entrius/gittensor',
        uid=1, hotkey='h', github_id='gh_alice',
        title=pr_raw['title'], author_login='miner_alice',
        merged_at=merged_at,
        created_at=datetime.fromisoformat(pr_raw['createdAt'].replace('Z', '+00:00')),
        pr_state=PRState(pr_raw['state']),
    )
    pr.token_score = 10.0  # above MIN_TOKEN_SCORE_FOR_BASE_SCORE
    return pr


def make_eval():
    return MinerEvaluation(uid=1, hotkey='h')


# SCENARIO 1: baseline — 5 MERGED + 3 CLOSED (never reopened)
print("SCENARIO 1: baseline — 5 MERGED + 3 CLOSED (never reopened)")
ev1 = make_eval()
for i, num in enumerate([101, 102, 103, 104, 105]):
    raw = mk_pr_raw(num, 'MERGED', created_offset_days=-20+i,
                    closed_at=None,
                    merged_at=(now + timedelta(days=-10+i)).isoformat().replace('+00:00', 'Z'))
    ev1.merged_pull_requests.append(build_pr_object(raw))

for num in [201, 202, 203]:
    raw = mk_pr_raw(num, 'CLOSED', created_offset_days=-25,
                    closed_at=(now + timedelta(days=-3)).isoformat().replace('+00:00', 'Z'))
    try_add_open_or_closed_pr(ev1, raw, 'CLOSED', lookback_date_filter)

cred1 = calculate_credibility(ev1.merged_pull_requests, ev1.closed_pull_requests)
elig1, _, reason1 = check_eligibility(ev1.merged_pull_requests, ev1.closed_pull_requests)
print(f"  merged={len(ev1.merged_pull_requests)}  closed={len(ev1.closed_pull_requests)}  open={len(ev1.open_pull_requests)}")
print(f"  credibility = {cred1:.4f}  (= 5/(5 + max(0, 3-1)) = 5/7)")
print(f"  eligible    = {elig1}  reason='{reason1}'")

# SCENARIO 2: same miner reopens all 3 CLOSED PRs — they now show as OPEN
# GitHub sets closedAt=null on reopen; fetcher has no history lookup.
print("\nSCENARIO 2: miner reopens all 3 closed PRs (closedAt becomes null)")
ev2 = make_eval()
for i, num in enumerate([101, 102, 103, 104, 105]):
    raw = mk_pr_raw(num, 'MERGED', created_offset_days=-20+i,
                    closed_at=None,
                    merged_at=(now + timedelta(days=-10+i)).isoformat().replace('+00:00', 'Z'))
    ev2.merged_pull_requests.append(build_pr_object(raw))

for num in [201, 202, 203]:
    raw = mk_pr_raw(num, 'OPEN', created_offset_days=-25,
                    closed_at=None, merged_at=None)
    try_add_open_or_closed_pr(ev2, raw, 'OPEN', lookback_date_filter)

cred2 = calculate_credibility(ev2.merged_pull_requests, ev2.closed_pull_requests)
elig2, _, reason2 = check_eligibility(ev2.merged_pull_requests, ev2.closed_pull_requests)
print(f"  merged={len(ev2.merged_pull_requests)}  closed={len(ev2.closed_pull_requests)}  open={len(ev2.open_pull_requests)}")
print(f"  credibility = {cred2:.4f}  (= 5/(5 + 0))")
print(f"  eligible    = {elig2}")

# Bypass is deterministic
assert cred1 < 0.80 and cred2 == 1.0
assert elig1 is False and elig2 is True
print("\n✓ Numerical assertions PASS — bypass is deterministic.")

Expected output:

SCENARIO 1: baseline — 5 MERGED + 3 CLOSED (never reopened)
  merged=5  closed=3  open=0
  credibility = 0.7143  (= 5/(5 + max(0, 3-1)) = 5/7)
  eligible    = False  reason='Credibility 0.71 < 0.8 minimum'

SCENARIO 2: miner reopens all 3 closed PRs (closedAt becomes null)
  merged=5  closed=0  open=3
  credibility = 1.0000  (= 5/(5 + 0))
  eligible    = True

✓ Numerical assertions PASS — bypass is deterministic.

The fetcher (try_add_open_or_closed_pr) routes the same 3 PRs to different lists based purely on current-state snapshot, with no historical closure event tracked.

Related policy precedent

The fetcher already acknowledges that closure history matters, at gittensor/utils/github_api_tools.py:815:

"Ignore stale PRs that were created before the scoring lookback window. This allows users to close old PRs without receiving a fresh credibility penalty."

This carve-out is deliberately scoped to PRs created OUTSIDE the lookback window. PRs created INSIDE the window and subsequently closed are meant to penalize credibility. Reopen within the window subverts this stated policy — the same PR receives the "stale carve-out" treatment without actually being stale.

Affected code

1. gittensor/utils/github_api_tools.py:778-820 — try_add_open_or_closed_pr routing function
2. gittensor/utils/github_api_tools.py:46-128 — PR GraphQL query (fetches state, closedAt, mergedAt, various timelineItems subqueries but no CLOSED_EVENT / REOPENED_EVENT)
3. gittensor/validator/oss_contributions/credibility.py:19-34 — calculate_credibility consumes the snapshot-based lists with no historical awareness

Suggested fix direction

Two paths, roughly equivalent effort:

Option A: Extend the PR GraphQL query to include timelineItems(itemTypes: [CLOSED_EVENT], first: 3). Wire into try_add_open_or_closed_pr so PRs with any CLOSED_EVENT within lookback route to closed_pull_requests regardless of current state.

Option B: Populate a historic_close_count_within_lookback field on the PullRequest dataclass in gittensor/classes.py from_graphql_response. Have calculate_credibility consume this field instead of (or in addition to) list lengths.

Estimated diff: 10-20 non-test lines plus tests. Incremental GraphQL budget is small since PR-node queries already include several timelineItems subqueries for other event types.

Happy to submit a PR with either fix if that helps — let me know which direction fits your architecture preferences.

Scope note

Filing as a behavior report. Whether resolution is a code change or a documentation/policy change is for maintainers to decide. Given current ecosystem pressure (cf. eed3si9n's "shutting down the goldmine", 2026-04-19, https://eed3si9n.com/shutting-down-the-goldmine/), mechanisms that let credibility state diverge from historical reality probably deserve attention.

---

Filed by sudoHayato.