fix: block stale cache fallback on GraphQL PAT auth failures

[bitloi]

Summary

Fixes stale cache fallback when a stored miner PAT passes REST /user validation but fails the PR GraphQL query used during scoring with an auth/scope error.

Previously, load_miners_prs() marked these GraphQL failures with github_pr_fetch_failed=True but left failed_reason=None. Since cache fallback only skips miners with failed_reason, validators could restore a previous successful cached evaluation and continue rewarding stale PRs even though the current PAT no longer had the GraphQL access required for scoring.

This change detects clear GraphQL PAT auth/scope errors, such as Resource not accessible by personal access token, marks the evaluation with a non-cacheable failed_reason, and preserves cache fallback for transient/non-auth GraphQL failures.

Related Issues

Closes #904

Type of Change

• Bug fix
• New feature
• Refactor
• Documentation
• Other (describe below)

Testing

• Tests added/updated
• Manually tested

Commands run:

uv run pytest tests/utils/test_github_api_tools.py tests/validator/test_validator_cache_fallback.py -q
uv run pytest tests/ -q
uv run ruff check gittensor/utils/github_api_tools.py tests/utils/test_github_api_tools.py tests/validator/test_validator_cache_fallback.py
uv run ruff format --check gittensor/utils/github_api_tools.py tests/utils/test_github_api_tools.py tests/validator/test_validator_cache_fallback.py
uv run pyright gittensor/utils/github_api_tools.py tests/utils/test_github_api_tools.py tests/validator/test_validator_cache_fallback.py
uv run pre-commit run --files gittensor/utils/github_api_tools.py tests/utils/test_github_api_tools.py tests/validator/test_validator_cache_fallback.py

Checklist

• Code follows project style guidelines
• Self-review completed
• Changes are documented (if applicable)

[anderdc]

Right bug, but the marker list is the brittle part — GitHub can rephrase any of those five strings and a real auth failure leaks back into cache fallback.

Simpler and more robust: in the non_resource_errors branch at gittensor/utils/github_api_tools.py:929, just always set failed_reason. Something like:

if non_resource_errors:
    bt.logging.error(f'GraphQL errors: {non_resource_errors}')
    miner_eval.failed_reason = f'GitHub GraphQL error: {non_resource_errors[0].get("message", "unknown")[:200]}'
    miner_eval.github_pr_fetch_failed = True
    break

This branch only fires when GitHub returns 200 with errors that aren't RESOURCE_LIMITS_EXCEEDED. Every other transient class — network exceptions, 5xx, rate limits — hits a different branch and keeps cache fallback. So the GraphQL-errors-array path is overwhelmingly auth / scope / schema / permission failures, none of which should restore stale cached scores. A genuinely transient INTERNAL_SERVER_ERROR from GitHub eats one round at 0, which is recoverable next round.

Drop the marker tuple, the helper, and the auth-specific test. Keep the integration test in test_validator_cache_fallback.py but assert it works for any GraphQL error message, not just the auth phrasing.

[bitloi]

Right bug, but the marker list is the brittle part — GitHub can rephrase any of those five strings and a real auth failure leaks back into cache fallback.

Simpler and more robust: in the non_resource_errors branch at gittensor/utils/github_api_tools.py:929, just always set failed_reason. Something like:

if non_resource_errors:
    bt.logging.error(f'GraphQL errors: {non_resource_errors}')
    miner_eval.failed_reason = f'GitHub GraphQL error: {non_resource_errors[0].get("message", "unknown")[:200]}'
    miner_eval.github_pr_fetch_failed = True
    break

This branch only fires when GitHub returns 200 with errors that aren't RESOURCE_LIMITS_EXCEEDED. Every other transient class — network exceptions, 5xx, rate limits — hits a different branch and keeps cache fallback. So the GraphQL-errors-array path is overwhelmingly auth / scope / schema / permission failures, none of which should restore stale cached scores. A genuinely transient INTERNAL_SERVER_ERROR from GitHub eats one round at 0, which is recoverable next round.

Drop the marker tuple, the helper, and the auth-specific test. Keep the integration test in test_validator_cache_fallback.py but assert it works for any GraphQL error message, not just the auth phrasing.

@anderdc Updated. Non-resource GraphQL errors now always set failed_reason, and tests no longer depend on auth-specific wording.