Skip to content

feat(cli): add submissions and predict commands for merge predictions - #226

Closed
fury0928 wants to merge 13 commits into
entrius:testfrom
fury0928:feat/cli-submissions-predict
Closed

feat(cli): add submissions and predict commands for merge predictions#226
fury0928 wants to merge 13 commits into
entrius:testfrom
fury0928:feat/cli-submissions-predict

Conversation

@fury0928

@fury0928 fury0928 commented Feb 21, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds two new miner-facing CLI commands for the merge prediction feature (Issue #223):

  • gitt issues submissions --id <N> — Lists open PRs for a bountied issue. Reads on-chain issue data, fetches PRs via GitHub GraphQL cross-references (same methodology as issue_competitions). Falls back to unauthenticated REST API when GITTENSOR_MINER_PAT is not set. Displays a Rich table (PR #, title, author, created date, review status, URL). Supports --json for scripting.
  • gitt issues predict --id <N> — Lets a miner assign merge probabilities to open PRs. Three input modes: interactive TTY prompts, --pr N --probability F, or --json-input '{"101": 0.85}'. Validates probabilities in [0,1], sum ≤ 1.0, and PR existence. Loads wallet, verifies hotkey registration on the metagraph (netuid read from contract). Supports --json output for scripting. Broadcast is stubbed with TODO — prints validated payload.

Design decisions

  • Zero local helpers in command filesubmissions.py follows the same flat structure as view.py and mutations.py (only imports + Click commands). All shared helpers live in helpers.py.
  • 5-phase fail-fast orderingpredict validates flags and ranges (Phase 1: zero network I/O), then reads on-chain + GitHub data (Phase 2), collects predictions (Phase 3), loads wallet (Phase 4: expensive, last), then outputs (Phase 5). Users with typos in flags get instant feedback.
  • Early flag-conflict validation — Mutually exclusive flags (--pr + --json-input), missing counterparts (--pr without --probability), and out-of-range probabilities are all rejected before any network calls.
  • Error handling matches mutations.py — Single try block with except ImportErrorexcept ClickException: raiseexcept Exception (no silent swallowing).
  • Cascading REST fallbackfind_prs_for_issue() cascades: GraphQL → authenticated REST → unauthenticated REST. Fine-grained PATs can filter out cross-reference timeline events, so the function retries with progressively less authentication until PRs are found. Each step logs via bt.logging.debug().
  • Graceful GitHub API errors — Both submissions and predict wrap the PR fetch in try/except, showing a yellow warning (submissions) or ClickException (predict) instead of crashing on API failures.
  • SRP-extracted helpersverify_miner_registration(), build_prediction_payload(), and get_github_pat() are extracted into helpers.py for single-responsibility and testability. Each has its own test class.
  • Ghost/deleted account handling — Null authors (deleted GitHub accounts) default to 'ghost' matching GitHub's convention. Both GraphQL and REST paths use or {} / or 'ghost' patterns. Tested end-to-end (table, JSON, database_id).
  • No behavioral regression — Refactored find_solver_from_cross_references() preserves mergedAt sorting and all bt.logging calls.

Shared infrastructure

  • find_prs_for_issue() in github_api_tools.py — modular PR discovery via GraphQL timeline + cascading REST fallback, with PRInfo TypedDict and Literal type for state_filter
  • _resolve_pr_state() — shared state resolution between GraphQL and REST paths
  • read_netuid_from_contract() in helpers.py — reads netuid from contract packed storage with graceful fallback
  • fetch_issue_from_contract() — reads issue from contract, validates existence and status (require_active kwarg for predict)
  • collect_predictions() — collects predictions from 3 input modes (--json-input, --pr/--probability, interactive TTY)
  • verify_miner_registration() — loads wallet, connects subtensor, reads netuid from contract, verifies hotkey in metagraph. Structured error handling for ImportError, wallet load failure, and unregistered hotkey.
  • build_prediction_payload() — constructs the prediction payload dict with string-keyed predictions for JSON serialization
  • get_github_pat() — centralized PAT retrieval from environment (empty string → None)
  • build_pr_table(), format_pred_lines(), validate_predictions() — display and validation helpers
  • Refactored find_solver_from_cross_references() to use find_prs_for_issue() internally (eliminates redundant GraphQL query)

Payload shape (for future synapse)

{
    "issue_id": 1,
    "repository": "owner/repo",
    "issue_number": 42,
    "miner_hotkey": "5Fake...",
    "predictions": {"123": 0.7, "456": 0.2}
}

Related Issues

Closes #223

Type of Change

  • New feature
  • Bug fix
  • Refactor
  • Documentation
  • Other

CLI Output (finney)

1. submissions --help

gitt issues submissions --help
Usage: gitt issues submissions [OPTIONS]

  List open pull requests for an issue bounty.

  Shows PRs that reference the issue, filtered to open PRs only. Uses
  GITTENSOR_MINER_PAT for authenticated GitHub API access (optional).

  Examples:
      gitt issues submissions --id 1
      gitt i submissions --id 1 --json
      gitt i submissions --id 1 --network test

Options:
  --id INTEGER                    On-chain issue ID to view submissions for  [required]
  -n, --network [finney|test|local]
                                  Network (finney/test/local)
  --rpc-url TEXT                  Subtensor RPC endpoint (overrides --network)
  --contract TEXT                 Contract address (uses default if empty)
  -v, --verbose                   Show debug output
  --json                          Output as JSON for scripting
  --help                          Show this message and exit.

2. predict --help

gitt issues predict --help
Usage: gitt issues predict [OPTIONS]

  Submit a prediction on which PR will solve an issue bounty.

  Predictions assign probabilities (0.0-1.0) to open PRs. The sum of all
  probabilities for an issue must not exceed 1.0.

  Three input modes:
    1. --pr N --probability F  (single prediction)
    2. --json-input '{"101": 0.85}'  (batch predictions)
    3. Interactive prompt (default, requires TTY)

  Examples:
      gitt issues predict --id 1 --pr 123 --probability 0.7 -y
      gitt issues predict --id 1 --json-input '{"123": 0.5, "456": 0.3}' -y
      gitt issues predict --id 1

Options:
  --id INTEGER                    On-chain issue ID to predict for  [required]
  --pr INTEGER                    PR number to predict (use with --probability)
  --probability FLOAT             Probability for the PR (0.0 to 1.0, use with --pr)
  --json-input TEXT               JSON dict of predictions: '{"101": 0.85, "102": 0.15}'
  -y, --yes                       Skip confirmation prompt
  --wallet-name, --wallet.name, --wallet TEXT
                                  Wallet name
  --wallet-hotkey, --wallet.hotkey, --hotkey TEXT
                                  Hotkey name
  -n, --network [finney|test|local]
                                  Network (finney/test/local)
  --rpc-url TEXT                  Subtensor RPC endpoint (overrides --network)
  --contract TEXT                 Contract address (uses default if empty)
  -v, --verbose                   Show debug output
  --json                          Output as JSON for scripting
  --help                          Show this message and exit.

3. Submissions — open PRs listed

gitt i submissions --id 2 --network finney
Network: finney • Contract: 5FWNdk8YNtNc...ew3MrD

Open PRs for Issue #2 (entrius/gittensor#223)

┏━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃ PR # ┃ Title            ┃ Author    ┃ Created    ┃ Review ┃ URL              ┃
┡━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│    1 │ feat(cli): add   │ MkDev11   │ 2026-02-20 │   -    │ https://github.… │
│      │ submissions and  │           │            │        │                  │
│      │ predict commands │           │            │        │                  │
│      │ for merge        │           │            │        │                  │
│      │ predictions      │           │            │        │                  │
│      │ (Closes #223)    │           │            │        │                  │
│  224 │ feat(cli): add   │ MkDev11   │ 2026-02-21 │   -    │ https://github.… │
│      │ submissions and  │           │            │        │                  │
│      │ predict commands │           │            │        │                  │
│      │ for merge        │           │            │        │                  │
│      │ predictions      │           │            │        │                  │
│  226 │ feat(cli): add   │ eureka928 │ 2026-02-21 │   -    │ https://github.… │
│      │ submissions and  │           │            │        │                  │
│      │ predict commands │           │            │        │                  │
│      │ for merge        │           │            │        │                  │
│      │ predictions      │           │            │        │                  │
└──────┴──────────────────┴───────────┴────────────┴────────┴──────────────────┘

Showing 3 open PR(s)
image

4. Submissions — JSON output

gitt i submissions --id 2 --network finney --json
[
  {
    "number": 1,
    "title": "feat(cli): add submissions and predict commands for merge predictions (Closes #223)",
    "author": "MkDev11",
    "state": "OPEN",
    "created_at": "2026-02-20T23:59:50Z",
    "merged_at": null,
    "url": "https://github.com/MkDev11/gittensor/pull/1",
    "review_status": null,
    "closes_issue": false
  },
  {
    "number": 224,
    "title": "feat(cli): add submissions and predict commands for merge predictions",
    "author": "MkDev11",
    "state": "OPEN",
    "created_at": "2026-02-21T00:00:12Z",
    "merged_at": null,
    "url": "https://github.com/entrius/gittensor/pull/224",
    "review_status": null,
    "closes_issue": false
  },
  {
    "number": 226,
    "title": "feat(cli): add submissions and predict commands for merge predictions",
    "author": "eureka928",
    "state": "OPEN",
    "created_at": "2026-02-21T07:24:42Z",
    "merged_at": null,
    "url": "https://github.com/entrius/gittensor/pull/226",
    "review_status": null,
    "closes_issue": false
  }
]
image

5. Submissions — completed issue warning

gitt i submissions --id 1 --network finney
Network: finney • Contract: 5FWNdk8YNtNc...ew3MrD

Warning: Issue 1 has status "Completed".
Open PRs for Issue #1 (entrius/gittensor#210)

No open PRs found for this issue.
GitHub issue: https://github.com/entrius/gittensor/issues/210

6. Predict — single prediction

gitt i predict --id 2 --pr 226 --probability 0.7 -y --network finney --wallet <wallet> --hotkey <hotkey>
Network: finney • Contract: 5FWNdk8YNtNc...ew3MrD

Miner hotkey: 5EyipBsWcWJ7hExxuZCS2SvAcfN37UQC4kFo2Ds3WMcQAXHS
╭──────────────────────────── Prediction Submitted ────────────────────────────╮
│ Prediction recorded (local-only)                                             │
│                                                                              │
│ Issue: #2 (entrius/gittensor#223)                                            │
│ Miner: 5EyipBsWcWJ7hExxuZCS2SvAcfN37UQC4kFo2Ds3WMcQAXHS                    │
│ Predictions:                                                                 │
│   PR #226: 70.00%                                                            │
│ Total: 70.00%                                                                │
╰──────────────────────────────────────────────────────────────────────────────╯
Note: Network broadcast is not yet implemented (TODO).

7. Predict — batch prediction via --json-input

gitt i predict --id 2 --json-input '{"224": 0.5, "226": 0.3}' -y --network finney --wallet <wallet> --hotkey <hotkey>
Network: finney • Contract: 5FWNdk8YNtNc...ew3MrD

Miner hotkey: 5EyipBsWcWJ7hExxuZCS2SvAcfN37UQC4kFo2Ds3WMcQAXHS
╭──────────────────────────── Prediction Submitted ────────────────────────────╮
│ Prediction recorded (local-only)                                             │
│                                                                              │
│ Issue: #2 (entrius/gittensor#223)                                            │
│ Miner: 5EyipBsWcWJ7hExxuZCS2SvAcfN37UQC4kFo2Ds3WMcQAXHS                    │
│ Predictions:                                                                 │
│   PR #224: 50.00%                                                            │
│   PR #226: 30.00%                                                            │
│ Total: 80.00%                                                                │
╰──────────────────────────────────────────────────────────────────────────────╯
Note: Network broadcast is not yet implemented (TODO).

8. Predict — JSON output

gitt i predict --id 2 --pr 226 --probability 0.7 -y --json --network finney --wallet <wallet> --hotkey <hotkey>
{
  "issue_id": 2,
  "repository": "entrius/gittensor",
  "issue_number": 223,
  "miner_hotkey": "5EyipBsWcWJ7hExxuZCS2SvAcfN37UQC4kFo2Ds3WMcQAXHS",
  "predictions": {
    "226": 0.7
  }
}

9. Predict — non-active issue rejected

gitt i predict --id 1 --pr 226 --probability 0.7 -y --network finney --wallet <wallet> --hotkey <hotkey>
Network: finney • Contract: 5FWNdk8YNtNc...ew3MrD

Error: Issue 1 has status "Completed" — predictions require Active status.

10. Predict — PR not found

gitt i predict --id 2 --pr 999 --probability 0.7 -y --network finney --wallet <wallet> --hotkey <hotkey>
Network: finney • Contract: 5FWNdk8YNtNc...ew3MrD

Error: PR #999 is not an open PR for this issue. Open PRs: [1, 224, 226]

11. Predict — probability out of range

gitt i predict --id 2 --pr 226 --probability 1.5 -y --network finney --wallet <wallet> --hotkey <hotkey>
Network: finney • Contract: 5FWNdk8YNtNc...ew3MrD

Error: Probability for PR #226 must be between 0.0 and 1.0 (got 1.5).

12. Validation — invalid issue ID

gitt i submissions --id 0 --network finney
Error: issue_id must be between 1 and 999999 (got 0)

13. Validation — missing PAT

gitt i predict --id 2 --pr 1 --probability 0.5 -y --network finney
Error: GITTENSOR_MINER_PAT environment variable is required for predict.

Testing

  • 183 tests in tests/cli/test_submissions.py across 20 test classes:
    • TestResolvePrState (8) — shared state normalization, merged flag, edge cases
    • TestValidateIssueId (6) — boundary values, custom param name
    • TestBuildPrTable (5) — Rich table rendering, review status styling, missing fields
    • TestFetchIssueFromContract (6) — contract reads, status validation, require_active
    • TestFindPrsForIssue (20) — GraphQL parsing, REST parsing, dedup, state filter, merged detection, field extraction, API errors
    • TestSubmissionsCommand (12) — table display, JSON output, field stripping, PR count, PAT warning, graceful API failure, empty PAT
    • TestPredictCommand (34) — all input modes, early flag-conflict validation, boundary probabilities, payload shape, wallet failure, JSON edge cases, hotkey display
    • TestValidatePredictions (12) — probability range, PR existence, sum constraints, float precision, boundary values
    • TestFormatPredLines (5) — display formatting, small/full probabilities
    • TestCollectPredictions (15) — all 3 input modes, interactive TTY (duplicate PR, invalid input, empty PRs), JSON parsing errors
    • TestCascadingFallback (7) — exact call sequence verification, state filter propagation, all-empty cascade
    • TestGetGithubPat (4) — PAT retrieval, empty string → None, unset → None
    • TestVerifyMinerRegistration (6) — registered/unregistered hotkey, wallet load failure, missing bittensor, custom wallet name, subtensor connect failure
    • TestBuildPredictionPayload (5) — payload construction, string keys, JSON serializable, empty predictions
    • TestReadNetuidFromContract (5) — netuid extraction, zero/None fallback, connection errors, mainnet default
    • TestFindPrsForIssueGraphql (7) — null nodes, case-insensitive repo matching, ghost author (null + empty login), state filter, mergedAt extraction
    • TestFindPrsForIssueRest (10) — auth header, non-PR event filtering, connection errors, quiet mode, REST field defaults, ghost author, merged state filter
    • TestGhostAuthorDisplay (4) — ghost PRs in table, submissions output, JSON output, database_id=None
    • TestPredictEdgeCases (8) — GitHub API failure, contract error, JSON output on failure, verify_miner ordering, network header
    • TestSubmissionsEdgeCases (5) — multi-PR display, JSON list shape, empty JSON, GitHub link, contract error
  • All 470 tests pass, ruff lint clean
  • Tests caught and fixed a real bug: null author in GraphQL response caused AttributeError
ruff check gittensor/cli/ gittensor/utils/github_api_tools.py tests/cli/test_submissions.py
python -m pytest tests/ -v

Files Changed

File Action Purpose
gittensor/utils/github_api_tools.py Modified find_prs_for_issue() with GraphQL → REST(auth) → REST(unauth) cascade, PRInfo TypedDict, _resolve_pr_state(), ghost-author handling, refactored find_solver_from_cross_references()
gittensor/cli/issue_commands/helpers.py Modified collect_predictions(), read_netuid_from_contract(), fetch_issue_from_contract(), build_pr_table(), format_pred_lines(), validate_predictions(), verify_miner_registration(), build_prediction_payload(), get_github_pat(), MAINNET_NETUID
gittensor/cli/issue_commands/submissions.py New issues_submissions and issues_predict Click commands (flat orchestration, 5-phase fail-fast, early flag validation, no local helpers)
gittensor/cli/issue_commands/__init__.py Modified Register new commands, update docstring and __all__
tests/cli/test_submissions.py New 183 tests across 20 test classes

Checklist

  • Code follows project style guidelines (ruff + isort clean)
  • Self-review completed
  • Changes documented (CLI docstrings and __init__.py command list updated)
  • Command file structure matches view.py/mutations.py convention (zero local helpers)
  • No behavioral regression in refactored find_solver_from_cross_references()
  • Early flag-conflict validation before network I/O
  • Graceful GitHub API error handling (warning, not crash)
  • Empty-string PAT edge case handled
  • SRP-extracted helpers with dedicated test classes
  • Tests caught and fixed null-author GraphQL bug
  • Ghost/deleted GitHub account handling with 'ghost' default (GitHub convention)

Add modular PR discovery via GitHub issue cross-references, reusing the
same GraphQL CROSS_REFERENCED_EVENT timeline approach as the existing
solver detection code.

- Add PRInfo TypedDict for structured return type
- Add find_prs_for_issue() with GraphQL + REST fallback paths
- Add _resolve_pr_state() helper to deduplicate state resolution
- Use Literal type for state_filter parameter
- Refactor find_solver_from_cross_references() to call find_prs_for_issue
  internally, eliminating a redundant second GraphQL query
Add a public helper to read the netuid from contract packed storage with
a graceful fallback to MAINNET_NETUID (74). Includes verbose logging on
failure instead of silently swallowing exceptions.
Add two new miner-facing CLI commands for the prediction feature:

- gitt issues submissions --id <N>: list open PRs for an issue bounty
  with Rich table display and --json output support
- gitt issues predict --id <N>: submit predictions on which PR will
  solve an issue, with three input modes (--pr/--probability, --json-input,
  interactive TTY), wallet/metagraph verification, and probability validation

Both commands share extracted helpers (_fetch_issue, _collect_predictions,
_validate_predictions) for clean separation of concerns. Network broadcast
is stubbed with a TODO for when the prediction protocol is implemented.
28 tests covering:
- find_prs_for_issue: GraphQL open/merged/closed filtering, wrong repo
  rejection, empty timeline, REST fallback, review status, closes_issue flag
- submissions command: table display, JSON output, invalid ID, not found,
  empty state, non-active status warning
- predict command: single/batch/JSON predictions, probability validation
  (>1, <0, sum >1), PR existence check, missing PAT, unregistered hotkey,
  --pr/--probability mutual requirement

Uses a pytest fixture for predict mocks to eliminate decorator stacking.
…overage

- Move build_pr_table, fetch_issue_from_contract, format_pred_lines,
  validate_predictions to helpers.py matching codebase convention
- Flatten submissions.py to match view.py/mutations.py pattern (no local helpers)
- Reorder predict command: fetch issue/PRs before wallet loading (fail-fast)
- Fix interactive duplicate PR bug (running_sum drift)
- Restructure bittensor error handling to match mutations.py pattern
- Add 16 new tests: helper unit tests + JSON edge case coverage (44 total)
@fury0928
fury0928 force-pushed the feat/cli-submissions-predict branch from 5184962 to 2b57612 Compare February 21, 2026 08:14
…T(unauth)

Fine-grained PATs can filter out cross-reference timeline events in both
GraphQL and authenticated REST responses. When either returns empty, cascade
to the next method. Suppresses misleading "No token" warning on intentional
unauthenticated fallback.
Move the three prediction input modes (--json-input, --pr/--probability,
interactive TTY) from issues_predict into a shared helper. Reduces
issues_predict from 223 to 150 lines, keeping it as pure orchestration.
Add 12 new tests (56 total):
- TestCollectPredictions (8): direct unit tests for all input modes
  including interactive TTY, JSON parsing, and edge cases
- TestCascadingFallback (4): verify exact call sequence of
  GraphQL → REST(auth) → REST(unauth) cascade
New test classes:
- TestResolvePrState (8): shared state normalization
- TestValidateIssueId (6): boundary validation
- TestBuildPrTable (5): Rich table rendering
- TestFetchIssueFromContract (6): contract reads and status

Expanded existing classes:
- TestFindPrsForIssue: 9→20 (REST parsing, dedup, state filter,
  merged detection, field extraction)
- TestSubmissionsCommand: 6→10 (JSON stripping, PR count, PAT
  warning, negative ID)
- TestPredictCommand: 18→27 (payload shape, boundary probs, wallet
  failure, hotkey display, TODO note)
- TestValidatePredictions: 8→12 (boundary values, float precision)
- TestFormatPredLines: 3→5 (small/full probability formatting)
- TestCollectPredictions: 8→15 (interactive duplicate, invalid input
  retry, empty PRs)
- TestCascadingFallback: 4→7 (all-empty, state filter propagation)

All 408 tests pass, ruff clean.
- Early flag-conflict checks in predict before any network I/O:
  reject --pr + --json-input, probability range, missing flags
- Graceful GitHub API failure in submissions (warning + empty,
  not traceback)
- Empty-string GITTENSOR_MINER_PAT treated as None (or None)
- 5-phase ordering with explicit comments
- 129 tests (8 new), 416 total pass
…, get_github_pat into helpers

- Extract 3 shared helpers from submissions.py into helpers.py for SRP/DRY
- verify_miner_registration: wallet loading + metagraph check with structured errors
- build_prediction_payload: payload construction with string-keyed predictions
- get_github_pat: centralized PAT retrieval (empty string → None)
- Wrap predict's find_prs_for_issue in try/except for graceful API errors
- 144 tests (was 129): add TestVerifyMinerRegistration (6), TestBuildPredictionPayload (5), TestGetGithubPat (4)
- All 431 tests pass, ruff clean
…or bug

- Add TestReadNetuidFromContract (5): netuid extraction, zero/None fallback, connection errors
- Add TestFindPrsForIssueGraphql (6): null nodes, case-insensitive repo, null author, state filter
- Add TestFindPrsForIssueRest (8): auth header, non-PR events, connection errors, quiet mode
- Add TestPredictEdgeCases (8): GitHub API failure, contract error, JSON keys, network header
- Add TestSubmissionsEdgeCases (5): multiple PRs, JSON list/empty, GitHub link, contract error
- Fix bug: null author in GraphQL response caused AttributeError (pr.get('author') or {})
- 176 tests (was 144), all 463 total pass, ruff clean
…author tests

- Use 'ghost' (GitHub convention) instead of 'unknown' for null/deleted authors
- Handle null user in REST path with `or {}` (same pattern as GraphQL)
- Add TestGhostAuthorDisplay (4): table render, submissions output, JSON, database_id
- Add ghost tests in GraphQL (2): null author, empty login string
- Add ghost test in REST (1): null user object
- Add REST state filter merged test (1)
- 183 tests (was 176), all 470 total pass, ruff clean
@anderdc

anderdc commented Feb 23, 2026

Copy link
Copy Markdown
Collaborator

I can tell you right now this has way too much testing, you'll have to cull the testing to 1500 lines max, keeping the most important/prominent tests, wasn't a requirement, but it's just too much.

@anderdc

anderdc commented Feb 23, 2026

Copy link
Copy Markdown
Collaborator

Hi strong submission but we're going with PR 231, this PR is similar to 224 (nothing wrong with that), but 231 goes a bit beyond and further separates the prediction/submission commands, creating more modularity and room for change

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

New CLI commands

2 participants