RFC: CI fails a pull request that moves a submodule pin backwards

[bong-water-water-bong]

Problem

A pull request can move a submodule pin backwards without meaning to. #154 (MoE streaming) committed a stale third_party/llama.cpp checkout: the pin went from f30cc43 to f59f6d37a, which has diverged from it. That dropped ZAYA1-VL and 23 other commits from main for about 20 minutes, until #151 moved it forward again. Review missed it because the PR description only mentioned the other pin. The weekly pins_on_branch check only runs on Sundays, and only asks whether a pin is on its branch, not whether it went back.

Design

• tools/check_pins.py <base>: for every submodule whose gitlink differs between the PR's base and HEAD, ask GitHub's compare API for old...new. ahead or identical passes; behind or diverged fails with the path and both commits.
• It runs as a step in ci.yml on pull requests, with the read-only GITHUB_TOKEN.
• Submodules the token cannot read (the private comfyui.cpp) are reported and skipped; non-GitHub URLs are skipped.
• A deliberate rollback carries a new pin rollback label, which skips the step. That's why ci.yml now also runs on labeled/unlabeled.

Alternatives

• Rely on review: this failed on MoE streaming: prefetch off the decode path, time split measured (pin llama.cpp-vulkan a29be7f) #154.
• Also require the pin to be on its .gitmodules branch on every PR, not just weekly. This is stricter, but agents often pin a fork PR's head before it merges, so it would block normal work. Left to the weekly check.

How it will be checked

Replayed on real history:

• MoE streaming: prefetch off the decode path, time split measured (pin llama.cpp-vulkan a29be7f) #154 (f96464552 against its parent) fails, on exactly third_party/llama.cpp: f30cc4399 -> f59f6d37a is diverged, while its llama.cpp-vulkan bump passes.
• third_party: bump llama.cpp pin for the HRX decode-split multipass fix (engine#115) #151 (038e4f31b against its parent) passes: f59f6d37a -> 8dd75eb48 is ahead.

Dependencies, licences, security

• No new dependency (Python stdlib).
• The token is the job's read-only token, and it only reads public compare data.

Open questions

None.

[bong-water-water-bong]

Accepted by the maintainer on 2026-09-26. Implemented in #159.