-
Notifications
You must be signed in to change notification settings - Fork 26
Description
Problem
The current spec tracking check (src/tier-check/checks/spec-tracking.ts) finds the first SDK release published after the latest spec release and measures the time gap between them. This doesn't validate anything meaningful for tiering:
- It doesn't know which SDK version is being submitted for evaluation
- The "first release after spec" could be a totally unrelated bugfix/patch
- It's disconnected from the spec version the submission claims conformance against
For example, the Java SDK tiering submission shows "9d gap" — but that just means some SDK release happened 9 days after the latest spec release, not that the submitted version (1.0.0) was released within 30 days of the spec version (2025-06-18) it's being evaluated against.
Proposed Change
The spec tracking check should:
- Accept the submitted SDK version (or release tag) as input
- Accept the target spec version being claimed
- Verify the submitted SDK release exists on GitHub
- Measure the gap between the target spec release date and the submitted SDK release date
- Pass if the SDK release was within 30 days of that spec version's release
This ties the check to the actual submission rather than being a context-free scrape of GitHub releases.
Context
Came up during review of the Java SDK Tier 2 assessment (modelcontextprotocol/modelcontextprotocol#2301).