Skip to content

feat(nebius): sync Token Factory models with automated catalog#2971

Closed
amrrs wants to merge 2 commits into
anomalyco:devfrom
amrrs:nebius-token-factory-sync
Closed

feat(nebius): sync Token Factory models with automated catalog#2971
amrrs wants to merge 2 commits into
anomalyco:devfrom
amrrs:nebius-token-factory-sync

Conversation

@amrrs

@amrrs amrrs commented Jul 1, 2026

Copy link
Copy Markdown

Adds a sync provider for Nebius Token Factory (verbose /v1/models endpoint) so model list, pricing, context length, and capability flags stay in sync automatically, resolving base_model references against canonical models/ metadata where available. Also switches the model sync automation from an hourly to a daily schedule.

Adds a sync provider for Nebius Token Factory (verbose /v1/models
endpoint) so model list, pricing, context length, and capability flags
stay in sync automatically, resolving base_model references against
canonical models/ metadata where available. Also switches the model
sync automation from an hourly to a daily schedule.

Co-authored-by: Cursor <cursoragent@cursor.com>
The previous commit inadvertently changed the global sync cadence from
hourly to daily. That change affects every provider, not just Nebius,
and is out of scope for the Nebius Token Factory sync PR. Reverting to
the original hourly schedule; a cadence change should be evaluated on
its own merits in a separate PR.

Co-authored-by: Cursor <cursoragent@cursor.com>
@rekram1-node

Copy link
Copy Markdown
Collaborator

Thanks for adding the Nebius Token Factory sync. A few things need addressing before this can go in:

1. Please resolve the merge conflicts

This PR is currently in a conflicting state against dev and can't be merged. Please rebase/merge dev and push.

2. context_length regression — the sync is producing invalid limits

The provider trusts model.context_length as authoritative (const context = model.context_length ?? existing?.limit?.context ?? 0), but the verbose=true endpoint appears to be returning a throttled/default 8000 for a lot of models. That clobbers correct values and yields entries where context < input, which is logically invalid:

model context (was → now) input
MiniMax-M2.5 204_800 (canonical) → 8_000 190_000
Qwen3.5-397B-A17B 262_144 → 8_000 250_000
Qwen3-Next-80B-A3B-Thinking 131_072 → 8_000 120_000
Qwen3-32B 131_072 → 40_960 120_000
Llama-3_1-Nemotron-Ultra-253B 128_000 → 8_000 120_000
Nemotron-3-Nano-Omni 65_536 → 8_000 60_000

All of your newly-created models (Kimi-K2.6, Nemotron-3-Ultra, Cosmos3, MiniCPM, etc.) also came through as context = 8_000, which reinforces that the API is handing back a limited value rather than the real context window. This looks like it may be tied to the API key/account tier used to generate the sync — worth double-checking. At minimum the sync should guard against overwriting a larger existing/canonical context with a smaller API value, and never emit context < input.

3. The "hourly → daily" schedule change was documented but not actually made

The PR description and sync.md both say the automation was switched from hourly to daily, but .github/workflows/sync-models.yml still has cron: "17 * * * *" (hourly). So the docs now contradict the workflow. Either make the cron change or revert the sync.md edit. Separately, this is a global change unrelated to adding Nebius — it'd be cleaner as its own PR rather than bundled here.

4. Inconsistent with the other sync providers

Every other provider that resolves canonical base models (baseten, huggingface, vercel, llmgateway) imports the shared resolveCanonicalBaseModel from openrouter.ts. This PR reimplements its own resolveCanonicalBaseModel, modelMetadataExists, and the modelMetadataFilesByProvider cache in nebius.ts. Your NEBIUS_ORG_TO_MODEL_PROVIDER map is essentially a re-keyed subset of openrouter's CANONICAL_PROVIDER_PREFIXES. Consider parameterizing the shared helper with a prefix→namespace map instead of duplicating the file-scan/cache logic, so this stays consistent with the rest of the sync providers.

@github-actions

Copy link
Copy Markdown
Contributor

Closing this pull request as stale because it has not been updated since feedback from @rekram1-node was left 7 days ago. Feel free to reopen it or submit a new pull request if the work is resumed.

@github-actions github-actions Bot closed this Jul 10, 2026
amrrs added a commit to amrrs/models.dev that referenced this pull request Jul 10, 2026
Addresses review feedback on PR anomalyco#2971:
- Only trust the API's context_length when it's >= the existing/canonical
  context and the input limit, since the verbose=true endpoint has been
  observed returning a throttled 8_000 default; repair the 8 already
  affected model files accordingly.
- Revert the two sync.md "daily" mentions back to "hourly" so the docs
  match the (already-hourly) workflow cron.
- Share canonical base-model resolution with openrouter.ts via a new
  createCanonicalBaseModelResolver factory instead of reimplementing
  resolveCanonicalBaseModel/modelMetadataExists/the file-scan cache.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@amrrs

amrrs commented Jul 10, 2026

Copy link
Copy Markdown
Author

Thanks for the thorough review — pushed fixes for all four items (13e9a4f3):

1. Merge conflicts — resolved; this branch is a clean fast-forward descendant of dev, nothing to rebase.

2. context_length regression — added a guard in nebius.ts (resolveContext()) that only accepts the API's context_length when it's >= the larger of the existing/canonical context and the input limit; otherwise it falls back to canonical/existing data, so context < input can no longer happen. Also repaired the 8 already-committed models that had the bad 8_000/40_960 override baked in (MiniMax-M2.5, Qwen3.5-397B-A17B, Qwen3-Next-80B-A3B-Thinking, Qwen3-32B, Llama-3.1-Nemotron-Ultra-253B, Nemotron-3-Nano-Omni, Kimi-K2.6, Nemotron-3-Ultra-550b-a55b) — they now correctly inherit context from canonical metadata instead of overriding it with the throttled value.

Disclosure: MiniCPM-V-4_5 and Cosmos3-Super-Reasoner still show context = 8_000. Both are brand-new models with no base_model match and no prior entry, so there's no in-repo ground truth to guard against — the check has nothing to compare the API value to. Fixing those needs a re-sync with a working API key/tier, per your note about this possibly being account-tier related.

3. hourly/daily contradiction — reverted the two sync.md mentions back to "hourly" (the cron itself was already reverted separately). Docs and workflow agree again.

4. Duplicated resolver/cache logic — refactored openrouter.ts to export createCanonicalBaseModelResolver(prefixes) (its own resolveCanonicalBaseModel is now just createCanonicalBaseModelResolver(CANONICAL_PROVIDER_PREFIXES)), plus exported modelMetadataExists and a new canonicalLimit helper. nebius.ts now calls into these instead of reimplementing its own resolver, existence check, and file-scan cache.

Disclosure: to get case-insensitive matching for Nebius's model IDs, I added modelID.toLowerCase() as a match candidate inside the shared canonicalCandidates() in openrouter.ts. That candidate list is also used by openrouter, baseten, huggingface, and vercel, so this is additive for all of them, not just Nebius — it can only add matches (exact case is still tried first), but it's a slightly wider blast radius than asked for, so flagging it rather than burying it. Happy to scope it to Nebius only (pre-lowercase model.id before calling in, matching the baseten/huggingface pattern) if you'd prefer that instead.

amrrs added a commit to amrrs/models.dev that referenced this pull request Jul 10, 2026
Addresses review feedback on PR anomalyco#2971:
- Only trust the API's context_length when it's >= the existing/canonical
  context and the input limit, since the verbose=true endpoint has been
  observed returning a throttled 8_000 default; repair the 8 already
  affected model files accordingly.
- Revert the two sync.md "daily" mentions back to "hourly" so the docs
  match the (already-hourly) workflow cron.
- Share canonical base-model resolution with openrouter.ts via a new
  createCanonicalBaseModelResolver factory instead of reimplementing
  resolveCanonicalBaseModel/modelMetadataExists/the file-scan cache.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
amrrs added a commit to amrrs/models.dev that referenced this pull request Jul 10, 2026
Addresses review feedback on PR anomalyco#2971:
- Only trust the API's context_length when it's >= the existing/canonical
  context and the input limit, since the verbose=true endpoint has been
  observed returning a throttled 8_000 default; repair the 8 already
  affected model files accordingly.
- Revert the two sync.md "daily" mentions back to "hourly" so the docs
  match the (already-hourly) workflow cron.
- Share canonical base-model resolution with openrouter.ts via a new
  createCanonicalBaseModelResolver factory instead of reimplementing
  resolveCanonicalBaseModel/modelMetadataExists/the file-scan cache.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants