Skip to content

feat(nebius): sync Token Factory models with automated catalog#3190

Open
amrrs wants to merge 6 commits into
anomalyco:devfrom
amrrs:nebius-token-factory-sync
Open

feat(nebius): sync Token Factory models with automated catalog#3190
amrrs wants to merge 6 commits into
anomalyco:devfrom
amrrs:nebius-token-factory-sync

Conversation

@amrrs

@amrrs amrrs commented Jul 10, 2026

Copy link
Copy Markdown

Replaces #2971, which was auto-closed as stale and couldn't be reopened (GitHub's reopen API kept failing with a generic error even with correct permissions — likely tied to how the stale-bot closure interacted with this repo). This branch is rebased onto current dev with the same fixes already posted at #2971 (comment), plus the rebase conflict resolution below.

What this does

Adds an automated sync provider for Nebius Token Factory (packages/core/src/sync/providers/nebius.ts), following the same review feedback already given on #2971 by @rekram1-node:

  1. Merge conflicts — resolved by rebasing onto current dev.
  2. context_length regressionresolveContext() now only accepts the API's context_length when it's >= the larger of the existing/canonical context and the input limit, so a throttled/default value from the verbose=true endpoint can no longer clobber a correct context or produce context < input. The models affected by the original regression (MiniMax-M2.5, Qwen3.5-397B-A17B, Qwen3-Next-80B-A3B-Thinking, Qwen3-32B, Llama-3.1-Nemotron-Ultra-253B, Nemotron-3-Nano-Omni, Kimi-K2.6, Nemotron-3-Ultra-550b-a55b) are corrected to inherit context from canonical metadata again.
    • MiniCPM-V-4_5 and Cosmos3-Super-Reasoner are new models with no base_model match and no prior entry, so there's no in-repo ground truth to guard their context = 8_000 against — fixing those needs a real re-sync with a working API key/tier.
  3. hourly/daily doc contradiction — reverted, docs match the (already-hourly) workflow cron.
  4. Duplicated resolver/cache logicopenrouter.ts now exports createCanonicalBaseModelResolver(prefixes); nebius.ts calls into it with its own org→namespace map instead of reimplementing resolveCanonicalBaseModel, modelMetadataExists, and the file-scan cache.

Rebase notes

dev moved forward substantially since this branch was opened (most notably a repo-wide "model descriptions" commit and an openai sync provider added at the same points in sync/index.ts and sync.md). Conflict resolution:

  • Legacy Nebius model files this PR intentionally removes/renames (the -fast variants, GLM-5, DeepSeek-V3.2, Kimi-K2.5, etc.) only conflicted because dev added a description line to them after they were deleted here — kept the deletions.
  • Nebius model files this PR keeps picked up dev's new description line.
  • description is now a required schema field for full (non-factored) entries; Nebius's factored entries already inherit it from canonical metadata, so nebius.ts now falls back to the same describeModel() helper OpenRouter uses for its two orphan full entries (Cosmos3-Super-Reasoner, MiniCPM-V-4_5).
  • openrouter.ts gained a CANONICAL_BASE_MODEL_OVERRIDES lookup on dev independently of this PR's createCanonicalBaseModelResolver refactor — merged both: the override check wraps OpenRouter's own resolver instance and isn't part of the generic factory Nebius uses.

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [medium] [possible mistake] providers/nebius/models/zai-org/GLM-5.1.toml:2 - Check: reasoning_options must reflect the inference provider's actual request surface (AGENTS.md checklist; .opencode/skills/audit-reasoning-options/SKILL.md). Why: The new Nebius reasoning models (GLM-5.1, Kimi-K2.6, Kimi-K2.7-Code, Nemotron-3-Ultra-550b-a55b, Cosmos3-Super-Reasoner) ship reasoning_options = [], and packages/core/src/sync/providers/nebius.ts asserts "The API only exposes a boolean reasoning capability, not the toggle/effort mechanism." That rationale is contradicted by providers/nebius/provider.toml, which documents that the Nebius chat-completions API accepts reasoning_effort = "low"|"medium"|"high" (citing the API reference), and by sibling Nebius entries that already declare effort options (GLM-5.2, gpt-oss-120b, DeepSeek-V4-Pro). An empty array means "no caller control verified," but a control is provider-documented here, so the new entries are internally inconsistent with the provider config and their siblings. Action: Per the audit evidence standard, verify per-model whether reasoning_effort is accepted and meaningful for these models and add the effort/toggle option where confirmed; or correct the sync-code rationale and document why [] is correct for these specific models (e.g., reproduced evidence that the field is silently ignored for them).
  • [medium] [possible mistake] providers/nebius/models/nvidia/Cosmos3-Super-Reasoner.toml:18 - Check: New model limit.context values must be factual, not the endpoint's throttled default. Why: resolveContext was added specifically to reject the throttled 8_000 value the verbose=true endpoint returns, yet the two new full-entry models with no canonical/existing ground truth ship limit.context = 8_000 (and output = 8_000): Cosmos3-Super-Reasoner.toml:18 and providers/nebius/models/openbmb/MiniCPM-V-4_5.toml:17. The PR body acknowledges these need "a real re-sync with a working API key/tier." Shipping a context window that the PR's own evidence identifies as a throttle default likely understates the real window for a reasoning model and a vision model. Action: Verify the actual context window for these two models via a non-throttled API response or provider documentation before merge, or mark the entries provisional; do not ship the known-throttled 8_000.
  • [low] [possible mistake] providers/nebius/models/google/gemma-3-27b-it.toml:6 - Check: Provider-specific modality/attachment must reflect the provider's real serving surface, not an incomplete API response. Why: gemma-3-27b-it is downgraded from attachment = true / modalities.input = ["text","image"] to text-only based on the API's architecture.modality. Gemma 3 27b-it is a known vision-capable model and the prior Nebius entry was hand-authored as multimodal. The same verbose=true endpoint is documented elsewhere in this PR as returning throttled/incomplete values (8_000 context), so the modality field may likewise be incomplete for this model. Action: Confirm via provider docs or a reliable API response whether Nebius serves gemma-3-27b-it with image input; if so, restore attachment = true and modalities.input = ["text","image"]. The same verification should be applied to the other API-derived capability downgrades in this PR (e.g., Qwen2.5-VL-72B-Instruct losing tool_call/structured_output).

amrrs and others added 5 commits July 11, 2026 02:24
Adds a sync provider for Nebius Token Factory (verbose /v1/models
endpoint) so model list, pricing, context length, and capability flags
stay in sync automatically, resolving base_model references against
canonical models/ metadata where available. Also switches the model
sync automation from an hourly to a daily schedule.

Co-authored-by: Cursor <cursoragent@cursor.com>
The previous commit inadvertently changed the global sync cadence from
hourly to daily. That change affects every provider, not just Nebius,
and is out of scope for the Nebius Token Factory sync PR. Reverting to
the original hourly schedule; a cadence change should be evaluated on
its own merits in a separate PR.

Co-authored-by: Cursor <cursoragent@cursor.com>
Addresses review feedback on PR anomalyco#2971:
- Only trust the API's context_length when it's >= the existing/canonical
  context and the input limit, since the verbose=true endpoint has been
  observed returning a throttled 8_000 default; repair the 8 already
  affected model files accordingly.
- Revert the two sync.md "daily" mentions back to "hourly" so the docs
  match the (already-hourly) workflow cron.
- Share canonical base-model resolution with openrouter.ts via a new
  createCanonicalBaseModelResolver factory instead of reimplementing
  resolveCanonicalBaseModel/modelMetadataExists/the file-scan cache.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
dev's "feat: model descriptions" made `description` a required schema
field for full (non-factored) models. Nebius's factored entries already
inherit it from canonical metadata (same as name), but the two orphan
full entries (Cosmos3-Super-Reasoner, MiniCPM-V-4_5) had no source to
inherit from, so give buildNebiusModel the same describeModel-based
fallback OpenRouter uses, and backfill the two already-committed files.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ng options

Addresses the automated review on PR anomalyco#3190:

- tool_call/structured_output only ever gain from an API sample, never
  lose a capability an existing entry already had confirmed; modalities
  are unioned with the existing entry rather than replaced. The same
  verbose=true endpoint that throttles context_length has been observed
  under-reporting these too (MiniMax-M2.5, Llama-3.1-Nemotron-Ultra-253B,
  Nemotron-3-Nano-30B-A3B, Llama-3.3-70B-Instruct, gemma-3-27b-it,
  Qwen2.5-VL-72B-Instruct all lost a real capability/modality/context
  value this way); restored the affected files to their correct values.
- Brand-new models with no canonical/existing reference and a
  context_length of exactly 8_000 (the confirmed throttled default) are
  now skipped rather than shipped with an unverifiable context; removed
  Cosmos3-Super-Reasoner and MiniCPM-V-4_5, which only existed because of
  this gap, and report the skip via skippedNotice for a future re-sync.
- Hand-curated reasoning_options for the new reasoning-capable models
  (GLM-5.1, Kimi-K2.6, Kimi-K2.7-Code, Nemotron-3-Ultra-550b-a55b) to
  match sibling entries, since provider.toml documents the underlying
  chat completions endpoint accepting reasoning_effort even though the
  models-discovery API doesn't expose per-model evidence for it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@amrrs amrrs force-pushed the nebius-token-factory-sync branch from ee85a51 to 679f1d6 Compare July 10, 2026 20:54
@amrrs

amrrs commented Jul 10, 2026

Copy link
Copy Markdown
Author

Thanks for the review — addressed all three action items, plus rebased onto dev again (it had moved forward since the last push and the branch had gone conflicting).

1. reasoning_options = [] on new reasoning models — you're right, provider.toml documents the chat completions endpoint accepting reasoning_effort, and the models-discovery API's lack of per-model evidence isn't a reason to ship [] when siblings (gpt-oss-120b, GLM-5.2, DeepSeek-V4-Pro) already declare it. Hand-curated [{type="effort", values=["low","medium","high"]}] for GLM-5.1, Kimi-K2.6, Kimi-K2.7-Code, and Nemotron-3-Ultra-550b-a55b, and clarified the code comment so this doesn't get missed again for future reasoning models.

2. limit.context = 8_000 on the two orphan models — agreed, shipping the confirmed-throttled default isn't defensible without a way to verify it. Removed Cosmos3-Super-Reasoner and MiniCPM-V-4_5 entirely rather than mark them provisional (no schema field for that), and added a code-level guard: a brand-new model with no canonical/existing reference and a context_length of exactly 8_000 is now skipped by the sync and reported via skippedNotice, so this can't recur silently — it'll need a real re-sync once the throttling is resolved.

3. gemma-3-27b-it / Qwen2.5-VL-72B-Instruct capability downgrades — this pointed at a broader issue than just those two. I ran a full inheritance-aware comparison of every Nebius file against its pre-PR baseline and found the same under-reporting had also silently dropped structured_output/tool_call on four more models (MiniMax-M2.5, Llama-3.1-Nemotron-Ultra-253B, Nemotron-3-Nano-30B-A3B, Llama-3.3-70B-Instruct) that weren't in your sample. Restored all of them, and fixed the root cause in nebius.ts: tool_call/structured_output now only ever gain from an API sample (never lose a capability an existing entry already confirmed), and modalities are unioned with the existing entry rather than replaced, so attachment (derived from modalities) can't regress either.

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [medium] [violation] providers/nebius/models/Qwen/Qwen2.5-VL-72B-Instruct.toml - Check: Use base_model when a matching models/<provider>/<model>.toml exists; do not duplicate provider-agnostic metadata inline. Why: models/alibaba/qwen2-5-vl-72b-instruct.toml already defines this same model (identical description and family = "qwen", same text+image modalities). This PR re-authors the Nebius entry as a full inline definition and adds family = "qwen", duplicating the canonical metadata. The sync's resolveCanonicalBaseModel never factors it because its candidates (Qwen2.5-VL-72B-Instruct, qwen2.5-vl-72b-instruct) don't match the on-disk qwen2-5-vl-72b-instruct.toml (the . vs - Qwen naming gap, which also affects OpenRouter's copy). Action: Factor this entry to base_model = "alibaba/qwen2-5-vl-72b-instruct" keeping only Nebius-specific overrides (cost, limit, attachment = true, structured_output, dates), or extend canonicalCandidates/the resolver to handle the 2.52-5 Qwen naming so the sync factors it automatically.
  • [low] [possible mistake] providers/nebius/models/Qwen/Qwen3-32B.toml (also nvidia/Llama-3_1-Nemotron-Ultra-253B-v1.toml, nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B.toml) - Check: reasoning_options must be applied consistently across reasoning models on the same provider. Why: The new packages/core/src/sync/providers/nebius.ts comment states "new reasoning models added here should get an explicit effort option hand-curated … rather than default to []," and the PR curates effort for four brand-new reasoning models (GLM-5.1, Kimi-K2.6, Kimi-K2.7-Code, Nemotron-3-Ultra-550b-a55b) citing provider.toml's generic reasoning_effort support. But three models that flip to reasoning = true in this same PR (via base_model inheritance from reasoning=true canonical entries) ship with reasoning_options = [], and existing same-family Nebius reasoning models (e.g. Nemotron-3-Nano-Omni, nemotron-3-super-120b-a12b) also keep [] — so Nemotron-3-Ultra-550b gets effort while sibling Nemotron reasoning models get [] under the same generic provider documentation. Action: Either hand-curate effort for the newly-reasoning and same-family models to match the documented standard, or scope the nebius.ts comment to brand-new catalog entries only and confirm [] is intentional for the inherited-reasoning models.

…idence

Addresses the second automated review round on PR anomalyco#3190:

- Extended openrouter.ts's canonicalCandidates with an alibaba-specific
  dot-to-dash candidate (2.5 -> 2-5), matching the legacy
  qwen2-5-vl-72b-instruct.toml naming that predates the qwen3.5-/qwen3.6-
  dotted convention; this also fixes OpenRouter's own un-factored copy
  of the same model on future syncs. Factored the Nebius entry to
  base_model = "alibaba/qwen2-5-vl-72b-instruct", keeping only the
  Nebius-specific overrides (attachment, structured_output, cost, limit).
- Reasoning models already catalogued on this provider diverge by family
  (GLM/gpt-oss/DeepSeek use `effort`, Qwen3.5 uses `toggle`, the Nemotron
  siblings use neither) — provider.toml's generic reasoning_effort
  acceptance doesn't mean every model's backend honors it. Reverted the
  `effort` curation on Kimi-K2.6/Kimi-K2.7-Code/Nemotron-3-Ultra-550b-a55b
  (no same-family sibling evidence on this provider) back to `[]`, kept
  it only for GLM-5.1 (direct GLM-5.2 sibling), and tightened the
  sync-code/docs rationale to require same-family sibling evidence rather
  than provider-wide capability.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@amrrs

amrrs commented Jul 10, 2026

Copy link
Copy Markdown
Author

Thanks — both fixed.

1. Qwen2.5-VL-72B-Instruct not factored (medium) — the resolver gap was real: canonicalCandidates had no rule for the 2.52-5 naming qwen2-5-vl-72b-instruct.toml predates (before the qwen3.5-/qwen3.6- dotted convention took over). Extended it with an alibaba-specific dot-to-dash candidate, which fixes this for OpenRouter's own un-factored copy too on its next sync, not just Nebius. Factored the Nebius entry to base_model = "alibaba/qwen2-5-vl-72b-instruct", keeping only the real Nebius-specific overrides (attachment, structured_output, cost, limit).

2. reasoning_options inconsistency (low) — you're right that I was inconsistent, but on reflection I don't think "curate effort everywhere" is the correct fix here — the sibling evidence actually points the other way. provider.toml only establishes that the chat completions endpoint's request schema accepts reasoning_effort; it says nothing about whether a given model's backend does anything with it. Looking at what's already curated on this same provider: GLM-5.2/gpt-oss-120b/DeepSeek-V4-Pro use effort, Qwen3.5-397B-A17B uses toggle, and the existing Nemotron siblings (Nemotron-3-Nano-Omni, nemotron-3-super-120b-a12b) use neither — genuinely divergent by family, not a provider-wide constant. So instead of extending effort to match, I reverted it on Kimi-K2.6/Kimi-K2.7-Code/Nemotron-3-Ultra-550b-a55b (no same-family sibling on this provider demonstrates it) back to [], and kept it only on GLM-5.1 (direct GLM-5.2 sibling). Tightened the nebius.ts comment and sync.md to state the rule precisely: hand-curate effort/toggle only when a same-family sibling on this exact provider already shows that mechanism, not from generic provider-level documentation. Qwen3-32B/Llama-3.1-Nemotron-Ultra-253B/Nemotron-3-Nano-30B-A3B only flipped to reasoning = true as a side effect of factoring onto more accurate canonical data — their [] was already correct and unrelated to this PR's curation choices, so left as-is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant