Skip to content

v2.0.13

Choose a tag to compare

@cyberofficial cyberofficial released this 23 Jul 07:08
· 44 commits to master since this release

v2.0.13 - Date: 2026-07-20

Which Build for me?

Nimbus Lite - You Dont care about the web_search mcp
Nimbus - You care about websearch mcp

Added

Token Limit Enforcement & Reasoning Budget Validation

  • NIM_MAX_TOKENS default reduced from 202000 to 32000 for more reasonable token limits
  • Token upgrade logic -- When NIM_MAX_TOKENS is set higher than what Claude sends (e.g., 32000 vs 256000), the proxy now upgrades the request's max_tokens to the configured value instead of capping down
  • Reasoning budget validation -- If reasoning_budget >= max_tokens (after upgrade), sets reasoning_budget=-1 (unlimited) and relies on max_tokens as the effective cap to prevent NVIDIA rejection
  • Explicit logging -- Logs both token upgrades and budget adjustments for audit trail

Empty NIM Stream Handling & Provider Resilience

  • Empty stream detection -- NvidiaNimProvider now detects empty/malformed NVIDIA NIM streaming responses (stream finishes with no finish_reason and no content: text, reasoning, or tool states) and raises StreamTruncatedError to trigger retry logic
  • Request queue attribute on BaseProvider -- Added default _request_queue = None on BaseProvider to enable queue stats in api/routes.queue_status with proper typing
  • Rate limit slot release on pre-response failures -- Buffered requests that fail before reaching the server now properly release rate-limit slots via release_last_slot()
  • Expanded retryable errors -- execute_with_rate_limit now treats httpx.TimeoutException, APIConnectionError, APITimeoutError, and 5xx server errors as retryable; releases slots, logs, and applies exponential backoff

Discord Bot Stability & DM Handling

  • DM/channel guards -- Added defensive checks for interaction.channel being None or not a messageable channel; returns early instead of crashing
  • Missing channel/category guards -- In discord_bot/views.py, validates resolved category is a discord.CategoryChannel before use; returns clear ephemeral error if not
  • Fallback channel/guild names -- Backup/backup commands now use getattr(channel, 'name', 'dm') and getattr(channel.guild, 'name', 'Unknown Guild') for DM-friendly filenames and messages
  • Conversation wrapper ergonomics -- add_message now calls add_message_with_user with explicit keyword arguments for tools_used and tool_results

Type Safety, Null Checks & Pylance Compliance

  • Added type: ignore comments for mypy/pyright compliance (sys._MEIPASS, termios, attribute access on unions)
  • Defensive null checks for model_name, detail, extracted, new_budget variables
  • Fixed bugs: removed malformed duplicate code in _rebuild_without_key(), corrected _has_thinking_params() removal
  • Simplified tiktoken cache logic by removing fallback handling
  • Fixed priority enum casting in request_queue.py (RequestPriority(priority) -> RequestPriority(priority))
  • Fixed pyright directive in config/nim.py (changed to # pyright: ignore[reportCallIssue])

Discord Bot & Views Resilience

  • CategoryChannel validation -- CreateChannelModal and list_channels button now verify interaction.guild.get_channel() returns a CategoryChannel before use
  • DM/guild guards -- Commands requiring guild channels now explicitly check interaction.guild first and return early for DMs
  • Member vs User typing -- CompactConfirmView and related classes use discord.abc.User to accept both User and Member

Pylance/Type Checking Fixes

  • Fixed pyright directive in config/nim.py (invalid disable= syntax -> ignore[...])
  • Added # type: ignore[union-attr] and # type: ignore[attr-defined] suppressions for union attribute access patterns
  • Added cast(Literal["user", "assistant", "system"], ...) for message role literal types
  • Fixed reportIndexIssue for Message indexing in bot.py

Discord Bot DM/Channel Safety

  • All command handlers now guard with if not interaction.guild: return before using interaction.channel
  • interaction.channel accesses wrapped with if channel is None: return
  • Channel name access uses getattr(channel, 'name', 'dm') for DM friendliness
  • isinstance(category, discord.CategoryChannel) check before using as category

Conversation API Ergonomics

  • add_message now calls add_message_with_user with keyword arguments for tools_used and tool_results instead of positional

Changed

NIM_MAX_TOKENS Behavior

  • Default reduced from 202000 to 32000 in code and .env.example
  • Upgrade logic added -- if request max_tokens < NIM_MAX_TOKENS env value, proxy upgrades to env value (log: "max_tokens X upgraded to NIM_MAX_TOKENS: Y")
  • Removed fallback to nim.max_tokens when request doesn't provide max_tokens; lets NIM model use its own defaults

.env.example Reorganization

  • Full restructure with section headers (CORE SERVER, NVIDIA NIM, PROVIDER RATE LIMITING, REQUEST QUEUE, MCP, DISCORD BOT, COMMAND TOGGLES)
  • Removed duplicated entries and improved examples/comments
  • NIM_MAX_TOKENS=32000 default shown

Rate Limiter Improvements

  • Method renamed -- reset_adaptive_backoff() -> reset_reactive_block()
  • UI messaging -- "Resets in" -> "Next Slot free in" for clarity
  • New release_last_slot() -- Explicit slot release for pre-response failures
  • Retriable error expansion -- Now handles httpx.TimeoutException, openai.APIConnectionError, openai.APITimeoutError, 5xx; releases slots, logs, applies exponential backoff
  • cast(MessagesResponse, response) -- Type-safe casting in optimization handler

Discord Bot DM/Channel Safety

  • All command handlers now guard with if not interaction.guild: return before using interaction.channel
  • interaction.channel accesses wrapped with if channel is None: return
  • Channel name access uses getattr(channel, 'name', 'dm') for DM friendliness
  • isinstance(category, discord.CategoryChannel) check before using as category

Conversation API Ergonomics

  • add_message now calls add_message_with_user with keyword arguments for tools_used and tool_results instead of positional

Fixed

  • Various Pylance/pyright type checking issues across codebase
  • Rate limit status display: "Resets in" -> "Next Slot free in" for clarity
  • config/nim.py pyright directive syntax (invalid disable= -> ignore[...])
  • Empty/malformed NIM stream handling (no finish_reason, no content -> retry)
  • BaseProvider missing _request_queue attribute
  • Discord DM crashes (missing channel/category/category type)
  • Rate limit slot leaks on pre-response failures
  • Conversation.add_message positional arg mismatch for tools_used/tool_results
  • Priority enum casting in request_queue.py (RequestPriority(priority) -> RequestPriority(priority))

Removed

  • Fallback to nim.max_tokens when request doesn't provide max_tokens (lets NIM model use its own defaults)

Files Touched

  • .env.example -- Full restructure with clarified defaults, removed duplicates, improved comments; NIM_MAX_TOKENS=32000 default
  • config/nim.py -- max_tokens default from 202000 -> 32000; pyright directive fix
  • providers/request.py -- Token upgrade logic, reasoning budget validation, logging
  • api/app.py -- Rate limit status message update
  • api/routes.py -- Queue status endpoint with _request_queue typing comment; DM/guild guards
  • api/optimization_handlers.py -- Type-safe casting
  • providers/request.py -- Token upgrade logic, reasoning budget validation, logging
  • providers/request_queue.py -- Priority enum casting fix
  • providers/rate_limit.py -- release_last_slot(), expanded retryable errors, reset_reactive_block(), exponential backoff
  • providers/provider.py -- Empty stream detection, StreamTruncatedError, rate limit slot release, BaseProvider._request_queue
  • providers/base.py -- _request_queue = None, _client = None
  • discord_bot/bot.py -- Type ignores for union attributes, DM message guards, getattr for channel names
  • discord_bot/cog.py -- getattr for channel/guild names, isinstance CategoryChannel checks, interaction.channel None guards, discord.abc.User typing, keyword args in add_message
  • discord_bot/views.py -- CategoryChannel validation, client.settings type ignores
  • discord_bot/conversation.py -- Keyword args for add_message_with_user
  • config/nim.py -- Pyright directive fix
  • config/settings.py -- (minor)
  • providers/base.py -- _request_queue = None, _client = None
  • providers/request.py -- Token upgrade, budget validation
  • providers/rate_limit.py -- release_last_slot(), retryable error expansion, reset_reactive_block(), backoff
  • packaged_entry.py -- (typing)
  • api/optimization_handlers.py -- cast(MessagesResponse, response)
  • tests/providers/test_nvidia_nim_request.py -- Updated tests for upgrade behavior
  • tests/providers/test_nvidia_nim.py -- Updated test expectations
  • tests/api/test_dependencies.py -- Mock settings fix