Skip to content

Releases

NIMbus v2.0.15

Choose a tag to compare

@cyberofficial cyberofficial released this 05 Sep 07:56

v2.0.15 - Date: 2026-09-05

Full Changelog: v2.0.14...v2.0.15

Fixed

  • DeepSeek-V4-Pro CoT leakage in streaming - Disabled the forced thinking flag for deepseek-v4-pro. With server-default thinking, DeepSeek-V4 on NIM returns Chain-of-Thought inline in message.content with a trailing (orphan) response instead of the separate reasoning_content field.
  • Inline thinking → thinking blocks (buffered mode) - Split inline CoT via split_think_content and emit proper Anthropic thinking blocks; optimization_response_to_sse now emits thinking blocks too instead of dropping them.
  • Degraded DeepSeek-V4 DSML tool-call parsing & stop_reason guard - Rewrote the DSML parser to fix degraded tool-call extraction and guard stop_reason correctly.
  • Dangling DSML tag stripping - Added _strip_residual_tags to providers/dsml_parser.py. When NVIDIA's server-side tool-call parser consumes the head of a DSML block, only the closing tags survive in message.content. These residuals are never legitimate text, so all DSML tags outside parsed tool-call blocks are now stripped, ensuring Claude Code never sees raw markup.
  • Missing tool_use blocks in buffered→SSE conversion - Emitted tool_use blocks when converting buffered responses to SSE.
  • SERVER_TYPE config - Fixed SERVER_TYPE handling and added DeepSeek-V4-Pro DSML tool-call parsing for stream mode.
  • NotFoundError & overloads retryability - NotFoundError and overload errors are now treated as retryable.
  • Bot protection bypass for localhost - Localhost requests are ignored by bot protection.
  • Mid-stream truncation retry - Streams that end without a finish_reason (NVIDIA backend cut off mid-response) now trigger a retry instead of silently returning a partial response. Previously only completely empty streams were detected.
  • Mid-stream connection drop retry (stream mode) - Connection errors (APIConnectionError, APITimeoutError, httpx.ReadError, httpx.TimeoutException) raised during chunk iteration are now retried in streaming mode. Previously they were treated as non-retryable and emitted an incomplete SSE sequence (error event without message_stop), causing Claude Code to fail with "empty or malformed response (HTTP 200)".
  • Stream retry duplicate message_start - Fixed a bug where a retried stream emitted a second message_start into the buffered SSE (one per attempt), leaving Claude Code with two message sequences and failing with "empty or malformed response (HTTP 200)". stream_response now keeps only the final attempt's events (from its last message_start onward), dropping partial earlier attempts.
  • stop_reason mismatch with streamed tool_use blocks - When DeepSeek emits tool calls as DSML markup, OpenAI's finish_reason is "stop" even though the proxy streams tool_use content blocks. Claude Code requires stop_reason="tool_use" when tool_use blocks are present and rejected the delivered end_turn + tool_use message as "empty or malformed response (HTTP 200)". The stop_reason guards in both streaming and buffered paths now also upgrade end_turn/other → tool_use when tool_use blocks were actually streamed (tracked via a new tool_blocks_started counter in SSEBuilder).
  • Reasoning budget sets max_tokens - The reasoning budget (from effort config, <nimeffort:level:budget> tag, or session store) now directly sets max_tokens instead of being silently capped. This allows models like deepseek-v4-flash-0731 to use their full reasoning capability (e.g. <nimeffort:ultracode:100> → max_tokens=100). DeepSeek uses reasoning_effort in chat_template_kwargs to control thinking intensity; Nemotron uses reasoning_budget in extra_body.
  • Nimeffort combined format - The <nimeffort> tag now supports <nimeffort:level:budget> format (e.g. <nimeffort:ultracode:100> sets effort to ultracode with a 100-token budget). The budget directly controls max_tokens. Budget is stored in the session store by _handle_nimeffort in routes.py and retrieved per-request.
  • DeepSeek reasoning_budget exclusion - reasoning_budget parameter is no longer sent to DeepSeek models (which reject it with 400 errors). Only Nemotron/default styles send it. DeepSeek uses reasoning_effort in chat_template_kwargs + max_tokens instead.
  • Partial response delivery on truncated SSE - Truncated NVIDIA streams with partial content (text, reasoning, or tool calls) now close their blocks and deliver the partial response instead of failing. Only empty (or empty-but-truncated) streams raise StreamTruncatedError for retry.
  • Partial-response proxy notice - Truncated/partial responses delivered to the client are appended with a visible notice ([PROXY NOTICE] ... partial reply ... split the work into smaller chunks) so the model knows the reply was cut short and can adapt instead of assuming the task finished.
  • Partial tool-use stop reason - The provider no longer emits end_turn for streamed tool-use blocks; it sets stop_reason="tool_use" when tool blocks were started, matching the normal completion path.
  • Reasoning-only partial stream retry - When a stream cuts mid-response with only reasoning content and no assistant text or tool calls, it is now treated as truncation and retried instead of delivering an untouchable thinking block to Claude Code.
  • httpx.RemoteProtocolError retry - NVIDIA's "incomplete chunked read" mid-body disconnects (httpx.RemoteProtocolError) are now retryable.
  • Unique tool_use IDs (Kimi-k3 deadlock fix) - _fresh_tool_id() always mints a globally-unique Claude-style tool_use id instead of forwarding NIM's raw tool_call id. Kimi-k3 returns ids in Name:index format (Bash:0, Read:1) that repeat per same-named call across turns; Claude Code de-duplicates assistant tool_use blocks on id, strips the repeated block, and replaces the turn with [Tool use interrupted] — an empty re-prompt loop. Always-fresh ids (like the real Anthropic API) fix the deadlock.
  • Bot protection localhost exemption & /api/hello - Localhost requests no longer accumulate failed-attempt strikes toward a ban in record_failed_attempt, and a keyless GET /api/hello route was added (returns {}) so clients that probe it don't trigger bot protection. @api/app.py and api/bot_protection.py.
  • Optimization SSE defaults - optimization_response_to_sse now falls back to a fresh msg_* id and "unknown" model when a mocked optimization response omits them, instead of emitting a malformed message_start.
  • Reasoning config module-cache fix - _load_reasoning_config now assigns the parsed JSON back to the _REASONING_CONFIG_RAW module cache, so runtime edits to reasoning_config.json are honored instead of the stale initial value.
  • sse_content_block_stop standalone helper - SSE builder exposes a standalone content_block_stop event emitter (no full SSEBuilder state needed) for stream-retry de-dup to close already-opened blocks; _keepalive typing tightened to AsyncGenerator.
  • Duplicate RequestQueue.get_stats removed - Removed the duplicate method definition (the @property get_stats was the remaining one).
  • kimi-k3 400/top_p fix & reasoning style - Force immutable top_p=0.95 for moonshotai/kimi-k3 (the server rejects any other value) and add a kimi thinking style that sends top-level reasoning_effort (low/high/max) via extra_body.
  • Real-time streaming delivery - Provider SSE events now stream to the client as they arrive instead of fully buffering, so the first token reaches Claude Code quickly and idle timeouts are avoided. This superseded the earlier SSE keepalive workaround (_keepalive/_buffered_stream removed).
  • Unlimited truncation retry - provider_retry_on_truncation default changed from 3 to 0, where 0 now means unlimited retries on truncated/mid-stream responses so the client always receives a complete message sequence.
  • Error diagnostics & transport logging - Enriched retry/error logs with request/model context, timeout subtype, __context__/__cause__ chains, total elapsed summaries, and periodic "still retrying" info; header capture now emits a timed failure line on transport timeout/connection errors.

Changed

  • SERVER_TYPE now defaults to buffer - config/settings.py default changed from stream to buffer (unset SERVER_TYPE now uses buffered mode with unlimited truncation retry). stream remains explicitly selectable via SERVER_TYPE=stream or <nimserver:stream>. This brings the code default in line with the reliability-first retry model.
  • reasoning_config.json kimi-k3 budgets - moonshotai/kimi-k3 efforts now use explicit token budgets (low:1000 → ultracode:32000) instead of -1, so the kimi reasoning style benefits from the budget→max_tokens behavior.
  • Discord bot & rate-limiter hardening - Tightened Discord channel/guild type-safety and checks (isinstance guards, getattr fallbacks), a minor rate-limiter typing fix, updated Playwright stealth usage, and a setup_wizard.py pyright ignore. Tests updated to async generators and aligned timeout-error construction.

Removed

  • UV references - Removed uv.lock and UV_* references from pyproject.toml, server.py, and start_server.py.
  • minimaxai/minimax-m3 from reasoning config - Dropped the Minimax entry from reasoning_config.json because the model was removed from the NVIDIA endpoint.

Files Touched

  • providers/provider.py - Inline thinking split logic; DeepSeek-V4 DSML parsing; retryable error handling.
  • providers/think_parser.py - split_think_content for separating CoT from content.
  • providers/dsml_parser.py - Rewritten DSML tool-call parsing for stream/buffered modes; _strip_residual_tags for dangling DSM...
Read more

v2.0.14

Choose a tag to compare

@cyberofficial cyberofficial released this 17 Aug 03:35

v2.0.14 - Date: 2026-08-16

Which Build for me?

Nimbus Lite - You Dont care about the web_search mcp
Nimbus - You care about websearch mcp

Fixed

  • Thinking block initialization in streaming mode - Fixed an issue where thinking blocks were not being started at the beginning of streaming responses when NIM_THINKING was enabled, causing thinking content to only appear after the stream completed. The fix correctly initializes a thinking block at the start of streaming when thinking is enabled and no other block has been started yet, allowing thinking to stream incrementally in Claude Code.
  • Setup wizard improvements:
    • Added prompt for resource_exhausted retries configuration in the setup wizard.
    • Fixed .env writing to use user-selected values for resource_exhausted_retries, nim_reasoning_budget, nim_enable_thinking, show_nim_reply, and other settings instead of hardcoded defaults.

Files Touched

  • providers/provider.py - Added import of get_settings from config.settings and corrected thinking block initialization to use settings.nim.thinking and sse.blocks.
  • setup_wizard.py - Added prompt for resource_exhausted retries configuration and fixed .env writing to use user-selected values instead of hardcoded defaults.

v2.0.13

Choose a tag to compare

@cyberofficial cyberofficial released this 23 Jul 07:08

v2.0.13 - Date: 2026-07-20

Which Build for me?

Nimbus Lite - You Dont care about the web_search mcp
Nimbus - You care about websearch mcp

Added

Token Limit Enforcement & Reasoning Budget Validation

  • NIM_MAX_TOKENS default reduced from 202000 to 32000 for more reasonable token limits
  • Token upgrade logic -- When NIM_MAX_TOKENS is set higher than what Claude sends (e.g., 32000 vs 256000), the proxy now upgrades the request's max_tokens to the configured value instead of capping down
  • Reasoning budget validation -- If reasoning_budget >= max_tokens (after upgrade), sets reasoning_budget=-1 (unlimited) and relies on max_tokens as the effective cap to prevent NVIDIA rejection
  • Explicit logging -- Logs both token upgrades and budget adjustments for audit trail

Empty NIM Stream Handling & Provider Resilience

  • Empty stream detection -- NvidiaNimProvider now detects empty/malformed NVIDIA NIM streaming responses (stream finishes with no finish_reason and no content: text, reasoning, or tool states) and raises StreamTruncatedError to trigger retry logic
  • Request queue attribute on BaseProvider -- Added default _request_queue = None on BaseProvider to enable queue stats in api/routes.queue_status with proper typing
  • Rate limit slot release on pre-response failures -- Buffered requests that fail before reaching the server now properly release rate-limit slots via release_last_slot()
  • Expanded retryable errors -- execute_with_rate_limit now treats httpx.TimeoutException, APIConnectionError, APITimeoutError, and 5xx server errors as retryable; releases slots, logs, and applies exponential backoff

Discord Bot Stability & DM Handling

  • DM/channel guards -- Added defensive checks for interaction.channel being None or not a messageable channel; returns early instead of crashing
  • Missing channel/category guards -- In discord_bot/views.py, validates resolved category is a discord.CategoryChannel before use; returns clear ephemeral error if not
  • Fallback channel/guild names -- Backup/backup commands now use getattr(channel, 'name', 'dm') and getattr(channel.guild, 'name', 'Unknown Guild') for DM-friendly filenames and messages
  • Conversation wrapper ergonomics -- add_message now calls add_message_with_user with explicit keyword arguments for tools_used and tool_results

Type Safety, Null Checks & Pylance Compliance

  • Added type: ignore comments for mypy/pyright compliance (sys._MEIPASS, termios, attribute access on unions)
  • Defensive null checks for model_name, detail, extracted, new_budget variables
  • Fixed bugs: removed malformed duplicate code in _rebuild_without_key(), corrected _has_thinking_params() removal
  • Simplified tiktoken cache logic by removing fallback handling
  • Fixed priority enum casting in request_queue.py (RequestPriority(priority) -> RequestPriority(priority))
  • Fixed pyright directive in config/nim.py (changed to # pyright: ignore[reportCallIssue])

Discord Bot & Views Resilience

  • CategoryChannel validation -- CreateChannelModal and list_channels button now verify interaction.guild.get_channel() returns a CategoryChannel before use
  • DM/guild guards -- Commands requiring guild channels now explicitly check interaction.guild first and return early for DMs
  • Member vs User typing -- CompactConfirmView and related classes use discord.abc.User to accept both User and Member

Pylance/Type Checking Fixes

  • Fixed pyright directive in config/nim.py (invalid disable= syntax -> ignore[...])
  • Added # type: ignore[union-attr] and # type: ignore[attr-defined] suppressions for union attribute access patterns
  • Added cast(Literal["user", "assistant", "system"], ...) for message role literal types
  • Fixed reportIndexIssue for Message indexing in bot.py

Discord Bot DM/Channel Safety

  • All command handlers now guard with if not interaction.guild: return before using interaction.channel
  • interaction.channel accesses wrapped with if channel is None: return
  • Channel name access uses getattr(channel, 'name', 'dm') for DM friendliness
  • isinstance(category, discord.CategoryChannel) check before using as category

Conversation API Ergonomics

  • add_message now calls add_message_with_user with keyword arguments for tools_used and tool_results instead of positional

Changed

NIM_MAX_TOKENS Behavior

  • Default reduced from 202000 to 32000 in code and .env.example
  • Upgrade logic added -- if request max_tokens < NIM_MAX_TOKENS env value, proxy upgrades to env value (log: "max_tokens X upgraded to NIM_MAX_TOKENS: Y")
  • Removed fallback to nim.max_tokens when request doesn't provide max_tokens; lets NIM model use its own defaults

.env.example Reorganization

  • Full restructure with section headers (CORE SERVER, NVIDIA NIM, PROVIDER RATE LIMITING, REQUEST QUEUE, MCP, DISCORD BOT, COMMAND TOGGLES)
  • Removed duplicated entries and improved examples/comments
  • NIM_MAX_TOKENS=32000 default shown

Rate Limiter Improvements

  • Method renamed -- reset_adaptive_backoff() -> reset_reactive_block()
  • UI messaging -- "Resets in" -> "Next Slot free in" for clarity
  • New release_last_slot() -- Explicit slot release for pre-response failures
  • Retriable error expansion -- Now handles httpx.TimeoutException, openai.APIConnectionError, openai.APITimeoutError, 5xx; releases slots, logs, applies exponential backoff
  • cast(MessagesResponse, response) -- Type-safe casting in optimization handler

Discord Bot DM/Channel Safety

  • All command handlers now guard with if not interaction.guild: return before using interaction.channel
  • interaction.channel accesses wrapped with if channel is None: return
  • Channel name access uses getattr(channel, 'name', 'dm') for DM friendliness
  • isinstance(category, discord.CategoryChannel) check before using as category

Conversation API Ergonomics

  • add_message now calls add_message_with_user with keyword arguments for tools_used and tool_results instead of positional

Fixed

  • Various Pylance/pyright type checking issues across codebase
  • Rate limit status display: "Resets in" -> "Next Slot free in" for clarity
  • config/nim.py pyright directive syntax (invalid disable= -> ignore[...])
  • Empty/malformed NIM stream handling (no finish_reason, no content -> retry)
  • BaseProvider missing _request_queue attribute
  • Discord DM crashes (missing channel/category/category type)
  • Rate limit slot leaks on pre-response failures
  • Conversation.add_message positional arg mismatch for tools_used/tool_results
  • Priority enum casting in request_queue.py (RequestPriority(priority) -> RequestPriority(priority))

Removed

  • Fallback to nim.max_tokens when request doesn't provide max_tokens (lets NIM model use its own defaults)

Files Touched

  • .env.example -- Full restructure with clarified defaults, removed duplicates, improved comments; NIM_MAX_TOKENS=32000 default
  • config/nim.py -- max_tokens default from 202000 -> 32000; pyright directive fix
  • providers/request.py -- Token upgrade logic, reasoning budget validation, logging
  • api/app.py -- Rate limit status message update
  • api/routes.py -- Queue status endpoint with _request_queue typing comment; DM/guild guards
  • api/optimization_handlers.py -- Type-safe casting
  • providers/request.py -- Token upgrade logic, reasoning budget validation, logging
  • providers/request_queue.py -- Priority enum casting fix
  • providers/rate_limit.py -- release_last_slot(), expanded retryable errors, reset_reactive_block(), exponential backoff
  • providers/provider.py -- Empty stream detection, StreamTruncatedError, rate limit slot release, BaseProvider._request_queue
  • providers/base.py -- _request_queue = None, _client = None
  • discord_bot/bot.py -- Type ignores for union attributes, DM message guards, getattr for channel names
  • discord_bot/cog.py -- getattr for channel/guild names, isinstance CategoryChannel checks, interaction.channel None guards, discord.abc.User typing, keyword args in add_message
  • discord_bot/views.py -- CategoryChannel validation, client.settings type ignores
  • discord_bot/conversation.py -- Keyword args for add_message_with_user
  • config/nim.py -- Pyright directive fix
  • config/settings.py -- (minor)
  • providers/base.py -- _request_queue = None, _client = None
  • providers/request.py -- Token upgrade, budget validation
  • providers/rate_limit.py -- release_last_slot(), retryable error expansion, reset_reactive_block(), backoff
  • packaged_entry.py -- (typing)
  • api/optimization_handlers.py -- cast(MessagesResponse, response)
  • tests/providers/test_nvidia_nim_request.py -- Updated tests for upgrade behavior
  • tests/providers/test_nvidia_nim.py -- Updated test expectations
  • tests/api/test_dependencies.py -- Mock settings fix

v2.0.12 - Adaptive Rate Limit Redesign & Thinking Fallbacks

Choose a tag to compare

@cyberofficial cyberofficial released this 18 Jul 07:17

v2.0.12 - Date: 2026-07-18

Which Build for me?

Nimbus Lite - You Dont care about the web_search mcp
Nimbus - You care about websearch mcp

Added

Rate Limiting (Adaptive Cap on 429)

  • Local adaptive RPM cap - On 429, caps effective RPM to the current sliding-window rate (actual throughput). After NIM_RPM_RESET (default NIM_RPM_RESET=5) successful requests, restores +1 RPM incrementally until reaching initial RPM.
  • Progressive decrement at cap - If 429 occurs again while capped, decrements RPM by 1 (floor = 1). Recovery follows same gradual restore.
  • Server-header-driven proactive limit - x-ratelimit-limit updates current RPM when not locally capped; x-ratelimit-reset updates rate window.
  • Reactive block from retry-after - 429 responses trigger blocking via retry-after header; <nimrpm:reset> clears block and cap immediately.
  • Auto-restore on success streak - Success counter resets on 429; after N successes without 429, recovers 1 RPM (when capped) or clears cap (when recovered).

Thinking / Effort Fallback for Nemotron Models

  • Effort-flag fallback - Models that accept enable_thinking but reject *_effort flags (low/medium/high_effort) now auto-retry with effort flags stripped, keeping enable_thinking=true. Model added to runtime cache on first error.
  • Reasoning_budget handling - reasoning_budget errors detected and handled via existing thinking-param fallback path.
  • Log buffering for streaming - THINKING/REPLY log lines buffered (120 chars) to reduce log spam under high throughput; flushed on finish_reason or stream end.

Inline Commands: Session ID for Effort Tracking

  • Per-session effort via x-claude-code-session-id - <nimeffort:level> and <nimeffort> now key off the x-claude-code-session-id header (falls back to API key). Fixes effort persistence across API key rotations.

Changed

Rate Limiter Redesign (providers/rate_limit.py)

  • Removed legacy adaptive params - NIM_RPM_INITIAL, NIM_RPM_DROP, NIM_RPM_MIN, NIM_RPM_HOLD_INITIAL, NIM_RPM_HOLD_MAX deleted from code, config, .env.example, wizard.
  • NIM_RPM_RESET semantics changed - Was: auto-restore after N seconds without 429 (default 300s). Now: auto-restore after N successful requests without 429 (default NIM_RPM_RESET=5).
  • Single RPM tracker - _current_rpm replaces _effective_rpm + _initial_rpm + _hold_delay + _drop_count. Source: server header or fallback.
  • New state - _capped_rpm (int | None) tracks local adaptive cap; _success_count tracks consecutive non-429 requests.
  • API changes - on_rate_limit_hit() → on_rate_limited(), reset_adaptive_backoff() → reset_reactive_block(), added on_success(), current_rpm() property.
  • get_status() output changed - Now returns current, max, initial_max, remaining, reset_in_seconds, is_blocked, blocked_seconds_remaining, worker_limit, worker_available, capped_rpm, initial_rpm. Removed effective_rpm, hold_delay, drop_count.

Environment & Config

  • New HOST variable - HOST=0.0.0.0 (bind address) added to .env.example.
  • NIM_ENABLE_THINKING=true - Global thinking toggle added to .env.example (read directly by provider, not via Pydantic settings).
  • FABLE_OVERRIDE="" - Uncommented in .env.example (was commented).
  • NIM_RPM_RESET=5 - Default changed from 300 (seconds) to 5 (requests).
  • Direct env vars section - Added TIKTOKEN_CACHE_DIR, CLAUDE_CONFIG_DIR, XDG_CONFIG_HOME, UV_PROJECT_ENVIRONMENT, WEB_SEARCH_DEBUG=false, PLAYWRIGHT_BROWSERS_PATH to .env.example (read directly by components, not via settings).
  • Optimization flags use Field(validation_alias=...) - fast_prefix_detection, enable_network_probe_mock, enable_title_generation_skip, enable_suggestion_mode_skip, enable_filepath_extraction_mock, enable_recap_skip now explicitly declare env aliases.

Model Override & Request Building

  • model_override passed to build_request_body() - Ensures thinking params match the swapped model, not the original Claude tier model.
  • Session ID from request header - _handle_nimeffort and _handle_nimeffort_status use request_data.session_id (from x-claude-code-session-id) instead of API key.

Documentation Updates (README.md)

  • Adaptive rate limiting section rewritten - Now describes server-header-driven proactive limit + reactive retry-after block + auto-restore on success streak.
  • Env var table updated - Removed legacy NIM_RPM_* vars; added NIM_RPM_RESET, PROVIDER_RATE_LIMIT, PROVIDER_RATE_WINDOW.
  • <nimrpm:reset> description updated - "Clear reactive rate limit block immediately (clears retry-after block)".

Fixed

  • Effort tracking tied to API key - Fixed by using x-claude-code-session-id header for per-session effort storage (commit a485eee).
  • Model override ignored thinking params - model_override now passed to build_request_body() so swapped models get correct thinking config (commit df7c5cc).
  • Rate limiter used hardcoded defaults - Fixed by removing legacy params; now reads only PROVIDER_RATE_LIMIT, PROVIDER_RATE_WINDOW, NIM_RPM_RESET from settings (commit 15ed3ed).
  • Stale rate limit on header updates - x-ratelimit-limit now updates _current_rpm only when not locally capped; x-ratelimit-reset updates window dynamically (commit 236d67f).

Removed

  • Legacy adaptive rate limit env vars - NIM_RPM_INITIAL, NIM_RPM_DROP, NIM_RPM_MIN, NIM_RPM_HOLD_INITIAL, NIM_RPM_HOLD_MAX removed from codebase, .env.example, settings, wizard, README.
  • ProviderConfig legacy fields - rpm_drop, rpm_min, hold_initial, hold_max removed from providers/base.py.
  • Settings legacy fields - nim_rpm_initial, nim_rpm_drop, nim_rpm_min, nim_rpm_hold_initial, nim_rpm_hold_max removed from config/settings.py.
  • Test-compat _rate_limit property - Kept temporarily in rate_limit.py for test compatibility (deprecated).

Files Touched

Core Rate Limiting

  • providers/rate_limit.py - Complete rewrite: adaptive cap, success-streak restore, header-driven limits, reactive retry-after block
  • providers/base.py - Removed legacy ProviderConfig fields; rpm_reset now int (requests)
  • config/settings.py - Removed legacy nim_rpm_* fields; added HOST, NIM_ENABLE_THINKING; optimization flags use Field(validation_alias=...)
  • api/app.py, api/dependencies.py - Removed legacy params from GlobalRateLimiter.get_instance() calls
  • setup_wizard.py - Removed legacy RPM prompts; added NIM_RPM_RESET (requests) prompt
  • .env.example - Added HOST, NIM_ENABLE_THINKING, FABLE_OVERRIDE, NIM_RPM_RESET; removed legacy vars; added direct env vars section
  • README.md - Rewrote adaptive rate limiting section; updated env var table; updated <nimrpm:reset> description

Provider & Request Building

  • providers/provider.py - Effort-flag fallback cache/logic; _rebuild_without_effort_flags(); log buffering (120 chars); on_success()/on_rate_limited() calls in stream/buffered paths
  • providers/request.py - model_override passed to build_request_body()
  • providers/thinking_effort_unsupported_cache (module-level cache added in provider.py)

Inline Commands / Sessions

  • api/routes.py - _handle_nimeffort, _handle_nimeffort_status use request_data.session_id (header x-claude-code-session-id) with API key fallback; removed blank lines
  • api/models/anthropic.py - session_id field on request model (already existed, now used)

Config & Documentation

  • reasoning_config.json - Added reasoning_budget entries (diff shows +12 lines)

NIMbus v2.0.11 - Inline Commands, Thinking Styles, Discord & MCP Enhancements

Choose a tag to compare

@cyberofficial cyberofficial released this 17 Jul 08:08

v2.0.11 - Date: 2026-07-17

Added

Inline Commands & Reasoning Control

  • <nimhelp> Inline Command - Send exactly <nimhelp> for a formatted list of all inline commands. Works in both streaming/buffered endpoints; not gated by SWAPPER_ENABLED.
  • <nimeffort:level> Tag Handler - Override reasoning effort per-message: low, medium, high, xhigh, max, ultracode. Stores per session (x-claude-code-session-id), returns mock response (no NVIDIA API call). Whitespace handling matches <nimrpm:reset>.
  • <nimeffort> / <nimeffort:status> - Show current session's stored effort level, mapped effort, and custom budget.
  • Custom Reasoning Budget - <nimeffort:NNN> accepts -1 to 1,000,000 (-1 = unlimited). New api/effort_store.py tracks per-session effort + budget separately. Priority: exact tag > request > session > config.
  • -1 Budget = Unlimited - Validator accepts -1, rejects < -1. Passed through to NVIDIA API per spec (disables reasoning token limit).
  • Ultracode Budget - reasoning_config.json sets ultracode budget = -1 (unlimited) for all Nemotron models.

Discord Bot Enhancements

  • Web Search Persistence - Tool results saved in conversation history (tool_results field). New get_history_with_tool_results() injects prior results as context. Config: DISCORD_WEB_SEARCH_MAX_RESULT_SIZE (5000), DISCORD_WEB_SEARCH_INCLUDE_IN_HISTORY (true).
  • Playwright Browser Fetch - When DISCORD_BROWSER_HEADLESS=false, forces browser for ALL fetches (30s timeout vs 10s HTTP). Reuses DuckDuckGo browser/context (cookies, stealth).
  • Embedded JSON Tool Call Parsing - Detects {"tool": "search", "query": "..."} in text streams (Nemotron 3 Ultra) → converts to tool_use events.
  • Empty Reply Fix - Final fallback retries 3× when model returns tool_use instead of text; adds "provide text answer only" prompt each retry; rejects tool_use in final response.
  • Smart Message Splitting - _split_at_word_boundary prefers newlines then spaces (preserves URLs); uses DISCORD_SPLIT_THRESHOLD (default 1900).
  • Web Search Resilience - "No results found" = valid result (not failure), reducing false tool removal. Buffered retries 3→5. Fallback synthesis when retries fail but tool results exist.

MCP Server

  • Browser Support for fetch_page - New MCP_BROWSER_HEADLESS (default true = HTTP only). use_browser param forces Playwright when headless=true. Reuses DuckDuckGo browser instance. Logs fetch mode.

Setup Wizard & Configuration

  • Extended Wizard Coverage - New prompts: request queue priorities (Discord HIGH / API NORMAL), Fable model override (FABLE_OVERRIDE), browser headless modes (MCP/Discord), web search debug logging (WEB_SEARCH_DEBUG), Discord web search settings.
  • Model-Specific Thinking Styles - New thinking_style in reasoning_config.json:
    • Nemotron: chat_template_kwargs.enable_thinking + reasoning_budget
    • DeepSeek: chat_template_kwargs.enable_thinking (boolean)
    • Minimax: chat_template_kwargs.thinking_mode = enabled/adaptive/disabled
    • Default: effort flags + reasoning_budget (backward compat)
    • Models NOT in config (or supports_thinking: false) send NO thinking params - NVIDIA defaults apply.
  • nim: Prefix Model Lookup - nim:minimax-m3 bypasses tier mapping, resolves via NVIDIA catalog.
  • SHOW_NIM_REPLY Toggle - Echo raw NVIDIA reply live to console (timestamped; THINKING = reasoning_content, REPLY = generated text). Both stream/buffer modes.
  • Discord Web Search Result Size - DISCORD_WEB_SEARCH_MAX_RESULT_SIZE (5000), DISCORD_WEB_SEARCH_INCLUDE_IN_HISTORY (true).

Nemotron Thinking Support

  • reasoning_budget - Max thinking tokens (model caps, max 32768)
  • chat_template_kwargs - enable_thinking + low_effort/medium_effort/high_effort flags
  • Effort→Budget: max/high/xhigh → max, medium → half, low → 2048
  • Effort→Flags: xhigh/high/max → top, medium → medium, low → low
  • Verified: Nemotron 3 Ultra (500B) & Nemotron 3 Super (120B) return reasoning content

Per-Session Reasoning Effort Tracking

  • Captures x-claude-code-session-id; in-memory per-session effort cache
  • Caches explicit request efforts; helper to clear session effort
  • Added "ultracode" level + Nemotron mappings/budgets

Rate Limiter Auto-Restore

  • New NIM_RPM_RESET (default 300s, 0 = disabled). After last 429, if no new 429 for duration, auto-restores initial RPM + clears hold delay. Complements manual <nimrpm:reset>.

THINKING Log Line

  • New INFO log: THINKING: <model> using style=<style> effort=<raw> -> <mapped> [budget=<N>]. Logs when thinking disabled or model not in config.

Changed

  • Reasoning Config Consolidation - Single reasoning_config.json replaces 3 files (reasoning_budgets.yaml, reasoning_effort_presets.json, user_reasoning_effort_mapping.json). Includes thinking_style, effort_mapping, budget_per_effort with glob patterns. Legacy files deleted.
  • Model Resolution for Config Lookup - Uses resolved NIM model ID (from get_model_for_claude) for reasoning config lookup. Fixes nim:minimax-m3 → minimaxai/minimax-m3.
  • Rate Limiter Initialization - GlobalRateLimiter initialized early in FastAPI lifespan with .env settings before sidecars trigger singleton.
  • Removed NIM_ENABLE_THINKING - Redundant flag removed; thinking now solely controlled by NIM_THINKING. Simplifies build_request_body.
  • Inline Command Detection (Multi-Block) - extract_last_text_content() checks only last content block (user's command), avoiding false positives from system prompts/context. All 5 handlers updated. Buffered endpoint mock responses use .model_dump().
  • Buffered Endpoint Inline Commands - <nimhelp> and <nimrpm:reset> now work in buffered endpoint, positioned BEFORE wait_if_blocked() to bypass rate limiting during backoff.
  • SSE Dispatch Consolidation - 3 copy-pasted SSE blocks → single _sse_response() helper. All 4 command dispatches now one-liners. Net -30 lines.
  • Default Model - deepseek-ai/deepseek-v4-flash → nvidia/nemotron-3-super-120b-a12b in .env.example and setup wizard.

Fixed

  • Rate Limiter Defaults - Was using hardcoded defaults (drop=10, min=20) instead of .env values (NIM_RPM_DROP=2, NIM_RPM_MIN=2). Now reads from settings at startup.
  • Minimax Effort Mapping - Efforts now map directly to thinking_mode values (disabled/adaptive/enabled) instead of budget numbers.
  • Streaming Idle Timeout - Disabled idle read timeout (set to None) to prevent timeout errors on sporadic chunk arrival.
  • Inline Command False Positives - Detection now checks only last content block, enabling inline commands with Claude Code's concatenated format (system reminder + user command blocks).
  • Tiktoken Special Token Crash - Added _safe_encode() helper in providers/sse_builder.py that catches ValueError from disallowed special tokens (e.g., <|endoftext|> from Nemotron models) and retries with disallowed_special=() to encode them as regular text. Prevents crash in estimate_output_tokens() during streaming.

Files Touched (Key Changes)

  • api/routes.py - Inline command handlers, _sse_response(), buffered commands, multi-block detection
  • api/swapper/parser.py - <nimeffort:level>, <nimhelp> parsers; <nimrpm:reset> exact-match
  • api/effort_store.py - New: Per-session effort + custom budget tracking
  • api/dependencies.py - GlobalRateLimiter.get_instance() with settings fallback; SHOW_NIM_REPLY
  • api/app.py - Early GlobalRateLimiter init in lifespan
  • api/middleware.py - Session ID capture
  • api/models/anthropic.py - Session ID field
  • providers/request.py - Thinking param rewrite (thinking_style-based); model resolution via NIM ID; -1 budget pass-through; THINKING log; removed session cache
  • providers/provider.py - SHOW_NIM_REPLY live logging; Nemotron thinking errors; streaming timeout; Discord log cleanup
  • providers/base.py - Adaptive rate limit fields; SHOW_NIM_REPLY in config
  • providers/rate_limit.py - rpm_reset, _last_429_time, auto-restore; _rate_limit property (test compat)
  • providers/sse_builder.py - _safe_encode() helper for special tokens; estimate_output_tokens() uses safe encoding
  • providers/text.py - extract_last_text_content() for multi-block detection
  • config/settings.py - nim_rpm_reset, SHOW_NIM_REPLY, ReasoningConfig + thinking_style, glob matching, Discord web search, wizard additions
  • config/nim.py - -1 budget validator; reasoning effort validation
  • reasoning_config.json - New: Consolidated config with thinking_style for all models; Minimax→thinking_mode; ultracode=-1 for Nemotron; glob patterns
  • discord_bot/bot.py - Web search persistence, Playwright fetch, JSON tool parsing, empty reply fix, smart splitting, resilience
  • discord_bot/conversation.py - tool_results field, get_history_with_tool_results()
  • discord_bot/persistence.py - Tool results serialization
  • discord_bot/tools/web_search.py - Fetch page schema, result size/history config
  • mcp_server.py - Browser support for fetch_page
  • setup_wizard.py - Extended config sections
  • .env.example - All new env var docs
  • config/logging_config.py - Timestamped log files per session
  • providers/error_mapping.py - 403 handling with NVIDIA key URL
  • server.py - Logging init at entry point
  • Deleted: config/reasoning_budgets.yaml, reasoning_effort_presets.json, user_reasoning_effort_mapping.json

Full Changelog: See CHANGELOGS.md


Assets: Binary builds will be attached on publish.

v2.0.10 - Fable Model Support, Auth Errors, Thinking Fallback, Log Rotation Fix

Choose a tag to compare

@cyberofficial cyberofficial released this 12 Jul 03:47

Fixed

  • Expired/Invalid API Keys - 403 Forbidden errors now show a clear message with a direct link to verify the NVIDIA API key: "Invalid or expired API key. Verify your key at https://build.nvidia.com/settings/api-keys"
  • nimrpm:reset False Positives - The adaptive rate limit reset tag now only triggers when it's the entire message content (optional whitespace allowed), preventing false positives from embedded content in file outputs, system prompts, or documentation
  • Thinking Parameter Compatibility - Models that don't support thinking/reasoning parameters (e.g., z-ai/glm-5.2) now automatically retry without thinking parameters on 500 errors; the model is cached as unsupported to skip thinking params on subsequent requests
  • Log Rotation on Windows - Fixed PermissionError: [WinError 32] The process cannot access the file because it is being used by another process during log rotation by switching to timestamped log files per session (no mid-session rotation). Each server start creates a new server.YYYY-MM-DD_HH-MM-SS_XXXXXX.log file; no server.log file is created anymore.
  • Windows Log Rotation TypeError - Fixed TypeError: argument should be a str or os.PathLike object, not 'TextIOWrapper' during log rotation by removing custom rotator that received file object instead of path

Added

  • Fable Model Support (FABLE Model Override) - New FABLE_OVERRIDE environment variable to configure the NIM model for the new Claude "Fable" model tier. Defaults to the Opus NIM model but can be overridden with any NIM model ID (supports [1m] context suffix like other models)

Changed

  • execute_with_retry Retry Logic - max_retries=0 now correctly means "infinite retries" (previously was treated as 1 retry); useful for critical background operations that should never give up
  • Logging Configuration - Moved logging initialization to entry points (server.py, packaged_entry.py, start_server.py) before importing api.app so log file path is set correctly for each run

Files Touched

  • api/swapper/parser.py - Fixed <nimrpm:reset> regex to require exact message match
  • config/logging_config.py - Timestamped log files per session, no rotation during runtime; removed custom rotator
  • config/settings.py - Added Fable model detection in get_model_for_claude(); added FABLE_OVERRIDE field with [1m] suffix stripping
  • providers/error_mapping.py - Added PermissionDeniedError handling with NVIDIA key URL
  • providers/provider.py - Added thinking parameter fallback on 500 errors; model caching for unsupported models
  • providers/rate_limit.py - Fixed max_retries=0 to mean infinite retries
  • server.py - Configure logging at entry point before importing app
  • .env.example - Added FABLE_OVERRIDE documentation and example

v2.0.9 - Portable Build, Role Mentions, Streaming Fixes, Swapper Resilience

Choose a tag to compare

@cyberofficial cyberofficial released this 06 Jul 05:27

Added

  • Portable Build Browser Bundling - Playwright Chromium pre-downloaded at build time and embedded via --add-data into single-file exe (~25 MB); no external ms-playwright folder needed; runs on any Windows machine without playwright install
  • Discord Bot Role Mention Support - Bot now responds when mentioned via role (@&role_id) in addition to direct user mentions; checks if any mentioned role is assigned to the bot

Fixed

  • MCP Server Headless Mode - start_server.py now loads .env before starting MCP server, so MCP_BROWSER_HEADLESS=false and DISCORD_BROWSER_HEADLESS=false are properly applied
  • Tool Choice Conversion - Anthropic tool_choice (any/auto/none/tool) correctly converted to OpenAI format in message_converter.py
  • Streaming 429 Retry Logic - Streaming requests now honor PROVIDER_RETRY_ON_TRUNCATION setting (was hardcoded to 3 retries). Buffered mode already respected this setting.
  • Swapper Model Validation Resilience - Added exponential backoff retry logic to model swapper validation (api/swapper/validator.py). Transient failures (rate limits, network errors, 5xx) during model catalog fetch or test request are now retried up to 3 times before reporting failure.
  • Error Mapping - Updated httpx error handling for better error messages.
  • MCP Server Entry Point - server.py --mcp now correctly starts MCP server (stdio transport) instead of proxy server.

Changed

  • Single-File Portable Build - Browsers extracted to _MEIPASS temp dir at runtime via updated runtime-playwright.py hook; source runs use system Playwright install
  • DuckDuckGo Search Robustness - Uses only lite.duckduckgo.com endpoint via Playwright for reliable results; removes fragile HTML endpoint parsing
  • Debug Logging - Final buffered response logged for Discord web search troubleshooting
  • Documentation - Fixed Request Queue version reference in README (was v2.1, now correctly v2.0.7).

Files Touched

  • build_exe.bat - Pre-download browsers, copy to build_resources, bundle via --add-data
  • build_hooks/runtime-playwright.py - Use _MEIPASS for embedded browsers
  • discord_bot/bot.py - Role mention detection, debug logging
  • websearch/duckduckgo_html.py - Source vs portable browser path logic, error handling
  • start_server.py - Load .env before MCP server launch
  • providers/message_converter.py - tool_choice conversion logic
  • requirements.txt - Added playwright, playwright-stealth, pywin32, dotenv
  • nimbus.spec - Full PyInstaller spec with browser datas and hidden imports
  • providers/provider.py - Pass max_retries=config.retry_on_truncation to streaming execute_with_retry calls
  • api/swapper/validator.py - Added _execute_with_retry helper with exponential backoff for catalog validation and model test requests
  • providers/error_mapping.py - Updated httpx error handling
  • server.py - Handle --mcp/--mcpwebsearch flag to start MCP server

v2.0.8 - Request Queue, Worker Slots, Discord Web Search, Adaptive Rate Limiting

Choose a tag to compare

@cyberofficial cyberofficial released this 27 Jun 05:01

Major Features

Built-in Request Queue (Prevents "Worker limit reached" Errors)

NIMbus now includes a production-ready request queue to prevent NVIDIA NIM's hard limit of 32 concurrent requests per worker:

  • Priority lanes: HIGH (Discord bot), NORMAL (API proxy), LOW (background)
  • Configurable limits: Max concurrent (<=32), max queue size (default 600), timeout (default 300s), worker threads (default 4)
  • Two-phase timeout: Wait for worker pickup, then wait indefinitely for processing to complete
  • Observability: GET /queue/status endpoint, worker utilization in /status, DEBUG-level queue depth logs

NVIDIA Worker Slot Accounting

  • Tracks per-worker concurrent requests (32 slots per NVIDIA worker)
  • Buffered and streaming requests acquire worker slots for their full lifecycle
  • Worker limit/available exposed via /status endpoint
  • Rate limiter adapts based on worker availability

Discord Web Search (Enabled by Default)

  • Bot can search web via DuckDuckGo and fetch pages for up-to-date answers
  • Multi-page pagination with deduping
  • Configurable: DISCORD_ENABLE_WEB_SEARCH, MAX_RESULTS, MAX_ITERATIONS
  • Results with disclaimer: "This response used online resources, please verify"

Adaptive Rate Limiting

  • Runtime NVIDIA RPM backoff: NIM_RPM_INITIAL (default 40), NIM_RPM_DROP (default 10), NIM_RPM_MIN (default 20), NIM_RPM_HOLD_INITIAL/_MAX (default 5/10s)
  • Reset with <nimrpm:reset> chat tag
  • Backs off automatically on 429 responses

HTTP Client Timeouts

  • Configurable HTTP_READ_TIMEOUT, HTTP_WRITE_TIMEOUT, HTTP_CONNECT_TIMEOUT

NIM Core Settings

  • NIM_MAX_TOKENS (default 202000) - max output tokens
  • NIM_REASONING_EFFORT (low/medium/high, default high)
  • NIM_REASONING_EFFORT_MAPPINGS - JSON mapping for custom effort levels

Model Swapper

  • Dynamic model switching via SWAPPER_ENABLED chat tag <modelswap:model-name>
  • SWAPPER_TEST_PROMPT and SWAPPER_TEST_TIMEOUT for validation
  • Short names resolved against NVIDIA's live catalog

MCP Cache TTL

  • MCP_CACHE_TTL for MCP web page fetch caching (default 600s, max 3600, 0 = disabled)

Fast Prefix Detection

  • FAST_PREFIX_DETECTION optimization (enabled by default)

Complete Setup Wizard Coverage

  • New sections: Provider Rate Limiting & Adaptive, HTTP Timeouts, NIM Core Settings, Model Swapper, MCP Cache, Discord Web Search
  • Update mode includes all sections

Fixes & Improvements

Retry & Error Handling

  • Detect NVIDIA "ResourceExhausted" (rate limit) and retry with exponential backoff
  • Handle model capability errors: auto-retry with system-to-user, thinking-to-non-thinking conversion
  • Fixed exception tuple bug for JSONDecodeError/OSError
  • Improved SSE error events on terminal failures
  • Discord message processing: up to 3 retries with exponential backoff (1s base)

Separate Retry Count for Resource Exhaustion

  • New RESOURCE_EXHAUSTED_RETRIES (default 10) for NVIDIA worker limit / ResourceExhausted errors (0 = endless)

Discord Bot Enhancements

  • Native replies: message.reply() for native Discord threading
  • Reply awareness: captures replied-to content; responds to replies even with require_mention=true
  • Complete history: user messages saved unconditionally

Configuration & Defaults

  • Default model: Nemotron 3 Super 120B (nvidia/nemotron-3-super-120b-a12b)
  • Setup wizard configures: queue, Discord web search, replies, adaptive rate limiting, HTTP timeouts, NIM settings, model swapper, MCP cache
  • New per-user user_reasoning_effort_mapping.json for custom effort mappings

New Environment Variables

# Request Queue
REQUEST_QUEUE_ENABLED=true
REQUEST_QUEUE_MAX_CONCURRENT=32
REQUEST_QUEUE_MAX_SIZE=600
REQUEST_QUEUE_TIMEOUT=300.0
REQUEST_QUEUE_NUM_WORKERS=4
REQUEST_QUEUE_DISCORD_PRIORITY=2
REQUEST_QUEUE_API_PRIORITY=1

# Discord Web Search
DISCORD_ENABLE_WEB_SEARCH=true
DISCORD_WEB_SEARCH_MAX_RESULTS=10
DISCORD_WEB_SEARCH_MAX_ITERATIONS=10

# Resource Exhaustion Retries
RESOURCE_EXHAUSTED_RETRIES=10

# Adaptive Rate Limiting
NIM_RPM_INITIAL=40
NIM_RPM_DROP=10
NIM_RPM_MIN=20
NIM_RPM_HOLD_INITIAL=5.0
NIM_RPM_HOLD_MAX=10.0

# HTTP Client Timeouts
HTTP_READ_TIMEOUT=300
HTTP_WRITE_TIMEOUT=10
HTTP_CONNECT_TIMEOUT=2

# NIM Core Settings
NIM_MAX_TOKENS=202000
NIM_REASONING_EFFORT=high
NIM_REASONING_EFFORT_MAPPINGS=

# Model Swapper
SWAPPER_ENABLED=false
SWAPPER_TEST_PROMPT=Please reply with pong only, nothing else
SWAPPER_TEST_TIMEOUT=120.0

# MCP Cache
MCP_CACHE_TTL=600

# Fast Prefix Detection
FAST_PREFIX_DETECTION=true

Installation

Windows (recommended): Download nimbus.exe from this release (uploaded separately)

nimbus.exe --init    # Interactive setup
nimbus.exe           # Start proxy

NIMbus v2.0.7: DISCORD_MODEL Config, Discord Bot Model Independence

Choose a tag to compare

@cyberofficial cyberofficial released this 23 Jun 07:42

Added

  • DISCORD_MODEL environment variable - Configure a separate model for the Discord bot independently from the main API proxy's MODEL setting. Resolution order: DISCORD_MODEL > MODEL > windows:settings.json (Opus > Sonnet > Haiku from Claude settings). If no model can be resolved, the Discord bot will skip startup.

Changed

  • Discord bot now uses settings.discord_model instead of model_name for all requests
  • Discord bot logs warning and skips startup if no model can be resolved (prevents running with no model)
  • Setup wizard now prompts for Discord Model in both full wizard and update modes
  • Replaced em dashes with normal dashes in documentation

Files Touched

  • config/settings.py - added discord_model_raw field and discord_model property
  • discord_bot/bot.py - uses discord_model, skips startup if None
  • discord_bot/cog.py - uses discord_model for all requests
  • setup_wizard.py - prompts for Discord Model in full wizard and update modes
  • .env.example - added DISCORD_MODEL documentation
  • README.md - added DISCORD_MODEL to config table and features list (em dashes fixed)
  • CHANGELOGS.md - added v2.0.7 entry (em dashes fixed)

Quick Start

# Windows
nimbus.exe --init    # first time setup
nimbus.exe           # start the proxy
claude               # use Claude Code normally

# MCP Server (global scope - all projects)
claude mcp add web_search -s user -- nimbus.exe --mcp

Full Changelog: v2.0.6...v2.0.7

NIMbus v2.0.6: Discord Bot Stability & Expanded Configuration

Choose a tag to compare

@cyberofficial cyberofficial released this 20 Jun 06:47

A Discord-focused release that fixes three classes of bugs the bot was hitting at scale (Discord interaction timeouts, mass-ping risks, and 429-rate-limited typing indicators) and significantly expands what the setup wizard exposes for Discord configuration.

Highlights

  • Slash command interaction timeouts fixed - compact, new, and status now defer the response immediately, so they no longer fail with the Discord 3-second interaction timeout under load
  • Accidental @everyone / @here mass-pings prevented - every user message, bot response, and stored conversation history entry has zero-width spaces inserted into @everyone / @here so a user cannot trick the bot into pinging the whole server
  • Typing indicator cooldown - typing is no longer requested on every message; the bot waits 5 seconds per channel between typing pings to avoid hitting Discord's 429 rate limit
  • Message queue removed - conversation messages are now handled directly per message instead of being drained FIFO, which cuts response latency
  • Setup wizard Discord section expanded - exposes max-tokens, compact-threshold, user / server rate limits, split threshold, and per-command toggles through the wizard

Features

  • DISCORD_ENABLED toggle - explicit kill-switch that prevents the bot from starting even if a token and guild ID are configured
  • Prefix commands - bot now supports a configurable text prefix (default !!) in parallel with slash commands for ask, compact, new, and status, each independently toggleable
  • Configurable live-conversation behavior - DISCORD_COMMAND_PREFIX and DISCORD_REQUIRE_MENTION let you tune whether the bot responds to every message vs. only @mentions or prefix invocations
  • Setup wizard exposes the full Discord config - token, multi-guild IDs, control channels (comma-separated), conversation category, specific conversation channel IDs, owner ID, owner-only mode, max-tokens per conversation, compact threshold, auto-compact toggle, per-user cooldown, server request limit / window, skip-files, message split threshold, and per-command on/off toggles
  • Multi-guild guidance in wizard - prompts now accept comma-separated guild IDs and explain how to find Developer Mode IDs

Fixes

  • Discord 3-second interaction timeout on /compact, /new, and /status - these commands now call interaction.response.defer(...) and reply via followup.send(...) instead of racing the 3-second window before doing real work
  • Assistant-message prefix drift in conversation history - the model no longer sees its own past responses prefixed with NIM: (which was reinforcing the prefix over many turns). Only user messages get the username prefix in stored history
  • Accidental mass-pings - user input, assistant output, and stored history all have @everyone / @here neutralized with zero-width-space insertion
  • Discord 429s on typing indicator - per-channel cooldown keeps typing pings to one per 5 seconds; the streaming pipeline is also no longer nested inside the typing() context, so a noisy channel can no longer block model responses on a typing budget
  • Response latency for queued messages - removed the FIFO _process_message_queue and processes each incoming message immediately in its own async task

Files Touched

  • discord_bot/bot.py - prefix command routing, message sanitization, typing cooldown, removed queue
  • discord_bot/cog.py - defer + followup for slow commands, conversation-history username prefix logic
  • discord_bot/conversation.py - only-prefix-user-messages behavior
  • config/settings.py - DISCORD_ENABLED, prefix toggles, DISCORD_REQUIRE_MENTION
  • setup_wizard.py - full Discord configuration section
  • .env.example - new variables documented

Quick Start

# Windows
nimbus.exe --init    # first time setup
nimbus.exe           # start the proxy
claude               # use Claude Code normally