Repository navigation
NIMbus v2.0.11 - Inline Commands, Thinking Styles, Discord & MCP Enhancements
v2.0.11 - Date: 2026-07-17
Added
Inline Commands & Reasoning Control
<nimhelp>Inline Command - Send exactly<nimhelp>for a formatted list of all inline commands. Works in both streaming/buffered endpoints; not gated bySWAPPER_ENABLED.<nimeffort:level>Tag Handler - Override reasoning effort per-message:low,medium,high,xhigh,max,ultracode. Stores per session (x-claude-code-session-id), returns mock response (no NVIDIA API call). Whitespace handling matches<nimrpm:reset>.<nimeffort>/<nimeffort:status>- Show current session's stored effort level, mapped effort, and custom budget.- Custom Reasoning Budget -
<nimeffort:NNN>accepts-1to1,000,000(-1= unlimited). Newapi/effort_store.pytracks per-session effort + budget separately. Priority: exact tag > request > session > config. -1Budget = Unlimited - Validator accepts-1, rejects< -1. Passed through to NVIDIA API per spec (disables reasoning token limit).- Ultracode Budget -
reasoning_config.jsonsetsultracodebudget =-1(unlimited) for all Nemotron models.
Discord Bot Enhancements
- Web Search Persistence - Tool results saved in conversation history (
tool_resultsfield). Newget_history_with_tool_results()injects prior results as context. Config:DISCORD_WEB_SEARCH_MAX_RESULT_SIZE(5000),DISCORD_WEB_SEARCH_INCLUDE_IN_HISTORY(true). - Playwright Browser Fetch - When
DISCORD_BROWSER_HEADLESS=false, forces browser for ALL fetches (30s timeout vs 10s HTTP). Reuses DuckDuckGo browser/context (cookies, stealth). - Embedded JSON Tool Call Parsing - Detects
{"tool": "search", "query": "..."}in text streams (Nemotron 3 Ultra) → converts totool_useevents. - Empty Reply Fix - Final fallback retries 3× when model returns
tool_useinstead of text; adds "provide text answer only" prompt each retry; rejectstool_usein final response. - Smart Message Splitting -
_split_at_word_boundaryprefers newlines then spaces (preserves URLs); usesDISCORD_SPLIT_THRESHOLD(default 1900). - Web Search Resilience - "No results found" = valid result (not failure), reducing false tool removal. Buffered retries 3→5. Fallback synthesis when retries fail but tool results exist.
MCP Server
- Browser Support for
fetch_page- NewMCP_BROWSER_HEADLESS(defaulttrue= HTTP only).use_browserparam forces Playwright when headless=true. Reuses DuckDuckGo browser instance. Logs fetch mode.
Setup Wizard & Configuration
- Extended Wizard Coverage - New prompts: request queue priorities (Discord HIGH / API NORMAL), Fable model override (
FABLE_OVERRIDE), browser headless modes (MCP/Discord), web search debug logging (WEB_SEARCH_DEBUG), Discord web search settings. - Model-Specific Thinking Styles - New
thinking_styleinreasoning_config.json:- Nemotron:
chat_template_kwargs.enable_thinking+reasoning_budget - DeepSeek:
chat_template_kwargs.enable_thinking(boolean) - Minimax:
chat_template_kwargs.thinking_mode=enabled/adaptive/disabled - Default: effort flags +
reasoning_budget(backward compat) - Models NOT in config (or
supports_thinking: false) send NO thinking params - NVIDIA defaults apply.
- Nemotron:
nim:Prefix Model Lookup -nim:minimax-m3bypasses tier mapping, resolves via NVIDIA catalog.SHOW_NIM_REPLYToggle - Echo raw NVIDIA reply live to console (timestamped;THINKING= reasoning_content,REPLY= generated text). Both stream/buffer modes.- Discord Web Search Result Size -
DISCORD_WEB_SEARCH_MAX_RESULT_SIZE(5000),DISCORD_WEB_SEARCH_INCLUDE_IN_HISTORY(true).
Nemotron Thinking Support
reasoning_budget- Max thinking tokens (model caps, max 32768)chat_template_kwargs-enable_thinking+low_effort/medium_effort/high_effortflags- Effort→Budget: max/high/xhigh → max, medium → half, low → 2048
- Effort→Flags: xhigh/high/max → top, medium → medium, low → low
- Verified: Nemotron 3 Ultra (500B) & Nemotron 3 Super (120B) return reasoning content
Per-Session Reasoning Effort Tracking
- Captures
x-claude-code-session-id; in-memory per-session effort cache - Caches explicit request efforts; helper to clear session effort
- Added "ultracode" level + Nemotron mappings/budgets
Rate Limiter Auto-Restore
- New
NIM_RPM_RESET(default300s,0= disabled). After last 429, if no new 429 for duration, auto-restores initial RPM + clears hold delay. Complements manual<nimrpm:reset>.
THINKING Log Line
- New
INFOlog:THINKING: <model> using style=<style> effort=<raw> -> <mapped> [budget=<N>]. Logs when thinking disabled or model not in config.
Changed
- Reasoning Config Consolidation - Single
reasoning_config.jsonreplaces 3 files (reasoning_budgets.yaml,reasoning_effort_presets.json,user_reasoning_effort_mapping.json). Includesthinking_style,effort_mapping,budget_per_effortwith glob patterns. Legacy files deleted. - Model Resolution for Config Lookup - Uses resolved NIM model ID (from
get_model_for_claude) for reasoning config lookup. Fixesnim:minimax-m3→minimaxai/minimax-m3. - Rate Limiter Initialization -
GlobalRateLimiterinitialized early in FastAPI lifespan with.envsettings before sidecars trigger singleton. - Removed
NIM_ENABLE_THINKING- Redundant flag removed; thinking now solely controlled byNIM_THINKING. Simplifiesbuild_request_body. - Inline Command Detection (Multi-Block) -
extract_last_text_content()checks only last content block (user's command), avoiding false positives from system prompts/context. All 5 handlers updated. Buffered endpoint mock responses use.model_dump(). - Buffered Endpoint Inline Commands -
<nimhelp>and<nimrpm:reset>now work in buffered endpoint, positioned BEFOREwait_if_blocked()to bypass rate limiting during backoff. - SSE Dispatch Consolidation - 3 copy-pasted SSE blocks → single
_sse_response()helper. All 4 command dispatches now one-liners. Net -30 lines. - Default Model -
deepseek-ai/deepseek-v4-flash→nvidia/nemotron-3-super-120b-a12bin.env.exampleand setup wizard.
Fixed
- Rate Limiter Defaults - Was using hardcoded defaults (
drop=10,min=20) instead of.envvalues (NIM_RPM_DROP=2,NIM_RPM_MIN=2). Now reads from settings at startup. - Minimax Effort Mapping - Efforts now map directly to
thinking_modevalues (disabled/adaptive/enabled) instead of budget numbers. - Streaming Idle Timeout - Disabled idle read timeout (set to
None) to prevent timeout errors on sporadic chunk arrival. - Inline Command False Positives - Detection now checks only last content block, enabling inline commands with Claude Code's concatenated format (system reminder + user command blocks).
- Tiktoken Special Token Crash - Added
_safe_encode()helper inproviders/sse_builder.pythat catchesValueErrorfrom disallowed special tokens (e.g.,<|endoftext|>from Nemotron models) and retries withdisallowed_special=()to encode them as regular text. Prevents crash inestimate_output_tokens()during streaming.
Files Touched (Key Changes)
api/routes.py- Inline command handlers,_sse_response(), buffered commands, multi-block detectionapi/swapper/parser.py-<nimeffort:level>,<nimhelp>parsers;<nimrpm:reset>exact-matchapi/effort_store.py- New: Per-session effort + custom budget trackingapi/dependencies.py-GlobalRateLimiter.get_instance()with settings fallback;SHOW_NIM_REPLYapi/app.py- EarlyGlobalRateLimiterinit in lifespanapi/middleware.py- Session ID captureapi/models/anthropic.py- Session ID fieldproviders/request.py- Thinking param rewrite (thinking_style-based); model resolution via NIM ID;-1budget pass-through; THINKING log; removed session cacheproviders/provider.py-SHOW_NIM_REPLYlive logging; Nemotron thinking errors; streaming timeout; Discord log cleanupproviders/base.py- Adaptive rate limit fields;SHOW_NIM_REPLYin configproviders/rate_limit.py-rpm_reset,_last_429_time, auto-restore;_rate_limitproperty (test compat)providers/sse_builder.py-_safe_encode()helper for special tokens;estimate_output_tokens()uses safe encodingproviders/text.py-extract_last_text_content()for multi-block detectionconfig/settings.py-nim_rpm_reset,SHOW_NIM_REPLY,ReasoningConfig+thinking_style, glob matching, Discord web search, wizard additionsconfig/nim.py--1budget validator; reasoning effort validationreasoning_config.json- New: Consolidated config withthinking_stylefor all models; Minimax→thinking_mode; ultracode=-1 for Nemotron; glob patternsdiscord_bot/bot.py- Web search persistence, Playwright fetch, JSON tool parsing, empty reply fix, smart splitting, resiliencediscord_bot/conversation.py-tool_resultsfield,get_history_with_tool_results()discord_bot/persistence.py- Tool results serializationdiscord_bot/tools/web_search.py- Fetch page schema, result size/history configmcp_server.py- Browser support forfetch_pagesetup_wizard.py- Extended config sections.env.example- All new env var docsconfig/logging_config.py- Timestamped log files per sessionproviders/error_mapping.py- 403 handling with NVIDIA key URLserver.py- Logging init at entry point- Deleted:
config/reasoning_budgets.yaml,reasoning_effort_presets.json,user_reasoning_effort_mapping.json
Full Changelog: See CHANGELOGS.md
Assets: Binary builds will be attached on publish.