Skip to content

v1.4.2 — Nemotron-Omni multimodal across all 4 API surfaces

Choose a tag to compare

@jjang-ai jjang-ai released this 01 May 00:47
· 3662 commits to main since this release

Highlights

  • /v1/responses + /v1/messages now route Nemotron-3-Nano-Omni image / audio / video through the OmniMultimodalDispatcher (previously only /v1/chat/completions had the hook). Live test 2026-04-30 verified all 6 surfaces dispatch correctly to RADIO + Parakeet encoders. T6 input_image previously returned "Could you clarify…" — now correctly identifies image content.
  • Laguna (poolside) — full integration: 33B/3B agentic-coding MoE with hybrid SWA+full attention, 256 routed experts top-8 + 1 shared, dual-RoPE. New loaders/load_laguna.py routes through jang_tools.laguna.runtime.load; silver-tier registry row.
  • Mistral-Medium-3.5-128B (ministral3 inner) — dense GQA 96/8 + 256K YaRN + PIXTRAL vision. New loaders/load_mistral3.py; silver-tier registry row.
  • is_mllm_model Omni branch — bundles return False so standard load goes through the LLM path (mlx_lm has nemotron_h); dispatcher loads encoders + LM at request time. Unblocks vmlx-serve <omni-bundle> boots that previously crashed in mlx_vlm.utils.get_model_and_args.
  • _responses_input_to_messages — extended _MEDIA_TYPES to include OpenAI Responses canonical input_image / input_video / input_audio / input_text, with normalization to chat-completions envelopes.

Live verification matrix (Nemotron-3-Nano-Omni-30B-A3B-JANGTQ4)

Surface Modality Result
/v1/chat/completions text ✓ 0.5s
/v1/chat/completions image_url (RADIO) ✓ identifies "JANG" + black squircle
/v1/chat/completions input_audio (Parakeet) ✓ "male voice"
/v1/messages Anthropic image ✓ describes "JANG_Q"
/api/chat Ollama images:[base64]
/v1/responses input_image ✓ FIXED (was empty)

Install

pip install vmlx==1.4.2

(PyPI upload pending fresh token; install from this release wheel:
pip install https://github.com/jjang-ai/vmlx/releases/download/v1.4.2/vmlx-1.4.2-py3-none-any.whl)