VLLM plugin for nanchat 2026 #92d63d4

[inaciose]

Hello

First, i manage to get an working script to convert the new nanochat model to tensorsafe format.
After that, was a working vllm plugin to use the converted model inside a vllm openai compatible api.

fast install vllm transformers plugin (just once, inside the vllm enve)

uv pip install git+https://github.com/inaciose/vllm-nanochat2

fast run

uv run vllm serve inaciose/nanochat-d20-v1-sft
--trust-remote-code
--served-model-name nanochat-d20
--max-model-len 2048
--gpu-memory-utilization 0.85

Check it out. If you have security concerns check it using some AI.

https://deepwiki.com/inaciose/vllm-nanochat2

I wold love to run Nanochat2ForCausalLM models directely from vllm, without this plugin, does anyone knows how to add this to the transformer lib?