Propagate the model loading from transformers serve to chat

[LysandreJik]

Provides a nicer feedback when transformers chat loads a model, instead of hanging

Screen.Recording.2026-03-17.at.00.00.52.mov

Adds a POST /load_model endpoint to transformers serve that streams model loading progress via Server-Sent Events (SSE). Previously, transformers chat would hang with no feedback while downloading and loading a model — now users see real-time progress for each phase.

transformers chat calls this endpoint on startup and renders the progress with Rich progress bars.

The /load_model SSE protocol

The endpoint uses a simple, implementation-agnostic SSE protocol designed to be easy to consume from any language or framework. No Python or library-specific details leak into the wire format.

Request:

POST /load_model
Content-Type: application/json
{"model": "HuggingFaceTB/SmolLM2-135M-Instruct"}

Response: text/event-stream — each frame is data: <json>\n\n

Every event has status and model. Three possible statuses:

Status	Terminal?	Extra fields	Description
loading	No	stage, optionally progress	Something is being loaded
ready	Yes	cached: bool	Model is available
error	Yes	message: str	Loading failed

download is skipped when files are already cached locally. Progress events include {"current": int, "total": int}.

Example stream (fresh load with download):

data: {"status": "loading", "model": "HuggingFaceTB/SmolLM2-135M-Instruct@main", "stage": "processor"}
data: {"status": "loading", "model": "HuggingFaceTB/SmolLM2-135M-Instruct@main", "stage": "config"}
data: {"status": "loading", "model": "HuggingFaceTB/SmolLM2-135M-Instruct@main", "stage": "download", "progress": {"current": 67020052, "total": 269060552}}
data: {"status": "loading", "model": "HuggingFaceTB/SmolLM2-135M-Instruct@main", "stage": "download", "progress": {"current": 269060552, "total": 269060552}}
data: {"status": "loading", "model": "HuggingFaceTB/SmolLM2-135M-Instruct@main", "stage": "weights", "progress": {"current": 136, "total": 272}}
data: {"status": "loading", "model": "HuggingFaceTB/SmolLM2-135M-Instruct@main", "stage": "weights", "progress": {"current": 272, "total": 272}}
data: {"status": "ready", "model": "HuggingFaceTB/SmolLM2-135M-Instruct@main", "cached": false}

When the model is already in memory, a single event is returned:

data: {"status": "ready", "model": "HuggingFaceTB/SmolLM2-135M-Instruct@main", "cached": true}

Integration in chat.py

The RichInterface.print_model_load() method in chat.py consumes this SSE stream and renders it with Rich progress bars. It's intentionally kept simple as a reference implementation — the client just reads events sequentially and reacts to each status/stage:

• processor / config → status text
• download → byte-progress bar with speed and ETA
• weights → item-progress bar
• ready → done message
• error → raise

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[SunMarc]

Thanks ! Just a few nits

[SunMarc]

Thanks a lot, just a few nits and we can merge !

[SunMarc]

should we also propagate to load_audio_model_and_processor ?

[LysandreJik]

I would be down for us to do that, but maybe in a follow-up? It's not a hugely used path for now and no client would benefit from it at this time

[SunMarc]

can you add some docstring for progress_callback ?

[SunMarc]

This is cropping the description sometimes when it is too long. Maybe we shouldn't put the model name in the description

[LysandreJik]

I put the model name in the desc only if we have the space to show it, otherwise I remove it

[SunMarc]

Thanks for this clean PR !

[SunMarc]

nice, we should add more concurrent test in general for our endpoints to see if they behaves correctly. I will probably do that in the refactor