API Endpoint for Listing Loaded Running Models

[strikeoncmputrz]

It would be excellent to be able to interrogate the API to determine which models are running at any given time, rather than just seeing which checkpoints were pulled.

I use a variety of clients to interact with Ollama's API. I sometimes run models with a long keep_alive and assume others have similar use cases.

The only way I know of to identify a running model is through processes: ps aux | grep -- '--model' | grep -v grep | grep -Po '(?<=--model\s).*' | cut -d ' ' -f1. This will give you the full path to the model's blob. From there, you can compare that with the output of ollama show --modelfile (or the /api/show endpoint).

I checked the open issues and reddit and didn't see any similar RFIs or requests.

I wrote a bash script (depends on jq) that implements this as POC.

[pdevine]

I think this would be great, along with an ollama ps which shows which models are currently loaded in memory. It should include when the model's TTL is going to expire as well.

[strikeoncmputrz]

TTL is a great idea!

[unmotivatedgene]

Yes please add this especially with the new concurrency options I want to know what models are sticking around and taking up all my VRAM.