-
Notifications
You must be signed in to change notification settings - Fork 5
Closed
Labels
enhancementNew feature or requestNew feature or request
Description
There are some config options that are discoverable through the /models API. Let's remove them and discover them via the API call.
We don't have to keep it in the env as this would be different for diff models and what the model has been deployed with. we can get using api call
ramishra@fedora:~/work/LLM/chatbot$ curl http://xxx:8000/v1/models \
-H "Authorization: Bearer xxxx"
{"object":"list","data":[{"id":"deepseek-ai/DeepSeek-R1-Distill-Llama-8B","object":"model","created":1746600067,"owned_by":"vllm","root":"deepseek-ai/DeepSeek-R1-Distill-Llama-8B","parent":null,**"max_model_len":32000**,"permission":[{"id":"modelperm-78f16fdc0769472c8ee8dc4b96aae500","object":"model_permission","created":1746600067,"allow_create_engine":false,"allow_sampling":true,"allow_logprobs":true,"allow_search_indices":false,"allow_view":true,"allow_fine_tuning":false,"orga
Originally posted by @rabi in #134 (comment)
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request