Skip to content

Remove config options that are discoverable through the API #140

@lpiwowar

Description

@lpiwowar

There are some config options that are discoverable through the /models API. Let's remove them and discover them via the API call.

We don't have to keep it in the env as this would be different for diff models and what the model has been deployed with. we can get using api call

ramishra@fedora:~/work/LLM/chatbot$ curl http://xxx:8000/v1/models \
  -H "Authorization: Bearer xxxx"
{"object":"list","data":[{"id":"deepseek-ai/DeepSeek-R1-Distill-Llama-8B","object":"model","created":1746600067,"owned_by":"vllm","root":"deepseek-ai/DeepSeek-R1-Distill-Llama-8B","parent":null,**"max_model_len":32000**,"permission":[{"id":"modelperm-78f16fdc0769472c8ee8dc4b96aae500","object":"model_permission","created":1746600067,"allow_create_engine":false,"allow_sampling":true,"allow_logprobs":true,"allow_search_indices":false,"allow_view":true,"allow_fine_tuning":false,"orga

Originally posted by @rabi in #134 (comment)

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions