Optillm Docker container returns 500 error when accessing /v1/models with custom base URL #194
Description
Description:
When running the Optillm Docker container with a custom base URL, the /v1/models endpoint returns a 500 error with the following traceback:
TypeError: The view function did not return a valid response. The return type must be a string, dict, list, tuple with headers or status, Response instance, or WSGI callable, but it was a SyncPage[Model].
This error occurs when attempting to retrieve models via the /v1/models endpoint after setting a custom base URL.
Steps to Reproduce:
- Run the Optillm Docker container with the following command:
docker run -e OPENAI_API_KEY=<API_KEY> -p 8000:8000 ghcr.io/codelion/optillm:latest --base_url http://<SERVER_DNS_NAME>:<PORT>/v1
- Send a GET request to the
/v1/modelsendpoint:curl -X GET http://localhost:8000/v1/models \ -H "Authorization: Bearer <API_KEY>" \ -H "Content--Type: application/json" - Observe the 500 error and the TypeError in the logs.
Expected Behavior:
The /v1/models endpoint should return a list of available models
Actual Behavior:
The endpoint returns a 500 error due to a TypeError, indicating the Flask app is returning an invalid response type (SyncPage[Model]) instead of a proper JSON response.
Additional Context:
- Optillm version:
latest(Docker imageghcr.io/codelion/optillm:0.1.14) - The backend is an Ollama server, which is fully operational.
- Optillm appears to function correctly for standard chat completion tasks.