Open
Description
I'm using the following script to test all available models at gh models list
:
~> for model_name in (gh models list | awk '/\// { print $1 }'); echo "---> testing against $model_name..." ; gh models run $model_name 'tell me a dad joke' ; echo -e "<--- done.\n" ; end
Here's the output for the failed ones (they're redacted for brevity):
{"error":{"code":"unknown_model","message":"Unknown model: deepseek/deepseek-v3","details":"Unknown model: deepseek/deepseek-v3"}}
{"error":{"code":"unknown_model","message":"Unknown model: mistral-ai/mistral-large-2407","details":"Unknown model: mistral-ai/mistral-large-2407"}}
{"error":{"code":"unknown_model","message":"Unknown model: mistral-ai/mistral-small","details":"Unknown model: mistral-ai/mistral-small"}}
---> testing against openai/o1...
Error: unexpected response from the server: 403 Forbidden
---> testing against openai/o1-mini...
Error: unexpected response from the server: 403 Forbidden
---> testing against openai/o1-preview...
Error: unexpected response from the server: 403 Forbidden
---> testing against openai/o3...
Error: unexpected response from the server: 403 Forbidden
---> testing against openai/o3-mini...
Error: unexpected response from the server: 403 Forbidden
---> testing against openai/o4-mini...
Error: bad request
{"error":{"code":"unavailable_model","message":"Unavailable model: o4-mini","details":"Unavailable model: o4-mini"}}
Metadata
Metadata
Assignees
Labels
No labels