OpenAI API - response data structure for the models endpoint is wrong

[byjlw]

🐛 Describe the bug

Output is wrong format. Please refer to the spec https://platform.openai.com/docs/api-reference/models/list
Currently returns this

{
  "data": [
    {
      "id": "string",
      "created": "integer",
      "owner": "string",
      "object": "string"
    }
  ],
  "object": "string"
}

Required format

{
  "object": "string",
  "data": [
    {
      "id": "string",
      "object": "string",
      "created": "integer",
      "owned_by": "string"
    }
  ]
}

Differences:
Owner Key Name:
In current, the key for the owner of the model is named "owner".
In required, the key for the owner of the model is named "owned_by".
Order of Keys:
In current, the "object" key at the root level appears after the "data" key.
In required, the "object" key at the root level appears before the "data" key.
Data Array Content:
current contains models with specific versions and instructions in their IDs (e.g., "meta-llama/Llama-2-7b-chat-hf").
required contains models with more generic IDs and versions (e.g., "llava:34b").

Versions

does not apply

[vmpuri]

Owner key name resolved in #1020

re: Order of Keys: Does the order of keys matter in JSON? IIUC this is just an unordered map. The order is currently just dictated by the order of arguments in Python dataclasses since we set the default "object" value to "list" (can't have default arguments before non-default args)

re: Data Array content: The specification doesn't mention a format for the model description, so I chose the most specific one available. We also have multiple aliases available for the same model (in torchchat.py list), but these can lack the specificity we need (i.e. would you, as a user, expect "codellama" to refer to the 7b or 34b param model? In this case, it refers to 7b but nothing in the Model object could give that description)

Model                                   Aliases                                     Downloaded 
--------------------------------------- ------------------------------------------- -----------
meta-llama/llama-2-7b-hf                llama2-base, llama2-7b                                 
meta-llama/llama-2-7b-chat-hf           llama2, llama2-chat, llama2-7b-chat                    
meta-llama/llama-2-13b-chat-hf          llama2-13b-chat                                        
meta-llama/llama-2-70b-chat-hf          llama2-70b-chat                                        
meta-llama/meta-llama-3-8b              llama3-base                                            
meta-llama/meta-llama-3-8b-instruct     llama3, llama3-chat, llama3-instruct                   
meta-llama/meta-llama-3-70b-instruct    llama3-70b                                             
meta-llama/meta-llama-3.1-8b            llama3.1-base                                          
meta-llama/meta-llama-3.1-8b-instruct   llama3.1, llama3.1-chat, llama3.1-instruct             
meta-llama/meta-llama-3.1-70b-instruct  llama3.1-70b                                           
meta-llama/codellama-7b-python-hf       codellama, codellama-7b                                
meta-llama/codellama-34b-python-hf      codellama-34b                                          
mistralai/mistral-7b-v0.1               mistral-7b-v01-base                                    
mistralai/mistral-7b-instruct-v0.1      mistral-7b-v01-instruct                                
mistralai/mistral-7b-instruct-v0.2      mistral, mistral-7b, mistral-7b-instruct               
openlm-research/open_llama_7b           open-llama, open-llama-7b                              
stories15m                                                                          Yes        
stories42m                                                                                     
stories110m  

We could create a Model object in the response for every alias - for example, we would respond with entries both "llama2-base" and "llama2-7b" even though they refer to the same model

[byjlw]

RE: order of the keys: Order of the keys do not matter.
RE: Data Array content: That makes sense. As is, is fine then.

Owner key name resolved in #1020

re: Order of Keys: Does the order of keys matter in JSON?

re: Data Array content: The specification doesn't mention a format for the model description, so I chose the most specific one available. We also have multiple aliases available for the same model (in torchchat.py list), but these can lack the specificity we need (i.e. would you, as a user, expect "codellama" to refer to the 7b or 34b param model? In this case, it refers to 7b but nothing in the Model object could give that description)

Model                                   Aliases                                     Downloaded 
--------------------------------------- ------------------------------------------- -----------
meta-llama/llama-2-7b-hf                llama2-base, llama2-7b                                 
meta-llama/llama-2-7b-chat-hf           llama2, llama2-chat, llama2-7b-chat                    
meta-llama/llama-2-13b-chat-hf          llama2-13b-chat                                        
meta-llama/llama-2-70b-chat-hf          llama2-70b-chat                                        
meta-llama/meta-llama-3-8b              llama3-base                                            
meta-llama/meta-llama-3-8b-instruct     llama3, llama3-chat, llama3-instruct                   
meta-llama/meta-llama-3-70b-instruct    llama3-70b                                             
meta-llama/meta-llama-3.1-8b            llama3.1-base                                          
meta-llama/meta-llama-3.1-8b-instruct   llama3.1, llama3.1-chat, llama3.1-instruct             
meta-llama/meta-llama-3.1-70b-instruct  llama3.1-70b                                           
meta-llama/codellama-7b-python-hf       codellama, codellama-7b                                
meta-llama/codellama-34b-python-hf      codellama-34b                                          
mistralai/mistral-7b-v0.1               mistral-7b-v01-base                                    
mistralai/mistral-7b-instruct-v0.1      mistral-7b-v01-instruct                                
mistralai/mistral-7b-instruct-v0.2      mistral, mistral-7b, mistral-7b-instruct               
openlm-research/open_llama_7b           open-llama, open-llama-7b                              
stories15m                                                                          Yes        
stories42m                                                                                     
stories110m  

We could create a Model object in the response for every alias - for example, we would respond with entries both "llama2-base" and "llama2-7b" even though they refer to the same model