[ChatGPT] GPT-4 models do not appear to actually be using GPT-4

[Aias]

Extension

https://www.raycast.com/abielzulio/chatgpt

Description

When setting the model to a version of GPT-4, then asking the AI what version it is, the AI claims to be GPT-3.

Compare this to Raycast's built-in GPT-4 chat, which readily offers up the correct model version.

Steps To Reproduce

1. Enter your API key in settings.
2. Select GPT-4 within a model.
3. Ask the following question with that model select: "What GPT model are you?"
4. Observe response.

Current Behaviour

Will output something along the lines of "I'm based on GPT-3."

Expected Behaviour

To output something along the lines of "I'm based on GPT-4."

[raycastbot]

Thank you for opening this issue!

🔔 @abielzulio @erodactyl @pernielsentikaer @shichongrui @timolins @pomdtr @CruelMoney @abielzulio @abielzulio @k8scat @Nauxscript you might want to have a look.

💡 Author and Contributors commands

The author and contributors of chatgpt can trigger bot actions by commenting:

• @raycastbot close this issue Closes the issue.
• @raycastbot rename this issue to "Awesome new title" Renames the issue.
• @raycastbot reopen this issue Reopen the issue.

[danielo515]

Oh, so the way to define what model to use is to edit the model? It is not very well explained in the instructions

[raycastbot]

This issue has been automatically marked as stale because it did not have any recent activity.

It will be closed if no further activity occurs in the next 10 days to keep our backlog clean 😊

[Aias]

Adding a comment to this so it's not automatically closed. It's highly misleading if people who are paying for GPT-4, and expecting GPT-4, are actually getting much less sophisticated responses from GPT-3.5.

[pernielsentikaer]

Bear in mind that this is a third party extension which uses your own key from OpenAI. Also keep in mind that you can not trust that AI gives you the right answer

[Aias]

@pernielsentikaer thanks, it looks like my assumptions were wrong here. I had thought that after the training process there would be some kind of metadata inserted into the model which allowed GPT to know specifics about its version, architecture, etc. Apparently not, but I'm not the only one with this question. See: https://community.openai.com/t/api-model-set-to-gpt-4-but-seems-to-respond-as-gpt-3/104757/6

Looks like, since GPT-4 didn't exist when GPT-4 was trained, it won't tell you it's GPT-4 unless you tell it to within the system instructions. This must be what's going on with the web version of ChatGPT, and/or the Raycast AI Chat, since both of those give you back the correct GPT version, probably to avoid this exact problem.

I'll close this issue, learned something new today.