Feat: Add ChatGPT import functionality

[bhulston]

For this issue: #337

Enable imports of ChatGPT histories (conversations.json), and ability to choose a new model upon import.

1. Added several functions to src/lib/utils/index.ts that are used at import before sending to createNewChat api
   a) Identify if import is a chatGPT one or Ollama-webui one (as they are very different)
   b) Parses and iterates through the chatGPT json to match the ollama-webui chat structure

2. Make sure new chats work with existing code
   a) Adjusted the "disabled" tag to let you choose what model to use for new chat
   b) Add one more check when message is sent that saves selectedModel when current saved model is different
   - With imported models, messages already exist but no model exists! so need to update. or if you import chats using models that you don't have (i.e. 'gpt-3.5-turbo')

** Note **: When we import from ChatGPT, we don't default to a model so users aren't forced to use chatgpt
Does not import context (i.e. pdf RAG) info from GPT chats.

[bhulston]

@tjbck I did a lot of testing in dev environment so there shouldn't be any issues. Original functionality of app stays the same and changes only affect imported chats really 👍

[tjbck]

Amazing work! LGTM, Thanks!

[bhulston]

Cool! I saw your refactoring, let me know if you want me to do that for you next time I don't mind.

Also saw the edge case handling and preferred formatting so I will keep those in mind next time too 👍

[justinh-rahb]

So, due to migrating my ChatGPT account from Plus to Team, I actually lost the ability to continue my prior chats in my Personal workspace with GPT-4. Now I can import those chats and continue them with API access. 👌

Edit: My Plus subscription finally expired and oddly enough I can still use GPT-4 from my "personal" workspace. Odd, but I ain't gonna question them on that LOL

[tjbck]

@bhulston yeah no worries, thanks for the hard work!

[madman2012]

Thanks for the hard work on this guys. I can import the Ollama chats no problem, and chat contents come. If I import the OpenAI json I see the title of the chats but no contents. Just FYI.

[bhulston]

@madman2012
Can you inspect on your browser and send the error you get? I'm gna try to reproduce the problem as well, might be just a small bug.

edit: on my end it works well so lmk:)

[justinh-rahb]

I've got the same issue. I am quite certain the content is present just not being read properly, as if I export the chats again it's almost the same size as the input OpenAI export. For obvious reasons I cannot provide full dumps. (Lol.. OpenAI chat history is the new browser history, gonna need to tell my bestie they better delete that too when I shuffle off this mortal coil 😂 )

There's no error in the container log so I think this is confined to the frontend code.

[bhulston]

@justinh-rahb
Strange, the chat imports I'm using both work. It's possible there's some edge cases that are causing this.
I'll take a look and try to reproduce it, going to ask some friends for their exports lol

[justinh-rahb]

@justinh-rahb Strange, the chat imports I'm using both work. It's possible there's some edge cases that are causing this. I'll take a look and try to reproduce it

I've been a daily user of ChatGPT since day 1, my export file is quite large, and I saw some issues with weird characters, and some chats with null titles. It's probable this is simply unclean data and unfortunately the only way I could help you to reproduce that is by sharing my export, which I cannot do.

[bhulston]

@justinh-rahb Strange, the chat imports I'm using both work. It's possible there's some edge cases that are causing this. I'll take a look and try to reproduce it

I've been a daily user of ChatGPT since day 1, my export file is quite large, and I saw some issues with weird characters, and some chats with null titles. It's probable this is simply unclean data and unfortunately the only way I could help you to reproduce that is by sharing my export, which I cannot do.

All good, my friend just sent me his (also since day1) and it reproduced the same error 👍 I should be able to fix it pretty quick. My guess is we just need to add some try catch blocks and skip the ones with some errors, we'll see

[bhulston]

@justinh-rahb @madman2012
Found the issue, looks like when chatGPT uses the Python tool(code interpreter), they change the output structure.
I'm going to try and edit the code so that it works with those, but in the worst case we could skip those outputs.

[bhulston]

Added one more PR: #535

I tested this on chats from 3 separate people and tried to cover 99% of edge cases.

Basically just adding one additional step of validation for inputs. Seems there are external tools gpt might use or corrupted json objects from multiple interruptions in the chat.

Not happy about the inefficiency of it but imports should really only happen one time... so this should fix any issues you all had.

[justinh-rahb]

You're a superstar @bhulston ! I'll build your branch and give this a try if I can get a few minutes to spare before the end of the day.

[bhulston]

Yah lmk!Trying my best so hopefully this one works, seems like it should work well

[justinh-rahb]

Working great @bhulston! Imports a few of my chats without titles but that's a minor issue I'm more than happy to live with.

[madman2012]

Working here too! Thank you, you are a superstar!