Regenerate respone

[arash-ashra]

Hii,
Thank you so much for this project.
Is it possible to regenerate the response using a command similar to the ChatGPT's interface here?

Cheers,
Arash

[arash-ashra]

As it is quite useful for my application, I'll give $100 to the person who implements this feature first ;)
What I have in mind is an emoji on the response in Discord that when clicked would regenerate and replace the previous response. @Zero6992 @aW3ikun @shinysocks @jacky-chay

[shinysocks]

@arashash, I implemented the feature on my fork and made a pull request #438. If you like the work that I did, please support me on Paypal or Ko-fi!

[arash-ashra]

Awesome! I'll check it out later today. Thank you 🙏

[arash-ashra]

I checked it out in Windows. I see two issues.
One is that it doesn't work with /draw.
The other one is that the counter. After clicking the emoji the counter goes from 1 to 2 then back to 1. I think it'd be cool to have it count how many it has generated so far similar to OpenAI interface.
Thank you for rising to the challenge, I'm sure many users will appreciate this feature as well.

P.S. I made the payment here.
Cheers,
Arash

[arash-ashra]

Ooh there is a big issue. It is not removing the previous output from the context memory. We can see in this example that each time it's generating it is incrementing, but it should not!

This is really important for the feature since allows us to sample the same thing multiple times.
Looking forward to the patch!

[shinysocks]

Alright,

• I didn't implement it for draw yet, that is my next task.
• I think i can get the number of reactions to keep increasing, i'll test this later
• The reason it's "memory" does not clear between messages is due to the way the bot interacts with the api. Unless you call the /reset command, chatgpt treats each new message as occuring in the same conversation. It wouldn't be hard to reset the conversation for each regeneration but it may not be the desired behavior for everyone... Maybe i'll make it an environment variable. Lemme know if I'm misunderstanding the problem at all.

[shinysocks]

out of curiosity, what are you building? is the source published yet? It seems really neat!

[arash-ashra]

I'm using it to tutor my students in Disocrd (so it helps as a TA). We can't reset the memory in a channel since the bot needs to know the context so we can refer to previous points. The problem I'm trying to solve is to remove the output from the context when the bot is giving a wrong or vague response. We just want to replace the last output. this is exactly how the chatGPT interface works, each time you get an output, you get the chance to replace it at that point (but not later when the next output is generated).
As this is the default in ChatGPT I think it makes sense to have it as default here as well. And you could prioritize this to the other issues since currently it's not solving the problem as it's still keeping the wrong outputs in the memory.

[arash-ashra]

another issue, the regenerate button is keep appearing within different lines of the same output. I think doing it with the slash command would have been more robust.

[shinysocks]

Hello,
I'm going to try and attach a regenerate button to only the latest interaction to solve this issue. Otherwise, I could also use a slash command to do the same thing but a button would look really nice.

[arash-ashra]

Right I agree, but until bugs fixed and become a general-purpose button like openAI interface, it may be better to have a /slash command that only users who know what they are doing be able to use it.

[shinysocks]

alright, sounds good

[shinysocks]

I moved the regeneration into a slash command, it does not work for images as of right now.

[shinysocks]

I wanted to get a reaction based counter on the message to show the number of regenerations but I cannot figure out how discord.Interaction works with reactions. But let me know if this is what you were looking for

[arash-ashra]

No worries about that, it's not really useful for my application anyway. The priority should be to remove the previous generation from context memory since currently it's not regenerating, but rather redoing given previous attempts. But I need a sampling function that can reattempt without memory of the previous attempts. Since when it keeps the previous attempts in memory, the new attempts look very similar so it quickly gets stuck in the same frame of mind, but having the ability to regenerate from scratch gives much more brainstorming power

[shinysocks]

I finally understand what you're saying! I'll take a look at the OpenAI API docs and figure how to do this correctly. Sorry for the confusion

[arash-ashra]

Thaanks. Sorry for the misunderstanding

[shinysocks]

I don't think what you want is possible due to limitations in OpenAI API: api reference

[arash-ashra]

Looks like possible if you use chatCompletion API which allows you to store and edit the entire conversation, basically it makes the API memoryless https://platform.openai.com/docs/guides/gpt/chat-completions-api

[arash-ashra]

also, it allows you to edit any part of the conversation in the past which would be a pretty cool feature to have as well

[shinysocks]

oh, awesome!

[arash-ashra]

Hey @shinysocks, any update or plan on this?

[shinysocks]

I haven't yet taken a look at rewriting the API requests. I'll check it out sometime this upcoming week hopefully

[arash-ashra]

Awesome. Looking forward to it

[shinysocks]

Where did you find information in the API to regenerate? I can't seem to find it.

[arash-ashra]

Here
Once you run the bot with this method (ChatCompletion), then regenerating is only a matter of replacing the last item in the messages array and calling again.

[shinysocks]

@Zero6992 Does this bot even use the official openai api?

[arash-ashra]

@shinysocks it's using this https://github.com/acheong08/ChatGPT, which inside is using the official openai api

[shinysocks]

Looking that this api's documentation, it doesn't look like they include a regeneration. If you want to utilize this specific feature you may need to open a pull request on that api.

[arash-ashra]

Yea looks like using that API was a mistake, as the project is dead. I found a better discord chatGPT, which also has the regenerate button. I'll give it a try: https://github.com/Kav-K/GPTDiscord.

[shinysocks]

Awesome, that one looks great.