add option to omit sending msg history with each openei completion

[k3it]

• reduces api token usage by letting the endpoint keep track of the conversation context
• fixes Prompt token consumption grows until /reset #34

[n3d1117]

Hi @k3it thanks for your contribution.

Indeed it looks like sending history also cosumes tokens.

However I don't see the point of the OMIT_MSG_HISTORY parameter you added. It's like resetting the chat on every message, losing all previous context. Would someone really use it? It may be cheaper but it's no longer a conversation.

Here's what the docs say:

Including the conversation history helps when user instructions refer to prior messages. [...] Because the models have no memory of past requests, all relevant information must be supplied via the conversation. If a conversation cannot fit within the model’s token limit, it will need to be shortened in some way.

On the other hand, I see the benefit in truncating the history. While I like your idea of removing elements from the top, I was wondering if we could use a separate ChatGPT instance to summarize and shorten it after its size reaches a cutoff value. Need to think about this. You can try and give it a shot if you have the time! I'll keep this PR open in the meantime

EDIT: Something like this here would be cool:

+ if len(self.sessions[chat_id]) > CUTOFF_SIZE:
+    summary  = /* somehow summarize the conversation in self.sessions[chat_id] */
+    self.reset_chat_history(chat_id)
+    self.__add_to_history(chat_id, role="assistant", content=summary)

self.__add_to_history(chat_id, role="user", content=query)

[k3it]

@n3d1117 I agree OMIT_MSG_HISTORY causes a severe short term amnesia in the bot and he becomes hard to deal with. Good idea about trying the summary approach. Also I wonder telegram group topics can work to keep multiple histories. this could mimic GPT web gui where we can have several parallel conversations with the bot.

[n3d1117]

Automatic summary generation added in 946f6a4. Closing this!