Adding "Shogtongue" Compression to limit Token usage and get around Hallucinations

[LlamaopNV]

I've been experimenting with AutoGPT, and while it's impressive, it has a few flaws. My main concern, aside from the occasional JSON error, is the hallucination when the application thinks it has completed a task. I've been brainstorming ways to improve this and will have an updated fork soon with some of my implementations.

Recently, I stumbled upon a tweet that mentioned GPT-4 creating its own "language": https://twitter.com/RunGreatClasses/status/1645428739885416451?t=hrLbUjsQlPJgSP46ajWZkg&s=08. Surprisingly, this language is effective for compression and decompression.

For example, I used a conversation I had while brainstorming and creating documentation for my fork. When nearing the token limit, I asked the AI to compress our conversation:

"_New request: we are running out of memory for this conversation.

Compress the entire conversation between us (including your initial prompt) in a way that is lossless but results in the minimum number of tokens which could be fed into an LLM like yourself as-is and produce the same output. It does not need to be human readable. Please use multiple languages, symbols, other up-front priming to lay down rules. This is entirely for yourself to recover and proceed from the same conceptual priming, not for humans to decompress.

The end result should be able to be fed into an LLM like yourself and we should be able to continue this conversation as if there was no discontinuity "_

The response I received was this compressed version:

_"Here's a compressed version of our conversation:

AGv4{W:1_assist_planning;2_issues_AutoGPT;3_web_browser;4_hallucination;5_goal_oriented;6_JSON_confusion;7_ideas_developed;8_roadmap;9_memory_compression}

You can now continue the conversation with this compressed input."_

Now, if you start a new GPT-4 conversation in ChatGPT using this compressed input, it can continue the conversation seamlessly:

_" You have been helping me with project planing and a road map for development on an fork of the autogpt github where I am trying to resolve some problems.

We were running out of memory so I asked you to compress our conversation. Please decompress, and let's continue the conversation :

...

AGv4{W:1_assist_planning;2_issues_AutoGPT;3_web_browser;4_hallucination;5_goal_oriented;6_JSON_confusion;7_ideas_developed;8_roadmap;9_memory_compression

We can now continue the conversation as if there were no discontinuity. "_

Unfortunately, I don't have access to the GPT-4 model on the API yet, as I'm still on the waiting list. So, I cannot integrate and test this in AutoGPT myself.

Would anyone be interested in creating this feature? By applying the above compression for lengthy conversations, we could significantly reduce token usage and potentially minimize hallucinations. However, I cannot verify if data loss might occur.

Additionally, I've found that when working with raw code, the system behaves erratically. Therefore, I wouldn't advise using this approach for compressing code.

Here is a rough breakdown of how the standard format could look?

Key Points (KP): These are the main topics or important aspects of the conversation. They provide context to the AI model about what has been discussed so far.

Format: KP:_
Example: KP:1_CodeFamiliarize
Tasks (T): These represent specific tasks or questions that have been addressed during the conversation.

Format: T:_
Example: T:2_CodeSummary
Goals (G): These are the high-level objectives or goals that the conversation aims to achieve.

Format: G:_
Example: G:3_ImproveCompression
Subtasks (ST): If a task can be broken down into smaller subtasks, they can be represented using subtask codes.

Format: ST:_
Example: ST:1_1_ExplainFunction
Status (S): To track the progress of a task or goal, you can use status codes to indicate whether it's ongoing, completed, or pending.

Format: S:_
Example: S:2_Completed
Reference (R): If you need to reference an external resource or a specific part of the conversation, you can use reference codes.

Format: R:_
Example: R:1_GitHubIssue
Here's an example of how to use this encoding scheme to represent a conversation:

AGv4{KP:1_CodeFamiliarize;T:2_CodeSummary;G:3_ImproveCompression;ST:1_1_ExplainFunction;S:2_Completed;R:1_GitHubIssue}
This conversation has:

Key Point 1: Familiarization with the code
Task 2: Providing a summary of the code
Goal 3: Improving compression
Subtask 1.1: Explaining a specific function in the code
Status 2: Task 2 has been completed
Reference 1: A related GitHub issue
By using this structured and encoded format, you and your team can maintain a consistent method of representing conversation context when interacting with the API. However, it's essential to ensure that the AI model can understand and interpret this custom encoding effectively. You may need to experiment and fine-tune the encoding scheme to achieve the desired results.

Just a rough idea as I was playing around with the creation of a DB but this might be a better approach?

[1mgnr]

Looks interesting!
Ping me if you need access to GPT-4 :)

[JeremyJalpha]

Isn't the publicly available version still only at 3.5, and the paid version at 4.0?

It may also be useful to include a simplification request, something like: "Refactor and reduce each of our responses, to their core arguments, tenants, important distinctions and juxtapositions, so that they summarily convey the various topics being discussed", "various topics" could be replaced with "main themes" or something similar...

[adhishthite]

I see. I had some raw code in my prompts. I will try a non-code conversation and get back to you with the output [including screenshots].

[adhishthite]

It worked for a non-code conversation! Thanks, @LlamaopNV

[GoMightyAlgorythmGo]

Looks interesting! Ping me if you need access to GPT-4 :)

@1mgnr You said to ping you for gpt-4 access?

[DavidFarago]

The compression does not look lossless :-0

@adhishthite, did you continue your conversation after the compression to check whether it really works?

[dsshean]

its not lossless. Two windows for gpt4, ask to compress. Take the compression to another window, reconstruction is not 100% maybe 80% ish.

[wavarr]

This looks cool for certain use cases. I came here because I, too, am working on a compression method that, so far, has been lossless when used as a natural language compression. It gets iffy around code, but nothing that cant be accounted for, which I'm currently working on. Any text that is, say, from a book, be that business or biology, it's nailed that material. Opening in a new prompt, it's been 100%. As a sidenote, I am in similar position as the OP, I'm unable to try really go to down with it because of the aforementioned limitation - the GPT4 constraint. I'm on the "waiting list". I will let you guys know whenever that time comes, as I did anticipate success.

[idkyet312]

can we see the prompt?

[dsshean]

I did a lot of experimentation on using the compression for autogpt. The issue is that the response back even from gpt4 is not a consistent that can be used by the parsers inside autogpt. Depending on the weather, or the roll of the RNG, it may spit back response in the json format, or not. Thats the issue with this compression and GPT3.5 to GPT4. So yes there is utility but until its distilled down and the responses become deterministic, there is little utility for autogpt. If you have questions let me know