Is there a way to keep 1 big context bubble?

[sb-43]

Hey, I am newbie to AI. I was thinking of trying building an AI character that would be locked in its context bubble. An AI that would get data by reacting to certain impulses, and build it's net around it. My first thought was to build on local AI. Though not Pew's first, my thought was more towards an simple model that I can run on my computer and try to program my way to the model I am searching for.

The main point for me is to have the ability to run it for a long period in the context and build on it, with it comes the issue keeping the context.

I then remembered, that pew released this, I would like some feedback from more experienced AI engineers, who might guide me towards the right direction ;)