Custom code interpreter

[alita-moore]

Is your feature request related to a problem? Please describe.

I want to execute code but that's very insecure. Running this on a server is dangerous.

Describe the solution you'd like

So instead it seems optimal to run that code somewhere else. Perhaps on a remote environment like https://github.com/e2b-dev/e2b provides. It would be nice to be able to provide a custom code interpreter so that it runs the code on a remote server instead of locally.

Describe alternatives you've considered

I have considered running this on a serverless function, which may forgo some of the complexities of running the code on a remote sandbox.

Additional context

No response

[alita-moore]

For reference, I think that this is a superior solution to what is proposed in #654 (running this on a serverless function). That's because it seems to make more sense to me that this project be focused on prompting the llm and parsing the response (and doing so for lots of llms) rather than the execution itself. Perhaps that's better offloaded to a different project?

[KillianLucas]

I see where you're coming from, but in the case of OpenAI, that's as simple as writing a function saying it can execute code. For non-function calling LLMs the implementation isn't much more complex, no more than 20 lines of code.To me the core functionality of Open Interpreter is running the code (securely)— figuring out how to run multiple languages and get great real-time information out of them (for example, which line is being run, and it's respective output) and yes, pipe back in the correct information to the LLM.I think the functionality you're looking for should come from the apps/interfaces built on top of this tech. This package should/does come with an optional terminal interface as a frontend for the core OI tech, but there are many other interfaces in development. Would you be interested in something like `chat.openinterpreter.com` which is hosted, looks like ChatGPT?On Oct 18, 2023, at 11:40 AM, Alita Moorewas ***@***.***> wrote: For reference, I think that this is a superior solution to what is proposed in #654 (running this on a serverless function). That's because it seems to make more sense to me that this project be focused on prompting the llm and parsing the response (and doing so for lots of llms) rather than the execution itself. Perhaps that's better offloaded to a different project? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[alita-moore]

I see, well I'm mostly interested in building a product similar to openai but for my own use case. The challenge being that running the code securely and managing the sessions can be quite tedious and dangerous. So ideally it would be something that can be handled for me.

If you were to build the chat.openinterpreter.com example that you said, how would you handle hosting the code execution? Would you run them in docker containers on a single server or run them in serverless functions (or something similar) or anything else?

[alita-moore]

by the way, do you intend on supporting a simpler input output? i.e. input a list of messages and output the answer rather than having to do interpreter.messages = messages, interpreter.chat(messages[-1]), interpreter.reset()?

[KillianLucas]

We're working on making it very easy to pass in a list of allowed functions and sandboxing the Python interpreter, so you can be sure it doesn't execute any unsafe code. We're also trying to find the right way to handle containerization/virtualization, but hosting these containers would be a different beast. As for that—hosting this and making a service out of it— E2B seems like the way to go. They handle isolated environments for arbitrary code execution, spinning up one for each user, etc. Re: Simpler input/output— great point. How about we make `interpreter.chat(messages)` work? It would just complete the message, same as OpenAI. This means you can manually handle storing/using messages more easily. - `interpreter.chat()` opens up the interactive terminal interface. - `interpreter.chat(message)` would take a string, and be stateful (saves everything to interpreter.messages). - `interpreter.chat(messages)` would take a list of dictionaries. Overwrites its own `interpreter.messages`, then starts generating. Is it okay that we use our own way of storing messages? I thought openai's "content" was confusing, and "function > function call > arguments... etc" doubly so. (We just have dictionaries with "role" (same as openai) but no "content"— instead it's "message", "code", or "output"). You've probably noticed this. We could switch to using openai, or perhaps accept openai message lists then convert them..? Any reason you might want to store/use fully openai compatible message lists?

…

On Wed, Oct 18, 2023 at 12:48 PM Alita Moore ***@***.***> wrote: by the way, do you intend on supporting a simpler input output? i.e. input a list of messages and output the answer rather than having to do interpreter.messages = messages, interpreter.chat(messages[-1]), interpreter.reset()? — Reply to this email directly, view it on GitHub <#655 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APHXIQZARLQCCKSQGFJOB2TYAAW7BAVCNFSM6AAAAAA6F64QPOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRZGIYTGNBRGE> . You are receiving this because you commented.Message ID: ***@***.***>

[alita-moore]

yeah that sounds great.

wrt the way you store messages it's hard to say for sure. The way we're doing it is by utilizing an even convoluted approach. Basically, all chats are made up of nodes, each node links only to its children, a chat is defined by a root node. Then, each node contains a list of content and that content can be of a certain type (e.g. code, message, output, etc.).

Generally, I try to maximize for flexibility. I'm by no means an authority on that topic, though. Sandi Metz has some awesome points about this in her book: http://www.r-5.org/files/books/computers/dev-teams/diagrams/Sandi_Metz-Practical_Object-Oriented_Design_in_Ruby-EN.pdf

I am trying to use my own message storage system because I am trying to defend my application's domain. I define domain as the data structures that define our systems. Meaning, that we communicate with our apps internally within our domain, and then we communicate with open interpreter through open interpreter's domain which is outside of our domain. So to maintain all information within our domain we want to map everything and store it according to our principles and use cases. But certainly as an out of the box solution having it managed by open interpreter makes a lot of sense.

[alita-moore]

oh also, it's a security risk to store user's messages even temporarily in a system that we don't control. Especially if that system is a shared execution environment where arbitrary code is executed..

[MikeBirdTech]

Closing this stale issue. Please create a new issue if the problem is not resolved or explained in the documentation. Thanks!