feat(agent): Persistent execute_code session #7078

MKdir98 · 2024-04-08T04:11:06Z

Add notebook libs
Add two tests for the execute_code part
Add a Python kernel to each agent
Change the execute_code command part to persist the state of the codes

Background

As we talked about in Persistent Python session / Jupyter Notebook integration , we need to change the execute_code command to save the state of each executed code. We attach a Python kernel and a notebook to each agent in initializing the agent part. After that now we can use that kernel to run Python code with that kernel and this kernel will save the session until the agent gets destroyed. The error part may be a little dirty and I think maybe we can find something better instead of kernel.execute. You can check tests for execute_code to see scenarios we can support right now.

Changes 🏗️

In the execute_code command instead of creating a temp file and running that file, we run the code directly by python kernel.

PR Quality Scorecard ✨

Have you used the PR description template? +2 pts
Is your pull request atomic, focusing on a single change? +5 pts
Have you linked the GitHub issue(s) that this PR addresses? +5 pts
Have you documented your changes clearly and comprehensively? +5 pts
Have you changed or added a feature? -4 pts
- Have you added/updated corresponding documentation? +4 pts
- Have you added/updated corresponding integration tests? +5 pts
Have you changed the behavior of AutoGPT? -5 pts
- Have you also run agbenchmark to verify that these changes do not regress performance? +10 pts

github-actions · 2024-04-08T04:11:25Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

netlify · 2024-04-08T04:11:37Z

✅ Deploy Preview for auto-gpt-docs canceled.

Name	Link
🔨 Latest commit	`02897f4`
🔍 Latest deploy log	https://app.netlify.com/sites/auto-gpt-docs/deploys/6638e4b505296d00086db219

github-actions · 2024-04-22T17:24:27Z

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

Swiftyos · 2024-04-23T14:14:25Z

@CodiumAI-Agent /review

CodiumAI-Agent · 2024-04-23T14:15:57Z

PR Review

(Review updated until commit `60cdfd7`)

⏱️ Estimated effort to review [1-5]	4, due to the complexity of integrating Jupyter kernels and the need to ensure that the implementation does not introduce any security vulnerabilities or logical errors in handling kernel sessions and outputs.
🧪 Relevant tests	Yes
🔍 Possible issues	Possible Bug: The error handling in `execute_python_code` might not correctly format exceptions. The `CommandExecutionError` is raised with `*e.args` which might not correctly pass the exception details if `e` has more than one argument.
🔍 Possible issues	Resource Management: The kernel sessions started with `start_new_kernel` are not explicitly terminated or cleaned up, which could lead to resource leaks if kernels are not properly managed.
🔒 Security concerns	No

Code feedback:

relevant file	autogpts/autogpt/autogpt/agents/base.py
suggestion	Consider managing the lifecycle of the Python kernel to prevent resource leaks. You should ensure that kernels are properly shut down when they are no longer needed or when the agent is destroyed. This can be done by implementing a cleanup method in the `BaseAgentSettings` class that calls `self.python_kernel.shutdown()`. [important]
relevant line	self.python_kernel = start_new_kernel(

relevant file	autogpts/autogpt/autogpt/commands/execute_code.py
suggestion	Improve the exception handling in `execute_python_code` to ensure that all parts of the exception are correctly formatted and passed to `CommandExecutionError`. Instead of using `*e.args`, you might want to directly pass the string representation of the exception or format it to include more context. [important]
relevant line	raise CommandExecutionError(*e.args)

relevant file	autogpts/autogpt/autogpt/commands/execute_code.py
suggestion	To enhance code readability and maintainability, consider abstracting the logic that handles the output and errors from the kernel into separate methods. This can help in isolating the functionality and making the code easier to manage and test. [medium]
relevant line	agent.python_kernel.execute(code)

relevant file	autogpts/autogpt/autogpt/agents/base.py
suggestion	Add error handling for the kernel start-up process in the `__init__` method of `BaseAgentSettings`. This should manage scenarios where the kernel fails to start, possibly due to configuration issues or system limitations. [important]
relevant line	self.python_kernel = start_new_kernel(

✨ Review tool usage guide:

Overview:
The review tool scans the PR code changes, and generates a PR review which includes several types of feedbacks, such as possible PR issues, security threats and relevant test in the PR. More feedbacks can be added by configuring the tool.

The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR.

When commenting, to edit configurations related to the review tool (pr_reviewer section), use the following template:

/review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...

With a configuration file, use the following template:

[pr_reviewer]
some_config1=...
some_config2=...

See the review usage page for a comprehensive guide on using this tool.

github-actions · 2024-05-02T12:43:53Z

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

MKdir98 · 2024-05-02T12:46:12Z

@CodiumAI-Agent /review

CodiumAI-Agent · 2024-05-02T12:47:37Z

Persistent review updated to latest commit 60cdfd7

codecov · 2024-05-02T14:20:58Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 44.81%. Comparing base (9bac6f4) to head (02897f4).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7078      +/-   ##
==========================================
+ Coverage   44.65%   44.81%   +0.16%     
==========================================
  Files         133      133              
  Lines        6306     6321      +15     
  Branches      822      824       +2     
==========================================
+ Hits         2816     2833      +17     
+ Misses       3379     3377       -2     
  Partials      111      111

Flag	Coverage Δ
Linux	`44.73% <100.00%> (+0.16%)`	⬆️
Windows	`42.72% <34.78%> (-0.01%)`	⬇️
autogpt-agent	`44.78% <100.00%> (+0.16%)`	⬆️
macOS	`43.94% <34.78%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Pwuts

Thank you for submitting this! It needs a bit of work but it will be very cool to get this merged :)

autogpts/autogpt/autogpt/agents/base.py

autogpts/autogpt/autogpt/agents/components.py

autogpts/autogpt/autogpt/agents/base.py

autogpts/autogpt/autogpt/commands/execute_code.py

autogpts/autogpt/pyproject.toml

autogpts/autogpt/tests/integration/test_execute_code.py

github-actions · 2024-05-04T18:37:44Z

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

github-actions · 2024-05-06T14:06:51Z

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

- Add notebook libs - Add two tests for the execute_code part - Add a Python kernel to each agent - Change the execute_code command part to persist the state of the codes

kcze

This looks ok to me overall and;

Introduces new dependencies
I'm not sure if this is going to work properly in container and on agent protocol server (please test) and also refer to feat(agent): Implement local temporary dir #7115

kcze · 2024-05-08T11:09:07Z

autogpts/autogpt/poetry.lock

-benchmark = ["agbenchmark @ git+https://github.com/Significant-Gravitas/AutoGPT.git#subdirectory=benchmark"]
+benchmark = ["agbenchmark @ file:///home/mk/.cache/pypoetry/virtualenvs/agpt-xgev2_OR-py3.11/src/AutoGPT/benchmark"]


This happens to me for some reason as well, please revert this to git address and be careful when updating poetry packages.

Oh sure thanks

MKdir98 · 2024-05-08T12:10:11Z

This looks ok to me overall and;

Introduces new dependencies

I'm not sure if this is going to work properly in container and on agent protocol server (please test) and also refer to feat(agent): Implement local temporary dir #7115

OK. I'll test it with agent protocol

ntindle · 2024-05-09T06:26:37Z

autogpts/autogpt/autogpt/commands/execute_code.py

        if not self.legacy_config.execute_local_commands:
            logger.info(
                "Local shell commands are disabled. To enable them,"
                " set EXECUTE_LOCAL_COMMANDS to 'True' in your config file."
            )
+        self.notebook = new_notebook()


With this, does it always make a new notebook on init between instances of the agent? Will that session be shared across multiple agents running within the agent protocol service?

@kcze not sure how components are init'ed yet

ntindle · 2024-05-09T06:27:41Z

autogpts/autogpt/autogpt/commands/execute_code.py

+    def _execute_code_in_agent_python_session(self, code: str) -> str:
+        """
+        This function will run code on python_kernel.`self.python_kernel.execute(code)`
+        does not return stdout and error direclty.The while part


nit

Suggested change

does not return stdout and error direclty.The while part

does not return stdout and error directly. The while part

ntindle · 2024-05-09T06:28:59Z

autogpts/autogpt/autogpt/commands/execute_code.py

+                    raise IOError(
+                        f"{io_msg['content']['evalue']}\n \
+                            {io_msg['content']['traceback']}"
+                    )


nit: is IOError appropriate here?

MKdir98 requested a review from a team as a code owner April 8, 2024 04:11

github-actions bot added AutoGPT Agent size/xl labels Apr 8, 2024

MKdir98 mentioned this pull request Apr 8, 2024

Persistent Python session / Jupyter Notebook integration #5132

Open

1 task

github-actions bot added the conflicts Automatically applied to PRs with merge conflicts label Apr 22, 2024

github-actions bot removed the conflicts Automatically applied to PRs with merge conflicts label May 2, 2024

MKdir98 force-pushed the feat/jupyter-notebook-integration branch from 970c95c to 60cdfd7 Compare May 2, 2024 12:45

MKdir98 force-pushed the feat/jupyter-notebook-integration branch 2 times, most recently from 42b8c17 to 4f91372 Compare May 2, 2024 14:16

MKdir98 force-pushed the feat/jupyter-notebook-integration branch 5 times, most recently from 23ffd50 to 97ac81b Compare May 2, 2024 14:56

Pwuts requested changes May 2, 2024

View reviewed changes

MKdir98 force-pushed the feat/jupyter-notebook-integration branch 3 times, most recently from 28b9510 to 6a8e728 Compare May 3, 2024 13:59

github-actions bot added the conflicts Automatically applied to PRs with merge conflicts label May 4, 2024

MKdir98 force-pushed the feat/jupyter-notebook-integration branch from 6a8e728 to 4fabc34 Compare May 6, 2024 14:06

github-actions bot removed the conflicts Automatically applied to PRs with merge conflicts label May 6, 2024

feat(agent): Persistent execute_code session

02897f4

- Add notebook libs - Add two tests for the execute_code part - Add a Python kernel to each agent - Change the execute_code command part to persist the state of the codes

MKdir98 force-pushed the feat/jupyter-notebook-integration branch from 4fabc34 to 02897f4 Compare May 6, 2024 14:09

kcze requested changes May 8, 2024

View reviewed changes

ntindle reviewed May 9, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(agent): Persistent execute_code session #7078

feat(agent): Persistent execute_code session #7078

MKdir98 commented Apr 8, 2024 •

edited

github-actions bot commented Apr 8, 2024

netlify bot commented Apr 8, 2024 •

edited

github-actions bot commented Apr 22, 2024

Swiftyos commented Apr 23, 2024

CodiumAI-Agent commented Apr 23, 2024 •

edited

github-actions bot commented May 2, 2024

MKdir98 commented May 2, 2024

CodiumAI-Agent commented May 2, 2024

codecov bot commented May 2, 2024 •

edited

Pwuts left a comment

github-actions bot commented May 4, 2024

github-actions bot commented May 6, 2024

kcze left a comment

kcze May 8, 2024

MKdir98 May 8, 2024

MKdir98 commented May 8, 2024

ntindle May 9, 2024

ntindle May 9, 2024 •

edited

ntindle May 9, 2024

		benchmark = ["agbenchmark @ git+https://github.com/Significant-Gravitas/AutoGPT.git#subdirectory=benchmark"]
		benchmark = ["agbenchmark @ file:///home/mk/.cache/pypoetry/virtualenvs/agpt-xgev2_OR-py3.11/src/AutoGPT/benchmark"]

	does not return stdout and error direclty.The while part
	does not return stdout and error directly. The while part

feat(agent): Persistent execute_code session #7078

Are you sure you want to change the base?

feat(agent): Persistent execute_code session #7078

Conversation

MKdir98 commented Apr 8, 2024 • edited

Background

Changes 🏗️

PR Quality Scorecard ✨

github-actions bot commented Apr 8, 2024

netlify bot commented Apr 8, 2024 • edited

✅ Deploy Preview for auto-gpt-docs canceled.

github-actions bot commented Apr 22, 2024

Swiftyos commented Apr 23, 2024

CodiumAI-Agent commented Apr 23, 2024 • edited

PR Review

(Review updated until commit 60cdfd7)

github-actions bot commented May 2, 2024

MKdir98 commented May 2, 2024

CodiumAI-Agent commented May 2, 2024

codecov bot commented May 2, 2024 • edited

Codecov Report

Pwuts left a comment

Choose a reason for hiding this comment

github-actions bot commented May 4, 2024

github-actions bot commented May 6, 2024

kcze left a comment

Choose a reason for hiding this comment

kcze May 8, 2024

Choose a reason for hiding this comment

MKdir98 May 8, 2024

Choose a reason for hiding this comment

MKdir98 commented May 8, 2024

ntindle May 9, 2024

Choose a reason for hiding this comment

ntindle May 9, 2024 • edited

Choose a reason for hiding this comment

ntindle May 9, 2024

Choose a reason for hiding this comment

MKdir98 commented Apr 8, 2024 •

edited

netlify bot commented Apr 8, 2024 •

edited

CodiumAI-Agent commented Apr 23, 2024 •

edited

(Review updated until commit `60cdfd7`)

codecov bot commented May 2, 2024 •

edited

ntindle May 9, 2024 •

edited