feat(forge): Implement a very very bad, but "functional" agent by default #7021

ntindle · 2024-03-17T05:36:00Z

Background

We’ve received some notes about Forge being difficult to use and a bit broken. This PR aims to solve some of the core issues with that.

Changes 🏗️

Fixes various issues
- Windows Compatibility
- FastAPI interface Issues
- Hardcoded paths removed
- Formatting
- Ability -> Action Fixes
- Prompt
- Imports
- Actions Files themselves
- Notes for the codebase
- Endpoint Error message fixes
Updates typing throughout
Swaps agent from one that always replies Washington DC to one that has a very bad agent loop

Do we want to keep Washington DCcand make a real agent instead?

PR Quality Scorecard ✨

Have you used the PR description template? +2 pts
Is your pull request atomic, focusing on a single change? +5 pts
Have you linked the GitHub issue(s) that this PR addresses? +5 pts
Have you documented your changes clearly and comprehensively? +5 pts
Have you changed or added a feature? -4 pts
- Have you added/updated corresponding documentation? +4 pts
- Have you added/updated corresponding integration tests? +5 pts
Have you changed the behavior of AutoGPT? -5 pts
- Have you also run agbenchmark to verify that these changes do not regress performance? +10 pts

Note, this is a very bad agent lol

github-actions · 2024-03-17T05:36:17Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

netlify · 2024-03-17T05:37:15Z

✅ Deploy Preview for auto-gpt-docs ready!

Name	Link
🔨 Latest commit	`26b582c`
🔍 Latest deploy log	https://app.netlify.com/sites/auto-gpt-docs/deploys/663caca5d05ea400076e1685
😎 Deploy Preview	https://deploy-preview-7021--auto-gpt-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

ntindle · 2024-03-17T05:40:43Z

autogpts/forge/forge/sdk/model.py

        description="The creation datetime of the task.",
        example="2023-01-01T00:00:00Z",
-        json_encoders={datetime: lambda v: v.isoformat()},


This is why the openapi docs doesn't work btw

ntindle · 2024-03-17T05:40:58Z

autogpts/forge/forge/sdk/prompting.py

@@ -108,6 +108,7 @@ def load_prompt(self, template: str, **kwargs) -> str:
            template = os.path.join(self.model, template)
            if self.debug_enabled:
                LOG.debug(f"Loading template: {template}")
+            template = template.replace("\\", "/")


windows compat

Why needed for Windows compat? Do you know of Path.as_posix()?

ntindle · 2024-03-17T05:41:23Z

autogpts/forge/forge/actions/file_system/files.py

-            "type": "string",
-            "required": True,
-        }
+        ActionParameter(


ntindle · 2024-03-17T05:41:29Z

autogpts/forge/forge/actions/file_system/files.py

    ],
    output_type="list[str]",
 )
-async def list_files(agent, task_id: str, path: str) -> List[str]:
+async def list_files(agent: Agent, task_id: str, path: str) -> List[str]:


ntindle · 2024-03-17T05:42:23Z

autogpts/forge/forge/actions/registry.py

@@ -186,8 +188,11 @@ async def run_action(

 if __name__ == "__main__":
    import sys
+    import asyncio


swifty removal and replacement with more amenable action to not actually having an agent seeded. list_files requires a full agent to be able to be pulled in to actually work

ntindle · 2024-03-17T05:42:38Z

autogpts/forge/forge/actions/web/web_selenium.py

@@ -231,7 +233,9 @@ async def read_webpage(
    except WebDriverException as e:
        # These errors are often quite long and include lots of context.
        # Just grab the first line.
-        msg = e.msg.split("\n")[0]
+        msg = "An error occurred while trying to load the page"


ntindle · 2024-03-17T05:43:33Z

autogpts/forge/forge/agent.py

+        # self.workspace.write(task_id=task_id, path="output.txt", data=b"Washington D.C")
+
+        # await self.db.create_artifact(
+        #     task_id=task_id,
+        #     step_id=step.step_id,
+        #     file_name="output.txt",
+        #     relative_path="",
+        #     agent_created=True,
+        # )
+
+        # step.output = "Washington D.C"
+
+        # LOG.info(
+        #     f"\t✅ Final Step completed: {step.step_id}. \n"
+        #     + f"Output should be placeholder text Washington D.C. You'll need to \n"
+        #     + f"modify execute_step to include LLM behavior. Follow the tutorial "
+        #     + f"if confused. "
+        # )
+
+        # return step


this is from the tutorial so left here, but replaced with something that "can" work. Its bad at responding though. Could really use some help on the prompts to maek this work sometimes

ntindle · 2024-03-17T05:43:58Z

autogpts/forge/forge/sdk/__init__.py

@@ -2,7 +2,7 @@
 The Forge SDK. This is the core of the Forge. It contains the agent protocol, which is the
 core of the Forge.
 """
-from ..llm import chat_completion_request, create_embedding_request, transcribe_audio
+# from forge.llm import chat_completion_request, create_embedding_request, transcribe_audio


Circular import fix.

Why do we need this file?

ntindle · 2024-03-17T05:44:22Z

autogpts/forge/forge/sdk/agent.py

+
+        Note: this is a placeholder method and must be overridden by the agent with the actions
+        that the agent should perform on one step. Override this method in the ForgeAgent class.


Pointer for users of this to look in the right place

ntindle · 2024-03-17T05:46:15Z

autogpts/forge/forge/sdk/routes/agent_protocol.py

+        raise HTTPException(
+            status_code=500,
+            detail=json.dumps(


pydantic prefers raise over response

ntindle · 2024-03-17T05:46:31Z

autogpts/forge/forge/sdk/routes/agent_protocol.py

+    page: Optional[int] = Query(default=1, ge=1),
+    page_size: Optional[int] = Query(default=10, ge=1, alias="pageSize"),


ntindle · 2024-03-17T05:46:44Z

autogpts/forge/forge/sdk/routes/agent_protocol.py

-            status_code=200,
-            media_type="application/json",
-        )
+        steps = await agent.list_steps(task_id, page or 1, page_size or 10)


type error fix

ntindle · 2024-03-17T05:46:56Z

autogpts/forge/forge/sdk/routes/agent_protocol.py

    try:
        # An empty step request represents a yes to continue command
        if not step:
-            step = StepRequestBody(input="y")
+            step = StepRequestBody(name=None, input="y")


type error fix

github-actions · 2024-04-03T21:47:00Z

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

Pwuts

Looks good overall

Pwuts · 2024-04-07T20:58:48Z

autogpts/forge/forge/sdk/prompting.py

@@ -108,6 +108,7 @@ def load_prompt(self, template: str, **kwargs) -> str:
            template = os.path.join(self.model, template)
            if self.debug_enabled:
                LOG.debug(f"Loading template: {template}")
+            template = template.replace("\\", "/")


Why needed for Windows compat? Do you know of Path.as_posix()?

autogpts/forge/forge/agent.py

…/AutoGPT into forge/fixes

Swiftyos · 2024-04-23T14:14:50Z

@CodiumAI-Agent /review

CodiumAI-Agent · 2024-04-23T14:17:47Z

PR Review

⏱️ Estimated effort to review [1-5]	4, because the PR includes a wide range of changes across multiple files and functionalities, including updates to action definitions, file operations, and agent behavior. The complexity and breadth of these changes require a thorough review to ensure that all modifications are correct and do not introduce new issues.
🧪 Relevant tests	No
🔍 Possible issues	Possible Bug: The new agent loop might not handle exceptions properly, leading to unhandled exceptions if the JSON response from `chat_completion_request` is malformed or if the action execution fails.
🔍 Possible issues	Performance Concern: The use of synchronous file operations in an asynchronous context could lead to blocking behavior and reduce the efficiency of the application.
🔒 Security concerns	No

Code feedback:

relevant file	autogpts/forge/forge/__main__.py
suggestion	Consider removing the placeholder ASCII art added in the main file, as it does not contribute to the functionality and clutters the codebase. [important]
relevant line

relevant file	autogpts/forge/forge/actions/file_system/files.py
suggestion	Replace the synchronous file operations with asynchronous versions to avoid blocking the event loop in an async environment. This change will improve the performance and responsiveness of the application. [important]
relevant line	return agent.workspace.list(task_id=task_id, path=str(path))

relevant file	autogpts/forge/forge/agent.py
suggestion	Implement error handling for JSON parsing and action execution within the `execute_step` method to prevent the agent from crashing due to unexpected exceptions. [important]
relevant line	chat_response = await chat_completion_request(**chat_completion_kwargs)

relevant file	autogpts/forge/forge/actions/registry.py
suggestion	Ensure cross-platform compatibility by handling path separators correctly when registering actions. The current replacement might not cover all edge cases. [medium]
relevant line	.replace("\\", "/")

✨ Review tool usage guide:

Overview:
The review tool scans the PR code changes, and generates a PR review which includes several types of feedbacks, such as possible PR issues, security threats and relevant test in the PR. More feedbacks can be added by configuring the tool.

The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR.

When commenting, to edit configurations related to the review tool (pr_reviewer section), use the following template:

/review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...

With a configuration file, use the following template:

[pr_reviewer]
some_config1=...
some_config2=...

See the review usage page for a comprehensive guide on using this tool.

Pwuts

Application crashes when the LLM picks a non-existent command, which is also why it fails CI

ntindle added 12 commits March 16, 2024 23:14

fix: remove json_encoders. FastAPI does that by default

f8e9582

fix: typing and return objects

26b902d

fix: type fixes on the parameters

177d62f

fix: windows won't load actions

d22762c

fix: remove swifty's paths

a655ebf

fix: type hinting

2f394d3

feat: add note about execute_step in the sdk

7090ca7

fix: remove circular import

a00f65e

fix: abilities -> Actions

a8741db

fix: windows compatibility

5dff243

feat: basic agent rather than just returning washington dc

3604484

Note, this is a very bad agent lol

fix: uncommented line

be4f0d5

ntindle added behavior change behaviour change Arena Agents Entering the Arena Forge labels Mar 17, 2024

ntindle requested a review from Pwuts March 17, 2024 05:36

github-actions bot removed the Arena Agents Entering the Arena label Mar 17, 2024

github-actions bot added the size/xl label Mar 17, 2024

ntindle commented Mar 17, 2024

View reviewed changes

autogpts/forge/forge/actions/file_system/files.py

"type": "string",

"required": True,

}

ActionParameter(

Copy link

Member Author

ntindle Mar 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

type fix

ntindle commented Mar 17, 2024

View reviewed changes

Merge branch 'master' into forge/fixes

fe966c8

Pwuts previously approved these changes Apr 7, 2024

View reviewed changes

Pwuts removed the behavior change label Apr 7, 2024

ntindle added 2 commits April 19, 2024 10:50

Merge branch 'master' into forge/fixes

fb60162

fix: linting error

ef6cc4a

ntindle dismissed Pwuts’s stale review via ef6cc4a April 20, 2024 03:22

fix: pull request changes

aed8f0e

ntindle requested a review from Pwuts April 20, 2024 03:30

ntindle added 7 commits April 22, 2024 09:03

fix: isort

2899e60

Merge branch 'forge/fixes' of https://github.com/Significant-Gravitas…

87eabfd

…/AutoGPT into forge/fixes

fix: isort after merge

4fc9735

fix: incorrectly refactored param template-> template_name

b215505

fix: docstring

36d66c6

Merge branch 'master' into forge/fixes

94962df

Merge branch 'master' into forge/fixes

960fe39

ntindle added 5 commits April 26, 2024 19:05

Merge branch 'master' into forge/fixes

9d7672f

Merge branch 'master' into forge/fixes

1d50476

Merge branch 'master' into forge/fixes

9728fec

Merge branch 'master' into forge/fixes

2af3724

Merge branch 'master' into forge/fixes

45b6867

Pwuts changed the title ~~Update the forge to have a very very bad, but "functional" agent by default~~ feat(forge): Implement a very very bad, but "functional" agent by default May 3, 2024

Pwuts requested changes May 3, 2024

View reviewed changes

Merge branch 'master' into forge/fixes

26b582c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(forge): Implement a very very bad, but "functional" agent by default #7021

feat(forge): Implement a very very bad, but "functional" agent by default #7021

ntindle commented Mar 17, 2024 •

edited

github-actions bot commented Mar 17, 2024

netlify bot commented Mar 17, 2024 •

edited

ntindle Mar 17, 2024

ntindle Mar 17, 2024

Pwuts Apr 7, 2024

ntindle Mar 17, 2024

ntindle Mar 17, 2024

ntindle Mar 17, 2024

ntindle Mar 17, 2024

ntindle Mar 17, 2024

ntindle Mar 17, 2024

ntindle Mar 17, 2024

ntindle Mar 17, 2024

ntindle Mar 17, 2024

ntindle Mar 17, 2024

ntindle Mar 17, 2024

github-actions bot commented Apr 3, 2024

Pwuts left a comment

Pwuts Apr 7, 2024

Swiftyos commented Apr 23, 2024

CodiumAI-Agent commented Apr 23, 2024

Pwuts left a comment


		Note: this is a placeholder method and must be overridden by the agent with the actions
		that the agent should perform on one step. Override this method in the ForgeAgent class.

		page: Optional[int] = Query(default=1, ge=1),
		page_size: Optional[int] = Query(default=10, ge=1, alias="pageSize"),

feat(forge): Implement a very very bad, but "functional" agent by default #7021

Are you sure you want to change the base?

feat(forge): Implement a very very bad, but "functional" agent by default #7021

Conversation

ntindle commented Mar 17, 2024 • edited

Background

Changes 🏗️

PR Quality Scorecard ✨

github-actions bot commented Mar 17, 2024

netlify bot commented Mar 17, 2024 • edited

✅ Deploy Preview for auto-gpt-docs ready!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Apr 3, 2024

Pwuts left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Swiftyos commented Apr 23, 2024

CodiumAI-Agent commented Apr 23, 2024

PR Review

Pwuts left a comment

Choose a reason for hiding this comment

ntindle commented Mar 17, 2024 •

edited

netlify bot commented Mar 17, 2024 •

edited