SELA end2end by Trustccc · Pull Request #1621 · FoundationAgents/MetaGPT

Trustccc · 2024-12-03T12:34:13Z

Features

Added an end2end function to streamline the SELA process.
Modified the parameters of related functions to accommodate the new end-to-end functionality.
Updated the calling functions to ensure compatibility with the new end-to-end architecture.

Influence

This update will significantly reduce the complexity of the SELA process by automating multiple steps.
The new end2end function will allow for faster deployment and testing of SELA workflows, potentially improving overall system performance.

Other

This PR includes refactoring of several existing functions to ensure compatibility with the new end2end function.

metagpt/ext/sela/end2end_demo.py

garylin2099 · 2024-12-04T07:12:54Z

use pre-commit for formatting
improve README, add a quick start section
write PR description

…y`, and modified the related calling functions, and updated the README.md.

metagpt/ext/sela/run_sela.py

garylin2099 · 2024-12-04T11:30:06Z

metagpt/ext/sela/runner/sela.py

+        Extract and parse JSON content from the given string.
+        """
+        try:
+            return json.loads(json_string[json_string.find("{") : json_string.rfind("}") + 1])


you can ask the LLM to generate a ```json block and use metagpt.utils.common.CodeParser.parse_code to parse it

metagpt/ext/sela/end2end_demo.py

metagpt/ext/sela/runner/sela.py

hohoCode · 2024-12-04T19:54:22Z

Great work and cannot wait to try it when ready.

Just a general comment: can you make sure o1 is also fully supported with SELA? Because currently it seems o1 with metagpt's examples/customized_agents is having some combability issue.

garylin2099 · 2024-12-05T09:45:39Z

Great work and cannot wait to try it when ready.

Just a general comments: can you make sure o1 is also fully supported with SELA? Because currently it seems o1 with metagpt's examples/customized_agents is having some combability issue.

Thanks for your interest! We will test SELA with o1 and see if it works. Btw, SELA is able to achieve competitive performance even with deepseek, I assume you want to see if o1 can push up the limit further, or are there other considerations?

hohoCode · 2024-12-05T10:19:31Z

@garylin2099 Thanks! Yeah cannot wait to test its limits!

cyzus · 2024-12-05T15:30:42Z

metagpt/ext/sela/runner/sela.py

+        "use_fixed_insights": False,
+        "low_is_better": False,
+        "start_task_id": 2,
+        "from_scratch": False,


for the default config, maybe set from_scratch to be True since it's an end-to-end task and no insights are pre-generated

cyzus · 2024-12-05T15:33:52Z

metagpt/ext/sela/run_sela.py

+        "Report rmse on the eval data. Do not plot or make any visualizations."
+    )
+
+    data_dir = "/home/coder/project/chenxin/MetaGPT/metagpt/ext/sela/data/SELA-datasets/05_house-prices-advanced-regression-techniques"


remove this data dir

cyzus · 2024-12-05T15:35:13Z

metagpt/ext/sela/data/dataset.py

+        else:
+            data_path = os.path.join(datasets_dir, dataset["dataset"])
        split_datasets = {
            "train": os.path.join(data_path, "split_train.csv"),


maybe use Path from pathlib instead of os.path

…y`, and modified the related calling functions, and updated the README.md.

# Conflicts: # metagpt/ext/sela/README.md # metagpt/ext/sela/data/dataset.py # metagpt/ext/sela/run_sela.py # metagpt/ext/sela/runner/sela.py

garylin2099 · 2024-12-06T09:37:33Z

metagpt/ext/sela/runner/sela.py

+        Extract and parse JSON content from the given string using CodeParser.
+        """
+        try:
+            json_code = CodeParser.parse_code("", json_string, "json")


you dont need try except, let the error be raise up

garylin2099 · 2024-12-06T09:39:04Z

metagpt/ext/sela/runner/sela.py

+from metagpt.llm import LLM
+from metagpt.utils.common import CodeParser
+
+SELA_INSTRUCTION = """


REQ_PARSING_PROMPT

Trustccc had a problem deploying to unittest December 3, 2024 12:34 — with GitHub Actions Failure

add end2end

e5140c9

Trustccc force-pushed the dev branch from 64c7820 to e5140c9 Compare December 4, 2024 02:31

Trustccc had a problem deploying to unittest December 4, 2024 02:31 — with GitHub Actions Failure

Trustccc had a problem deploying to pre-commit December 4, 2024 07:03 — with GitHub Actions Failure

garylin2099 reviewed Dec 4, 2024

View reviewed changes

metagpt/ext/sela/end2end_demo.py Outdated Show resolved Hide resolved

metagpt/ext/sela/end2end_demo.py Outdated Show resolved Hide resolved

metagpt/ext/sela/end2end_demo.py Outdated Show resolved Hide resolved

metagpt/ext/sela/end2end_demo.py Outdated Show resolved Hide resolved

Refactored the end-to-end code with the entry point being `run_sela.p…

54c70d4

…y`, and modified the related calling functions, and updated the README.md.

Trustccc had a problem deploying to unittest December 4, 2024 10:36 — with GitHub Actions Failure

Trustccc had a problem deploying to unittest December 4, 2024 10:42 — with GitHub Actions Failure

Trustccc had a problem deploying to pre-commit December 4, 2024 11:14 — with GitHub Actions Failure

garylin2099 reviewed Dec 4, 2024

View reviewed changes

cyzus reviewed Dec 5, 2024

View reviewed changes

Trustccc added 2 commits December 6, 2024 11:50

Refactored the end-to-end code with the entry point being `run_sela.p…

c831303

…y`, and modified the related calling functions, and updated the README.md.

Merge remote-tracking branch 'origin/dev' into dev

535a2be

# Conflicts: # metagpt/ext/sela/README.md # metagpt/ext/sela/data/dataset.py # metagpt/ext/sela/run_sela.py # metagpt/ext/sela/runner/sela.py

Trustccc had a problem deploying to unittest December 6, 2024 03:55 — with GitHub Actions Failure

Trustccc had a problem deploying to unittest December 6, 2024 04:02 — with GitHub Actions Failure

Trustccc had a problem deploying to unittest December 6, 2024 06:26 — with GitHub Actions Failure

change dataset.py, delete end2end_demo.py

3a47b4e

Trustccc force-pushed the dev branch from 70045a2 to 3a47b4e Compare December 6, 2024 06:36

Trustccc had a problem deploying to unittest December 6, 2024 06:36 — with GitHub Actions Failure

garylin2099 reviewed Dec 6, 2024

View reviewed changes

Update sela README

db00fc4

Trustccc had a problem deploying to unittest December 6, 2024 10:59 — with GitHub Actions Failure

change the sela.py、run_sela.py

7895adb

Trustccc had a problem deploying to unittest December 9, 2024 06:00 — with GitHub Actions Failure

Comments

Conversation

Trustccc commented Dec 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

garylin2099 commented Dec 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

garylin2099 Dec 4, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hohoCode commented Dec 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

garylin2099 commented Dec 5, 2024

Uh oh!

hohoCode commented Dec 5, 2024

Uh oh!

cyzus Dec 5, 2024

Choose a reason for hiding this comment

Uh oh!

cyzus Dec 5, 2024

Choose a reason for hiding this comment

Uh oh!

cyzus Dec 5, 2024

Choose a reason for hiding this comment

Uh oh!

garylin2099 Dec 6, 2024

Choose a reason for hiding this comment

Uh oh!

garylin2099 Dec 6, 2024

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Trustccc commented Dec 3, 2024 •

edited

Loading

garylin2099 commented Dec 4, 2024 •

edited

Loading

hohoCode commented Dec 4, 2024 •

edited

Loading