Add collate file and more tests from autogpt into testbed

[LeoLjl]

Why are these changes needed?

Add more tests from autogpt and reorganize file structure.

Related issue number

Checks

• I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
• I've added tests (if relevant) corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (1b4fb8f) 26.58% compared to head (2d762fe) 26.44%.
Report is 14 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #915      +/-   ##
==========================================
- Coverage   26.58%   26.44%   -0.14%     
==========================================
  Files          28       28              
  Lines        3732     3777      +45     
  Branches      847      858      +11     
==========================================
+ Hits          992      999       +7     
- Misses       2667     2706      +39     
+ Partials       73       72       -1     

Flag	Coverage Δ
unittests	26.44% <ø> (-0.08%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[LeoLjl]

The pipeline is ready to go, just more tests to be added.

[LeoLjl]

@afourney PTAL. I've added 13 tests, covering coding, scraping, file io. The pipeline is similar to that of HumanEval. I add some packages to requirements.txt because they are foundamental to task solving. I don't think it is neccesary to waste conversation turns for agents to install necessary packages.

[afourney]

reviewing now

[afourney]

@afourney PTAL. I've added 13 tests, covering coding, scraping, file io. The pipeline is similar to that of HumanEval. I add some packages to requirements.txt because they are foundamental to task solving. I don't think it is neccesary to waste conversation turns for agents to install necessary packages.

I think the question about installing packages is an interesting one. If they are common packages, then yes, we should expect them to already be installed. If, however, they are pretty esoteric and specific to the problem (e.g., yfinance) then identifying and installing the library is arguably part of the problem the agents need to work through to succeed. This is actually why I designed the Testbed the way that I did... so that each run would face identical obstacles.

Probably we want to identify some common packages and actually have them pre-installed on a Docker image. Identifying a core set of packages before looking at the problem set is likely ideal and free of bias. Perhaps something like https://learnpython.com/blog/most-popular-python-packages/

[afourney]

Looks good to me. I was able to run it no problem, and the documentation is good.

As per #976 there may be more work to do, but I am happy to accept it as is, then submit my own PR to standardize things as per #976 and #973.

Also, I think the jury is still out on if we want to allow the agents to continue after a failed test. This capability is no-doubt extremely useful to see how the agents adapt to a feedback signal (the same is true for HumanEval), but it makes the benchmarks incomparable to other reported numbers. An option to stop after the first attempt is something we may want to invest in, (and perhaps turn on by default). We should also at least make clear that our implementation diverges in this way