Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MATH tests to testbed #914

Merged
merged 4 commits into from
Dec 18, 2023
Merged

Add MATH tests to testbed #914

merged 4 commits into from
Dec 18, 2023

Conversation

yiranwu0
Copy link
Collaborator

@yiranwu0 yiranwu0 commented Dec 8, 2023

Why are these changes needed?

Add 17 level-5 problems from MATH to testbed.

Related issue number

Checks

@codecov-commenter
Copy link

codecov-commenter commented Dec 8, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (dd4a2da) 26.54% compared to head (1d2946d) 26.54%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #914   +/-   ##
=======================================
  Coverage   26.54%   26.54%           
=======================================
  Files          28       28           
  Lines        3805     3805           
  Branches      865      865           
=======================================
  Hits         1010     1010           
  Misses       2724     2724           
  Partials       71       71           
Flag Coverage Δ
unittests 26.49% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@afourney
Copy link
Member

afourney commented Dec 8, 2023

Thanks. Looks great from what I can see. I'll formally test and review this ASAP.

@afourney
Copy link
Member

afourney commented Dec 9, 2023

Looks good. I was able to get everything to run. As we onboard more benchmarks, I might do some modifications to how requirements files are handled, but that's for a later time.

I left a few comments. Address them, and I'll approve.

@yiranwu0
Copy link
Collaborator Author

yiranwu0 commented Dec 9, 2023

@afourney I didn't see any comments in the PR. Where do you put them?

@afourney
Copy link
Member

@kevin666aa You don't see 4 comments in this thread? What about via this link: https://github.com/microsoft/autogen/pull/914/files/3f22352755910ce88178731d71c322dfc67e6b36
?

@yiranwu0
Copy link
Collaborator Author

yiranwu0 commented Dec 12, 2023

@kevin666aa You don't see 4 comments in this thread? What about via this link: https://github.com/microsoft/autogen/pull/914/files/3f22352755910ce88178731d71c322dfc67e6b36 ?

Emm.. This is so weird, I just don't see any comments, even through the link.

Update: Asked @skzhang1, he cannot see it either.

@afourney
Copy link
Member

@kevin666aa That's so weird. Here are my comments:
image

@skzhang1
Copy link
Collaborator

@kevin666aa That's so weird. Here are my comments: image

You need to submit your comments.
https://github.com/orgs/community/discussions/10369

@afourney
Copy link
Member

Today I learned

Copy link
Member

@afourney afourney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

samples/tools/testbed/scenarios/MATH/README.md Outdated Show resolved Hide resolved
samples/tools/testbed/scenarios/MATH/problems.jsonl Outdated Show resolved Hide resolved
samples/tools/testbed/scenarios/MATH/scenario.py Outdated Show resolved Hide resolved
@afourney
Copy link
Member

Update looks good. Approved

@qingyun-wu qingyun-wu added this pull request to the merge queue Dec 18, 2023
Merged via the queue into main with commit aa946b3 Dec 18, 2023
16 checks passed
rlam3 pushed a commit to rlam3/autogen that referenced this pull request Dec 19, 2023
* add MATH eval to testbed

* update

---------

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
whiskyboy pushed a commit to whiskyboy/autogen that referenced this pull request Apr 17, 2024
* added documentation about small time budget

* small change for better clarity

* Update website/docs/Use-Cases/Task-Oriented-AutoML.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
whiskyboy pushed a commit to whiskyboy/autogen that referenced this pull request Apr 17, 2024
* add MATH eval to testbed

* update

---------

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants