Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

code for evaluation #10

Open
zthang opened this issue Aug 15, 2023 · 2 comments
Open

code for evaluation #10

zthang opened this issue Aug 15, 2023 · 2 comments

Comments

@zthang
Copy link

zthang commented Aug 15, 2023

Good work! Can you provide the evaluation code for the results presented in the paper (exactly math, f1, recall and precision)? Thanks!

@jennybae1024
Copy link

same request here! cannot reproduce the result on the paper when evaluating by whether the lowercase version of the output is in the lower case of the gold answer.

@LLSuzy
Copy link

LLSuzy commented Apr 3, 2024

Line 131 "examples = self_consistency(examples, anchor_text=anchor_text)" in prep.py file. This line reports an error.
The error message is that the self_consistency function is not defined.
Can you please tell me if the code for this function is publicly available. I can't find this function in any of the files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants