Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: EleutherAI/lm-evaluation-harness
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: main
Choose a base ref
...
head repository: abacusai/lm-evaluation-harness
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref
Able to merge. These branches can be automatically merged.
  • 1 commit
  • 1 file changed
  • 1 contributor

Commits on Jun 24, 2024

  1. Fix regexp parsing for bbh_cot_fewshot.

    @arkapal3
    arkapal3 committed Jun 24, 2024
    Loading
    Loading status checks…

    Verified

    This commit was signed with the committer’s verified signature.
    hickford M Hickford
    SSH Key Fingerprint: XuoV1Drlvt14K6ywmLij1mATILPXpcV4OGWy0l7Nlzc
    Verified
    Learn about vigilant mode
    Copy the full SHA
    3e75c5e View commit details
Showing with 1 addition and 1 deletion.
  1. +1 −1 lm_eval/tasks/bbh/cot_fewshot/_cot_fewshot_template_yaml
2 changes: 1 addition & 1 deletion lm_eval/tasks/bbh/cot_fewshot/_cot_fewshot_template_yaml
View file
Original file line number Diff line number Diff line change
@@ -23,7 +23,7 @@ filter_list:
- name: "get-answer"
filter:
- function: "regex"
regex_pattern: "(?<=the answer is )(.*)(?=.)"
regex_pattern: "(?<=the answer is )(.*?)(?=\\s*$|\\s*\\.)"
- function: "take_first"
num_fewshot: 3
metadata: