Update nq_open.yaml (EleutherAI#1305)

* Update nq_open.yaml change regex * Bump NQ version --------- Co-authored-by: Hailey Schoelkopf <65563625+haileyschoelkopf@users.noreply.github.com>
anjor · Jan 31, 2024 · 8783281 · 8783281
1 parent 0013399
commit 8783281
Showing 1 changed file with 2 additions and 2 deletions.
diff --git a/lm_eval/tasks/nq_open/nq_open.yaml b/lm_eval/tasks/nq_open/nq_open.yaml
@@ -27,6 +27,6 @@ metric_list:
     ignore_case: true
     ignore_punctuation: true
     regexes_to_ignore:
-    - "\ban|a|the\b"
+    - "\\b(?:The |the |An |A |The |a |an )"
 metadata:
-  version: 2.0
+  version: 3.0