Skip to content

Pull requests: EleutherAI/lm-evaluation-harness

New pull request New
133 Open 1,523 Closed
133 Open 1,523 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Refactor ConfigurableTask.process_results into modular helpers
#3085 opened Jun 25, 2025 by mfisher35 • Draft
1
[Proposal] Change hyphens in n-shot and n-samples to underscores
#3084 opened Jun 24, 2025 by kiersten-stokes Loading…
Fix Typo in Answer Explanation
#3071 opened Jun 19, 2025 by leopardracer Loading…
2
Custom request headers | trust_remote_code param fix
#3069 opened Jun 18, 2025 by RawthiL Loading…
2
improve include-path precedence handling
#3068 opened Jun 18, 2025 by parkhs21 Loading…
1
3
Gracefully skip BigBench tasks with no data & guard final aggregation
#3066 opened Jun 17, 2025 by NourFahmy Loading…
1
Sync with base repository
#3064 opened Jun 17, 2025 by OliverSkola Loading…
1
Adding EgyMMLU and EgyHellaSwag
#3063 opened Jun 16, 2025 by houdaipha Loading…
2 of 6 tasks
Truthfulqa multi harness
#3062 opened Jun 16, 2025 by BlancaCalvo Loading…
2
Update Copyright Year in LICENSE.md to 2025
#3059 opened Jun 16, 2025 by maximevtush Loading…
1
Fix IFEval description
#3058 opened Jun 15, 2025 by vikhyat Loading…
1
Introduciton of TurthfulQA evaluation in Spanish, Catalan, Basque and Galician
#3052 opened Jun 11, 2025 by BlancaCalvo Loading…
4
add video modality and video demo task
#3049 opened Jun 10, 2025 by artemorloff Loading…
1
Ignore seed when splitting batch in chunks with groupby
#3047 opened Jun 9, 2025 by slimfrkha Loading…
1
first commit of Hungarian generative benchmark
#3043 opened Jun 6, 2025 by SzentmihalyiKr Loading…
1
[HF] fix quantization config
#3039 opened Jun 3, 2025 by baberabb Loading…
1
2
Added KazMMLU Task
#3037 opened Jun 2, 2025 by mukhituly Loading…
add kwargs passing into filters
#3036 opened Jun 2, 2025 by artemorloff Loading…
1
Add Wisent Guard hallucination evaluation pipeline
#3032 opened May 31, 2025 by e1washere Loading…
4
Tarjma bench
#3031 opened May 30, 2025 by Elma-dev Loading…
1
Try fixing issue 3026
#3029 opened May 29, 2025 by shanhx2000 Loading…
3
Update vLLM compatibility
#3024 opened May 28, 2025 by DarkLight1337 Loading…
1
Enable CUDA Graphs with vLLM Data Parallel
#3020 opened May 27, 2025 by ihebchaa Loading…
3
support distributed executor backend - torchrun
#3018 opened May 24, 2025 by kaiyuyue Loading…
1
Fix attribute error for quantization_config
#3016 opened May 23, 2025 by Justin900429 Loading…
1
ProTip! Add no:assignee to see everything that’s not assigned.