Skip to content

Pull requests: bigcode-project/bigcode-evaluation-harness

New pull request New
21 Open 105 Closed
21 Open 105 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Fix unnecessary repeated overwrite
#249 opened Jun 29, 2024 by nielstron Loading…
fix: Multiple-E dataset fix go_test.go path for test execution
#225 opened Apr 20, 2024 by hitesh-1997 Loading…
1
4
Add llama3 instruction prompts
#222 opened Apr 19, 2024 by TechxGenus Loading…
1
Leaderboard README improvements
#217 opened Apr 14, 2024 by nikita1503 Loading…
4
remove pad tokens added by the accelerator.pad_across_processes
#216 opened Apr 13, 2024 by IQ17 Loading…
Ensure generations get saved in generation_only mode
#212 opened Mar 31, 2024 by Vipitis Loading…
1
fix apps evaluate error: local variable 'level' referenced before assignment
#206 opened Mar 10, 2024 by koking0 Loading…
1
Update README.md
#204 opened Mar 2, 2024 by AnitaLiu98 Loading…
4
Fix loading PAL-GSM few-shot examples
#196 opened Feb 8, 2024 by sxjscience Loading…
1
Make main.py compatible with OpenAI compatible APIs
#189 opened Jan 23, 2024 by hmellor Loading…
4
Fix typo in README.md
#177 opened Jan 2, 2024 by ab-10 Loading…
[WIP] Shadereval tasks
#173 opened Dec 16, 2023 by Vipitis • Draft
1 of 4 tasks
Add support for Ollama, Palm, Claude-2, Cohere, Replicate, Llama2 CodeLlama (100+LLMs) [LiteLLM]
#160 opened Nov 9, 2023 by ishaan-jaff Loading…
8
Dockerfile-multiple no longer fetches pip dependencies needlessly
#157 opened Nov 1, 2023 by RemcoSchrijver Loading…
Adding additional optional args for decoding flags and AutoModel kwargs to support models like ReplitLM
#115 opened Jul 12, 2023 by madhavatreplit Loading…
4
Support Seq2SeqLM model class (to facilitate the CodeT5+ models)
#104 opened Jun 26, 2023 by keyboardAnt Loading…
1
Attempt to make MultiPl-E's evaluation parallelization over all completions at once rather than just over each problem.
#86 opened Jun 7, 2023 by esslushy Loading…
15
Add: learning performance-improving code edits 🥧
#65 opened Apr 23, 2023 by SwayamInSync Loading…
6
Program repair
#64 opened Apr 22, 2023 by keyboardAnt Loading…
18
Add batch evaluation support when batch_size > 1
#36 opened Jan 23, 2023 by infinitylogesh Loading…
1
18
Add CodeXGLUE-code-refinement (few-shot) setting
#13 opened Oct 22, 2022 by manandey Loading…
9
ProTip! What’s not been updated in a month: updated:<2024-06-04.