Skip to content

Issues: stanford-crfm/helm

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

116 Open 970 Closed
116 Open 970 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Support few-shot chain-of-thought in GPQA / MMLU
#3088 opened Oct 22, 2024 by yifanmai
Issue with running HEIM documentation Improvements or additions to documentation HEIM (Text2Image) user question
#3080 opened Oct 22, 2024 by sudhir-mcw
2
flake8 in Python 3.12 produces a large number of errors
#3072 opened Oct 18, 2024 by yifanmai
Optimum Intel OpenVino fails with segmentation fault bug Something isn't working models
#3066 opened Oct 16, 2024 by yifanmai
1
How to use this package when I have the pompts and images across models HEIM (Text2Image) user question
#3062 opened Oct 14, 2024 by snehith3195
1
Not able to install install-heim-extras.sh for heim leaderboard HEIM (Text2Image) user question
#3060 opened Oct 11, 2024 by snehith3195
1
Proposal: Multi-turn support
#3059 opened Oct 10, 2024 by yifanmai
Proposal: User-provided custom adapters
#3058 opened Oct 10, 2024 by yifanmai
Proposal: Add free-form data fields to Instance
#3057 opened Oct 10, 2024 by yifanmai
1
HatefulMemesScenario get_instances returning error bug Something isn't working HEIM (Text2Image) methodology Evaluation methodology user question
#3056 opened Oct 10, 2024 by dxwu2
3
Add new model (upstage/solar-pro-preview-instruct)
#3040 opened Oct 7, 2024 by remy-rec
1
Add MMLU-Pro additions New models or scenarios good first issue Good for newcomers scenarios
#3018 opened Sep 24, 2024 by yifanmai
@siyagoel
Help me understand the HELM Classic Leaderboard's missing results user question
#2994 opened Sep 16, 2024 by PaulJoeMaliakel
1
Add amazon bedrock 3p models to the model configuration. user question
#2967 opened Sep 3, 2024 by subhaviv
2
Incorrect scoring due to answer format mismatch in MMLU evaluation user question
#2939 opened Aug 16, 2024 by DerryChan
1
Add Windows support
#2907 opened Aug 8, 2024 by yifanmai
Add sorting/filtering functionality to predictions pages frontend Frontend issues
#2887 opened Aug 1, 2024 by farzaank
Add metadata field to HELM predictions enhancement New feature or request frontend Frontend issues
#2886 opened Aug 1, 2024 by farzaank
2
HEIMHumanEvalScenario requires permissions to download data from codalab HEIM (Text2Image) user question
#2865 opened Jul 30, 2024 by slymane
1
pip needs to be temporarily downgraded to 24.1.2
#2855 opened Jul 30, 2024 by yifanmai
Update tutorial for helm-server
#2805 opened Jul 17, 2024 by farzaank
Link to Image2Struct is broken user question
#2785 opened Jul 2, 2024 by zhimin-z
1
Sort by correctness functionality on Predictions page in front end
#2783 opened Jun 29, 2024 by farzaank
@farzaank
Annotator issues
#2697 opened May 31, 2024 by yifanmai
4 tasks
@farzaank
Support for Hugging Face Local Models documentation Improvements or additions to documentation
#2690 opened May 30, 2024 by NoushNabi
1
ProTip! Follow long discussions with comments:>50.