Skip to content

Issues: open-compass/opencompass

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

100 Open 264 Closed
100 Open 264 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

No module named 'opencompass'
#1210 opened May 29, 2024 by hellostronger
2 tasks done
@liushz
6
[Bug] hf_chatglm3_6b评测AFQMC数据集时，自测结果与官方不一致。且自测结果不稳定。
#1202 opened May 28, 2024 by jieheroli
2 tasks done
@bittersweet1999
opencompass榜单更新情况
#1201 opened May 28, 2024 by cobraheleah
1 task
@bittersweet1999
1
[Feature] config的bug，提示下载configs，然后下载了又出现以下bug
#1197 opened May 28, 2024 by 1518630367
1 task
@bittersweet1999
[Feature] 如何在needlebench 中使用api model？
#1196 opened May 28, 2024 by 1518630367
1 task
@DseidLi @bittersweet1999
1
[Feature] 支持openai/GPT4-o的评测seting
#1187 opened May 23, 2024 by Ezra-Yu
1 task
@tonysy
1
[Bug] 评测lawbench数据集时偶现异常
#1185 opened May 22, 2024 by yaoyingyy
2 tasks done
@liushz
[Bug] 使用api测评时mode参数不起作用，超出max_seq_len并没有按mode切分输入
#1181 opened May 22, 2024 by wz0424
2 tasks done
@liushz
[Feature] Leval数据集少了两个config：codeU和sci_fi
#1179 opened May 21, 2024 by liuxize-dhu
1 task
@bittersweet1999
[Bug] CLUEWSC的测试结果全是 50%
#1174 opened May 21, 2024 by SefaZeng
2 tasks done
@bittersweet1999
[Feature] Add -f for evaluation stage
#1172 opened May 17, 2024 by liushz
1 task
@tonysy
[Bug] 根据官方的Quick Start运行，输出结果为空
#1168 opened May 16, 2024 by MansonHua
2 tasks done
@kennymckormick
2
[Bug] 评测 Hellaswag 失败
#1166 opened May 16, 2024 by SefaZeng
2 tasks done
2
自定义的模型API都要配置步骤有哪些，除了写config文件外
#1165 opened May 15, 2024 by zhe123tc
1 task
@liushz
3
[Bug] RACE数据集在hf-llama-7b上的评测结果异常
#1164 opened May 15, 2024 by Li-jiaxian
2 tasks done
@liushz
[Feature] MMLU-PRO
#1161 opened May 15, 2024 by Ezra-Yu
1 task
@liushz
1
the prompt for triviaqa dataset
#1158 opened May 15, 2024 by mengrusun
1 task done
@liushz
[Bug] run with transformers==4.40.2, error "HuggingFacewithChatTemplate does not support ppl-based evaluation".
#1157 opened May 15, 2024 by zhulinJulia24
2 tasks done
@Leymore
1
[Feature] AGIEval数据集是否可以支持few-shot评测设置？
#1150 opened May 14, 2024 by cabbagecabbage
1 task
@bittersweet1999
[Bug] math_gen数据集评估随机失败
#1148 opened May 14, 2024 by berton820
2 tasks done
@bittersweet1999
4
[Feature] Typos in the official document
#1147 opened May 13, 2024 by Galaxy-Husky
1 task done
@Leymore
1
storycloze_gen 测试不通过，显示torch.distributed.elastic.multiprocessing.api.SignalException [Feature]
#1142 opened May 12, 2024 by dh12306
1 task
CMB + Qwen1.5-72B-Chat got empty answers
#1141 opened May 11, 2024 by qy1026
2 tasks done
@tonysy
1
cmb_gen [bug]
#1137 opened May 11, 2024 by songyao199681
2 tasks done
@tonysy
官网上每个数据集所使用的config文件的版本如何知道？
#1130 opened May 9, 2024 by jyshee
@kennymckormick
3
ProTip! no:milestone will show everything without a milestone.