Skip to content
This repository has been archived by the owner on May 17, 2024. It is now read-only.

Issues: yujonglee/eval

28 Open 38 Closed
28 Open 38 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Accept multiple evaluator in runner
#173 opened Oct 22, 2023 by yujonglee
Remove evaluate module from HF
#170 opened Oct 15, 2023 by yujonglee
Add telemetry
#169 opened Oct 12, 2023 by yujonglee
Reference will not work with COT and grade mapping
#167 opened Oct 10, 2023 by yujonglee
Add grading docs for other providers
#166 opened Oct 10, 2023 by yujonglee
3 tasks
Return number directly from grading evaluator
#165 opened Oct 10, 2023 by yujonglee
1
Add dialog generator
#145 opened Sep 29, 2023 by yujonglee
RAGAS require OPENAI_API_KEY on import
#139 opened Sep 26, 2023 by yujonglee
Add variance in Analyzer
#128 opened Sep 21, 2023 by yujonglee
Add E2E test on Windows and MacOS
#117 opened Sep 18, 2023 by yujonglee
Better project description
#113 opened Sep 15, 2023 by yujonglee
Meta-eval should also consider None prediction
#103 opened Sep 10, 2023 by yujonglee
Use chat-bison for our CI
#100 opened Sep 10, 2023 by yujonglee
Better text formatting on thought generation
#99 opened Sep 10, 2023 by yujonglee
Add EvalResult type
#96 opened Sep 10, 2023 by yujonglee
Better None handling in Kappa calculation
#95 opened Sep 10, 2023 by yujonglee
1
Fix max_token=2 for togetherAI
#89 opened Sep 7, 2023 by yujonglee
1
Add Auto-fixing parser
#88 opened Sep 6, 2023 by yujonglee
Configure debias strategy for each LLM
#71 opened Aug 31, 2023 by yujonglee
Implement browser based human-eval
#62 opened Aug 31, 2023 by yujonglee
Human evaluation should be applied to a portion of the dataset
#55 opened Aug 29, 2023 by yujonglee
Use outlines's controlled generation
#54 opened Aug 29, 2023 by yujonglee
Adapt prompt from papers
#51 opened Aug 29, 2023 by yujonglee
Add sklearn.preprocessing.LabelEncoder
#39 opened Aug 26, 2023 by yujonglee
2
Add compare-based model eval
#36 opened Aug 25, 2023 by yujonglee
2
ProTip! Type g p on any issue or pull request to go back to the pull request listing page.