-
Notifications
You must be signed in to change notification settings - Fork 317
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
LLM Autograding manual graded questions #9887
LLM Autograding manual graded questions #9887
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #9887 +/- ##
==========================================
- Coverage 69.23% 69.01% -0.23%
==========================================
Files 484 486 +2
Lines 76262 76380 +118
Branches 6447 6439 -8
==========================================
- Hits 52799 52710 -89
- Misses 23012 23219 +207
Partials 451 451 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is by no means a complete or formal review, I'm only providing some initial thoughts.
...elearn/src/pages/instructorAssessmentManualGrading/assessmentQuestion/assessmentQuestion.ejs
Outdated
Show resolved
Hide resolved
...ielearn/src/pages/instructorAssessmentManualGrading/assessmentQuestion/assessmentQuestion.ts
Show resolved
Hide resolved
...ielearn/src/pages/instructorAssessmentManualGrading/assessmentQuestion/assessmentQuestion.ts
Outdated
Show resolved
Hide resolved
...ielearn/src/pages/instructorAssessmentManualGrading/assessmentQuestion/assessmentQuestion.ts
Outdated
Show resolved
Hide resolved
...ielearn/src/pages/instructorAssessmentManualGrading/assessmentQuestion/assessmentQuestion.ts
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've been working closely with Victor throughout this PR, so most of the comments and changes, I've just shared with him directly offline.
There is obviously a lot more work to do in order to make these features work well, but I think this represents a good MVP for something that works minimally enough so that we should merge it and make additional changes incrementally.
...ielearn/src/pages/instructorAssessmentManualGrading/assessmentQuestion/assessmentQuestion.ts
Outdated
Show resolved
Hide resolved
@Victorsss-Orz it looks like you have conflicts with the yarn.lock file. One way to fix this would be to re-check out the version of yarn.lock from the master branch, and then re-run your |
9bf9717
to
43e776e
Compare
Done |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A few more comments, then this should be good to merge.
...ielearn/src/pages/instructorAssessmentManualGrading/assessmentQuestion/assessmentQuestion.ts
Outdated
Show resolved
Hide resolved
Looking forward to getting this merged |
This is a part of the SIIP grant for Automatic Short Answer Grading using Large Language Models. The first PR works on the real basic functions, and more features will be added later.
![image](https://private-user-images.githubusercontent.com/97915774/333123425-4d8994af-863d-4b5a-9c17-5e3cfd748154.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTk0MDYxMDAsIm5iZiI6MTcxOTQwNTgwMCwicGF0aCI6Ii85NzkxNTc3NC8zMzMxMjM0MjUtNGQ4OTk0YWYtODYzZC00YjVhLTljMTctNWUzY2ZkNzQ4MTU0LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MjYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjI2VDEyNDMyMFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWE5NGU2MWQyOThlYzVkYzY5OTk1NzFmZTRhZmNkNmI4NGMxNjE2ZmM1YzNlZjJkNGU0MmQyNTllMGZmYjJlZGImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.ezKiie1tK050hWJObVRivrQYr1yc1x1mWlPkIjbP-2c)
![image](https://private-user-images.githubusercontent.com/97915774/333123501-177f9df7-6e50-493a-9e16-6821fb424d78.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTk0MDYxMDAsIm5iZiI6MTcxOTQwNTgwMCwicGF0aCI6Ii85NzkxNTc3NC8zMzMxMjM1MDEtMTc3ZjlkZjctNmU1MC00OTNhLTllMTYtNjgyMWZiNDI0ZDc4LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MjYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjI2VDEyNDMyMFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTU1YjdhN2ZlZDlkZGY5ZWI2YmNhNmYxNDAyZWM2ZDQ2NDBlNDgxNjM5YTE3ODk3NTQ1ZjU3ZjIwZTJjMTYxYTQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.ybiUbvSgtL_joUNH3O90AOZZtKEM-vWAbgpn34qi9cE)
![image](https://private-user-images.githubusercontent.com/97915774/333123595-1505b7d6-54f3-4eda-8d83-b2077c166515.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTk0MDYxMDAsIm5iZiI6MTcxOTQwNTgwMCwicGF0aCI6Ii85NzkxNTc3NC8zMzMxMjM1OTUtMTUwNWI3ZDYtNTRmMy00ZWRhLThkODMtYjIwNzdjMTY2NTE1LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MjYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjI2VDEyNDMyMFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTAwZjE3YmZkODY4MzljNTAxNzVkYWE2YmI3MGUyNDQwMWEzYWIxMDMwZjExMGUyYTQyODc2MzUzNzZiOWQwMjcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.pUWwoGDQwzDw0YgGSWEjj4m3q3t_2z30fGdFUKoWvw0)
![image](https://private-user-images.githubusercontent.com/97915774/333123658-f5cb6acb-82df-4746-b417-867d2d46128b.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTk0MDYxMDAsIm5iZiI6MTcxOTQwNTgwMCwicGF0aCI6Ii85NzkxNTc3NC8zMzMxMjM2NTgtZjVjYjZhY2ItODJkZi00NzQ2LWI0MTctODY3ZDJkNDYxMjhiLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MjYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjI2VDEyNDMyMFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTQ3ZGI3ZmRkYjJmY2NjZTNlOWQ4OTU4NWQxNGM2M2JmYmQ4ZmNiNWYwMzk1ZmUyYjkxMDkwOTVkODI2MmZmNmQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.rDdj6opfUmgweQT13Nv0TPqhnknvUS0K5KI-JCTAMjw)
We added a grade button in the manual grading assessment question page that utilizes large language models to autograde all problems, and update the scores and feedbacks.