-
Notifications
You must be signed in to change notification settings - Fork 0
Open
0 / 10 of 1 issue completedLabels
documentationImprovements or additions to documentationImprovements or additions to documentationenhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is needed
Description
-
Evaluation
One of the big problem in this space is that there is no public benchmark for what thorough reviews should look like. We should have a scalable way to collect this benchmark. -
Algorithm design
The current approach uses incremental summarization. It will have trouble for long-term dependency.
Test recursive language modeling for this task.
Turn the key idea into a claude skill
Integrate with https://github.com/ChicagoHAI/MechEvalAgent/ to have execution-grounded evaluation -
Interaction
This tool would be much more useful if the users can address the comments interactively.
Reactions are currently unavailable
Sub-issues
Metadata
Metadata
Assignees
Labels
documentationImprovements or additions to documentationImprovements or additions to documentationenhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is needed