Open
Conversation
This skill distills code review feedback from GitHub PRs into reusable skills and repository guidelines. It analyzes review comments, filters out noise, identifies recurring patterns, and generates: - Repository-specific skills for domain patterns - AGENTS.md updates for general conventions Fixes #86 Co-authored-by: openhands <openhands@all-hands.dev>
all-hands-bot
left a comment
all-hands-bot
left a comment
There was a problem hiding this comment.
🟡 Acceptable - Interesting concept, but the execution needs refinement. The workflow is overly complex and lacks validation. See inline comments for specifics.
Contributor
Author
|
@OpenHands read all the review comments and reflect them |
|
I'm on it! neubig can track my progress at all-hands.dev |
- Add Error Handling section documenting edge cases (few PRs, no patterns, token access issues, gh CLI unavailable) - Remove manual git commands, use create_pr tool instead - Replace Configuration Options table with simple defaults statement (30 days lookback) Co-authored-by: openhands <openhands@all-hands.dev>
Summary of Work CompletedI read all review comments on PR #87 and implemented the changes requested by @neubig based on his responses to the code review bot's feedback. Changes Made to
|
| Review Feedback | Action Taken |
|---|---|
| Add error handling for edge cases | ✅ Added "Error Handling" section covering: few PRs, no patterns, token access issues, gh CLI unavailable |
| Remove manual git commands, use create_pr tool | ✅ Simplified Step 6 from ~15 lines of git commands to a single instruction to use create_pr |
| Remove Configuration Options table | ✅ Replaced with simple "Defaults" section stating 30-day lookback |
| Test skill on real repository with evidence | ✅ Ran workflow on OpenHands/software-agent-sdk, created evidence PR |
Evidence Generated:
- Test PR: DRAFT: Add learnings from code review analysis software-agent-sdk#2280
- Analyzed 15 merged PRs, 25+ review comments
- Identified 4 recurring patterns → generated skill + AGENTS.md updates
Commits Pushed:
d80dd04- "Address review feedback on learn-from-code-review skill"- PR description updated to include Evidence section
All requested changes have been implemented, pushed to the branch, and the PR has been updated. The changes are concise and directly address only the feedback items that @neubig agreed should be fixed.
Focus generated skills on actionable guidelines rather than implementation details of how they were created. Co-authored-by: openhands <openhands@all-hands.dev>
AGENTS.md typically already contains general coding guidelines, so generated output should focus on creating focused skill files. Co-authored-by: openhands <openhands@all-hands.dev>
Make it explicit that producing no skills is fine when the codebase already has strong conventions or comments don't cluster into recurring themes. Co-authored-by: openhands <openhands@all-hands.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds a new skill that distills code review feedback from GitHub PRs into reusable skills and repository guidelines.
Closes #86
What this skill does
The
learn-from-code-reviewskill analyzes PR review comments from a repository and extracts recurring patterns of feedback. These patterns are transformed into:.openhands/skills/) for domain-specific patterns (e.g., database queries, API design)Workflow
Trigger phrases
/learn-from-reviewsFiles changed
skills/learn-from-code-review/SKILL.md- Main skill definitionskills/learn-from-code-review/README.md- Documentationmarketplaces/default.json- Added skill to marketplaceTesting
test_skills_have_readme.py)Evidence
The skill was tested on the
OpenHands/software-agent-sdkrepository. The workflow analyzed 15 merged PRs from the past 30 days, processed 25+ review comments, and identified 4 recurring patterns.Test PR: OpenHands/software-agent-sdk#2280
The test generated:
.openhands/skills/eval-risk-assessment/SKILL.mdAGENTS.mdwith a Code Review Learnings sectionRelated
code-review,github-pr-review,skill-creator,agent-memory