Reorganize PR testing docs into .claude/agents/ #2112

justin808 · 2025-11-23T22:20:38Z

Summary

Reorganizes the PR testing documentation into a clearer directory structure, following the established pattern of .claude/commands/.

Changes

New structure:

.claude/
├── agents/              # Agent definitions (NEW)
│   ├── pr-testing-agent.md
│   └── pr-testing-guide.md
├── commands/            # Slash commands
└── docs/               # Supporting documentation
    ├── analysis/
    ├── testing-build-scripts.md
    ├── master-health-monitoring.md
    └── ...

What changed:

Created .claude/agents/ directory
Moved pr-testing-agent.md and pr-testing-guide.md from docs/ to agents/
Updated all cross-references in both files (now point to ../docs/ for supporting docs)
Updated prompt examples to use new path (.claude/agents/pr-testing-agent.md)

Benefits

Clear separation: Agents vs commands vs supporting documentation
Consistent structure: Follows the .claude/commands/ pattern
Easy to find: Agent definitions have their own directory
Future-proof: Ready for additional agents
No functionality changes: Only file moves and reference updates

Pull Request checklist

Add/update test to cover these changes - N/A, documentation reorganization
Update documentation - Updated all cross-references
Update CHANGELOG file - N/A, documentation reorganization

Summary by CodeRabbit

Documentation
- Updated PR Testing Agent and PR Testing Guide: fixed several relative links and references to point to the new documentation locations and corrected prompt/example references. No functional changes.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

coderabbitai · 2025-11-23T22:20:48Z

Walkthrough

Update: two Markdown docs under .claude/agents had internal relative links changed to point to new locations under ../docs/ (testing-build-scripts.md and master-health-monitoring.md). No functional code changes.

Changes

Cohort / File(s)	Summary
Agent docs (link updates) `.claude/agents/pr-testing-agent.md`, `.claude/agents/pr-testing-guide.md`	Updated multiple relative links and inline references to use `../docs/testing-build-scripts.md` and `../docs/master-health-monitoring.md`; also adjusted internal agent doc reference paths. Changes are purely documentation/link corrections.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Review primarily for correct relative paths and no broken links.
Check both files for consistent references and any remaining old paths.

Possibly related PRs

Add PR Testing Agent documentation #2109 — related doc addition/relocation of pr-testing-agent content that this PR now points to.
Break CI circular dependency with non-docs change #2065 — introduced/renamed testing-build-scripts.md and master-health-monitoring.md under docs/, which these links reference.

Poem

🐇 I hopped through links and fixed the trail,
Moved breadcrumbs so docs won't fail.
A tidy path, short and neat,
Now readers find what they should meet.
Cheers from this rabbit, light on feet.

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: reorganizing PR testing documentation by moving files into a new .claude/agents/ directory.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch jg/pr-testing-agent

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 22d9a0e and 9480326.

📒 Files selected for processing (2)

.claude/agents/pr-testing-agent.md (3 hunks)
.claude/agents/pr-testing-guide.md (4 hunks)

✅ Files skipped from review due to trivial changes (1)

.claude/agents/pr-testing-guide.md

🚧 Files skipped from review as they are similar to previous changes (1)

.claude/agents/pr-testing-agent.md

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

GitHub Check: claude-review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

claude · 2025-11-23T22:23:27Z

Code Review - PR #2112

Summary

This PR reorganizes PR testing documentation from .claude/docs/ to a new .claude/agents/ directory. The reorganization follows a logical structure and improves discoverability.

✅ What Looks Good

Clear organizational structure: Creating a dedicated .claude/agents/ directory makes sense and follows the pattern established by .claude/commands/
Complete cross-reference updates: All internal references in the new files correctly use relative paths (pr-testing-guide.md → pr-testing-agent.md, references to ../docs/, references to ../../)
Documentation quality: The content itself is comprehensive and well-structured
No functional changes: As stated, this is purely organizational

⚠️ Issues Found

CRITICAL: Old files not deleted

The old files still exist in .claude/docs/:

.claude/docs/pr-testing-agent.md
.claude/docs/pr-testing-guide.md

Problem: This creates duplicate documentation and will lead to confusion:

Which version is authoritative?
Future updates might only update one location
Search results will return both versions
Wastes repository space

Required action: Delete the old files from .claude/docs/:

git rm .claude/docs/pr-testing-agent.md .claude/docs/pr-testing-guide.md
git commit -m "Remove old PR testing docs from .claude/docs/"

🎯 Recommendations

Before merging:

Delete old files (REQUIRED):
git rm .claude/docs/pr-testing-agent.md .claude/docs/pr-testing-guide.md
Verify no other references (optional but recommended):
grep -r "\.claude/docs/pr-testing" --exclude-dir=node_modules --exclude-dir=.git .
Update PR description (optional): Mention that old files are deleted, not just moved

📊 Overall Assessment

Code Quality: ✅ Excellent - Well-organized, clear structure
Completeness: ⚠️ Incomplete - Old files need deletion
Documentation: ✅ Good - Content is comprehensive
Testing: N/A - Documentation only
Security: ✅ No concerns

Recommendation: Approve with changes required - Delete old files before merging.

Great work on improving the documentation structure! Just need to clean up the old files and this will be ready to merge. 🚀

coderabbitai

Actionable comments posted: 0

🧹 Nitpick comments (4)

.claude/agents/pr-testing-agent.md (2)
155-161: Add language specification to fenced code block. Line 155 begins a fenced code block without specifying the language, which reduces syntax highlighting clarity.

Apply this diff to add the language identifier:
-```
+```bash
❌ CANNOT test build scripts in isolated workspace
⚠️  MANUAL TESTING REQUIRED before merge:
354-364: Add language specification to fenced code block. Line 354 has a code fence without language specification. This block contains status message template text.

Apply this diff to add the language identifier:
-```
+```markdown
⚠️  UNTESTED - Requires environment not available in Conductor workspace
.claude/agents/pr-testing-guide.md (2)
56-56: Add language specifications to fenced code blocks. Multiple code blocks throughout the guide lack language identifiers, which reduces syntax highlighting and readability. Most should be bash (for shell examples), markdown (for template examples), or left unspecified if deliberately generic.

Representative examples of fixes needed:
-```
+"```bash
"I changed lib/react_on_rails/helper.rb. According to the PR Testing Agent,
and
-```
+"```markdown
"Based on PR Testing Agent Section 1, you need:
Also applies to: 70-70, 82-82, 92-92, 101-101, 110-110, 119-119, 130-130, 160-160, 191-191, 204-204, 229-229, 264-264, 288-288, 295-295, 302-302, 309-309

555-555: Add language specifications to remaining fenced code blocks. Lines 555, 599, 628, and 667 in the "Communicating Test Status" and "Real-World Scenarios" sections have code blocks without language specifications. These should be bash for shell commands or markdown for message templates.

Examples:
-```
+"```markdown
Fix webpack CSS modules configuration for Shakapacker 9
and
-```
+"```bash
$ bundle exec rspec spec/react_on_rails/helper_spec.rb
Also applies to: 599-599, 628-628, 667-667

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 20d6969 and 22d9a0e.

📒 Files selected for processing (2)

.claude/agents/pr-testing-agent.md (1 hunks)
.claude/agents/pr-testing-guide.md (1 hunks)

🧰 Additional context used

🪛 LanguageTool

.claude/agents/pr-testing-guide.md

[style] ~32-~32: Try using a synonym here to strengthen your wording.
Context: ...h pr create`) - Responding to PR review comments about testing - CI failures occur and n...

(COMMENT_REMARK)

[grammar] ~494-~494: Ensure spelling is correct
Context: ...tion tests: rake run_rspec:dummy ### Environment Limitations **Mark any testing blocked...

(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)

🪛 markdownlint-cli2 (0.18.1)

.claude/agents/pr-testing-guide.md

56-56: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

70-70: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

82-82: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

92-92: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

101-101: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

110-110: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

119-119: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

130-130: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

160-160: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

191-191: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

204-204: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

229-229: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

264-264: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

288-288: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

295-295: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

302-302: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

309-309: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

555-555: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

599-599: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

628-628: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

667-667: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

.claude/agents/pr-testing-agent.md

108-108: Emphasis used instead of a heading

(MD036, no-emphasis-as-heading)

155-155: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

354-354: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

GitHub Check: claude-review

🔇 Additional comments (13)

.claude/agents/pr-testing-agent.md (5)

1-17: Clear, well-organized PR testing agent definition. The structure effectively separates role/principles from detailed requirements, with comprehensive cross-references to supporting documentation. The Quick Reference section correctly uses relative paths from the new .claude/agents/ location (e.g., ../docs/ for supporting docs, ../../ for repo-root files).

18-46: Excellent articulation of skeptical stance and communication standards. The Agent Behavior section clearly defines the default mindset and provides concrete DO/DON'T examples that guide implementation. This will help Claude Code apply the agent correctly and communicates expectations clearly to developers.

106-162: Build Configuration Changes section provides critical guidance on silent failures. The MANDATORY manual testing checklist (steps 1-6) and explicit reference to real examples in testing-build-scripts.md effectively communicate why build testing is non-negotiable. The Conductor workspace limitation call-out (lines 153-161) is particularly valuable for developers working in isolated environments.

331-385: Environment limitations section is pragmatic and helpful. The distinction between what CAN and CANNOT be tested in Conductor workspace (lines 335-351) sets realistic expectations, and the Ruby 3.4 + OpenSSL workaround (lines 366-384) shows awareness of real pain points developers face. The UNTESTED label template is actionable.

386-470: Seven success criteria effectively operationalize the agent's core principles. Criterion #4 (Testing Documentation) with the markdown template (lines 423-442) is particularly valuable—it makes the skeptical, transparent approach concrete. Criterion #5 (Clear Communication) reinforces UNTESTED labeling patterns defined earlier, creating a consistent vocabulary for testing status throughout the PR lifecycle.

.claude/agents/pr-testing-guide.md (8)

1-24: Excellent companion guide with clear practical orientation. The structure effectively supports the PR Testing Agent by showing when and how to use it with Claude Code. Cross-references to the agent definition and supporting docs are correct and help users navigate the documentation ecosystem.

45-62: Excellent invocation examples for Claude Code integration. The prompts at lines 54-62 show the specific pattern: reference the agent document directly (.claude/agents/pr-testing-agent.md) and request specific sections. This pattern will help Claude Code apply the agent consistently across different prompts.

64-124: Comprehensive workflow examples with realistic Claude Code interactions. The progression from Quick Start → Common Workflows → Specific Section Referencing establishes a clear learning path. Each workflow includes both the prompt pattern and relevant section reference, making it easy for developers to replicate successful invocations.

145-282: Complete lifecycle example effectively demonstrates agent usage in context. The walk-through from code change → testing guidance → execution → validation → documentation → CI investigation shows how developers move through the PR lifecycle while consulting the agent. The realistic Claude Code responses (e.g., lines 165-189, 209-221) help set expectations for what guidance developers can expect.

421-508: Pre-Merge Testing Checklist effectively operationalizes agent requirements. The organization by file type directly parallels the PR Testing Agent's section structure, making it easy for developers to find relevant items. The MANDATORY emphasis on build config testing (line 455-458) and environment limitations call-out (lines 494-508) reinforce the agent's core principles about testing comprehensiveness and transparency.

510-594: Communication templates provide clear structure for testing status updates. The PR comment template (lines 514-549) uses sections (Verified Locally, Requires Manual Verification, Cannot Test, CI Status) and visual indicators (✅, ⚠️, ❌) to make testing status scannable. The commit message template (lines 551-570) shows how to document testing in git history. Both effectively support the agent's emphasis on explicit, honest communication about what was/wasn't tested.

572-668: Real-world scenarios effectively teach the skeptical approach through contrast. The three scenarios (RSpec failures, dependency updates, CI reproduction) show the progression from insufficient → required documentation. Scenario 2's emphasis on mandatory build testing (clean install, build, prepack, yalc publish) reinforces the Build Configuration Changes guidance from the agent. Scenario 3's acknowledgment of environment limitations and proposed testing steps models how to handle "can't reproduce" situations transparently.

344-419: Integration guidance clearly positions the agent within the broader development workflow. The explanation of how it complements (but doesn't replace) the code-reviewer agent and CI checks prevents role confusion. The integration points (pre-commit, pre-push, during CI, pre-merge) create a natural progression, and the workflow example shows how to consult the agent across the PR lifecycle.

Creates a clearer structure following the /commands pattern: .claude/ ├── agents/ # Agent definitions (NEW) │ ├── pr-testing-agent.md │ └── pr-testing-guide.md ├── commands/ # Slash commands └── docs/ # Supporting documentation Benefits: - Clear separation: agents vs commands vs supporting docs - Consistent with /commands structure - Easy to find agent definitions - Future-proof for more agents Changes: - Created .claude/agents/ directory - Moved pr-testing-agent.md from docs/ to agents/ - Moved pr-testing-guide.md from docs/ to agents/ - Updated all cross-references to use ../docs/ paths - Updated prompt examples to use .claude/agents/ path No duplicate files remain - clean move from docs/ to agents/ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

claude · 2025-11-23T22:37:31Z

Code Review - PR #2112

Summary

This PR reorganizes the PR testing documentation by creating a dedicated .claude/agents/ directory, improving the clarity and discoverability of agent-related documentation. The changes are purely organizational with updated cross-references.

✅ Strengths

1. Clear organizational structure

Excellent separation of concerns: agents vs. commands vs. supporting docs
Follows the established pattern of .claude/commands/
Makes agent definitions easy to locate and maintain

2. Comprehensive cross-reference updates

All relative paths correctly updated from same-directory to ../docs/
Example prompts updated to reflect new paths (.claude/agents/pr-testing-agent.md)
Both files consistently updated

3. Low-risk change

File moves with no functionality changes
All referenced files verified to exist at expected locations
Markdown link checker passing

4. Future-proof design

Ready for additional agents to be added to .claude/agents/
Logical grouping makes documentation more discoverable

✅ Verified

Cross-reference accuracy:

✓ ../docs/testing-build-scripts.md - exists
✓ ../docs/master-health-monitoring.md - exists
✓ ../../SWITCHING_CI_CONFIGS.md - exists
✓ ../../spec/dummy/TESTING_LOCALLY.md - exists
✓ ../../CLAUDE.md - exists

Updated paths:

✓ Prompt examples updated to .claude/agents/pr-testing-agent.md
✓ All 7 occurrences of ../docs/ references correct
✓ CI passing (markdown-link-check SUCCESS)

📋 Review Checklist

✅ Code quality: N/A (documentation only)
✅ Best practices: Follows established .claude/ organizational pattern
✅ Potential bugs: None - file moves with correct reference updates
✅ Performance: N/A
✅ Security: N/A
✅ Test coverage: N/A (per PR description, documentation reorganization)

🎯 Recommendation

LGTM - Ready to merge

This is a clean, well-executed documentation reorganization that:

Improves discoverability of agent documentation
Maintains all cross-references correctly
Follows project conventions
Poses zero risk to functionality

No changes needed.

📝 Minor Note (optional enhancement)

Consider adding a brief README.md in .claude/agents/ to document:

Purpose of the agents directory
How agents differ from commands
How to create new agents
Index of available agents

This is not required for this PR but could help future contributors. Could be addressed in a follow-up if desired.

Nice work on maintaining clean documentation structure! 🚀

The script/convert tool is used by CI 'minimum' dependency tests to downgrade to minimum supported versions (Ruby 3.2, Node 20, Shakapacker 8.2.0, React 18). After the monorepo restructure (PR #2112), the script was failing because it was looking for spec/dummy/package.json at the old root location, but these files are now under react_on_rails/spec/dummy/ and react_on_rails_pro/spec/dummy/. Updated all file path references: - spec/dummy/package.json → react_on_rails/spec/dummy/package.json - spec/dummy/config/shakapacker.yml → react_on_rails/spec/dummy/config/shakapacker.yml Fixes CI failure: https://github.com/shakacode/react_on_rails/actions/runs/19624872033/job/56191736323 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

### Summary Reorganizes the PR testing documentation into a clearer directory structure, following the established pattern of `.claude/commands/`. ### Changes **New structure:** ``` .claude/ ├── agents/ # Agent definitions (NEW) │ ├── pr-testing-agent.md │ └── pr-testing-guide.md ├── commands/ # Slash commands └── docs/ # Supporting documentation ├── analysis/ ├── testing-build-scripts.md ├── master-health-monitoring.md └── ... ``` **What changed:** - Created `.claude/agents/` directory - Moved `pr-testing-agent.md` and `pr-testing-guide.md` from `docs/` to `agents/` - Updated all cross-references in both files (now point to `../docs/` for supporting docs) - Updated prompt examples to use new path (`.claude/agents/pr-testing-agent.md`) ### Benefits - **Clear separation**: Agents vs commands vs supporting documentation - **Consistent structure**: Follows the `.claude/commands/` pattern - **Easy to find**: Agent definitions have their own directory - **Future-proof**: Ready for additional agents - **No functionality changes**: Only file moves and reference updates ### Pull Request checklist - [x] Add/update test to cover these changes - N/A, documentation reorganization - [x] Update documentation - Updated all cross-references - [x] Update CHANGELOG file - N/A, documentation reorganization  ## Summary by CodeRabbit * **Documentation** * Updated PR Testing Agent and PR Testing Guide: fixed several relative links and references to point to the new documentation locations and corrected prompt/example references. No functional changes. <sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub>  Co-authored-by: Claude <noreply@anthropic.com>

coderabbitai bot added the review-needed label Nov 23, 2025

coderabbitai bot reviewed Nov 23, 2025

View reviewed changes

coderabbitai bot approved these changes Nov 23, 2025

View reviewed changes

justin808 force-pushed the jg/pr-testing-agent branch from 22d9a0e to 9480326 Compare November 23, 2025 22:34

justin808 merged commit 79e334c into master Nov 23, 2025
3 checks passed

justin808 deleted the jg/pr-testing-agent branch November 23, 2025 22:57

claude bot mentioned this pull request Nov 25, 2025

Migrate from Yarn Classic to pnpm #2121

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Reorganize PR testing docs into .claude/agents/ #2112

Reorganize PR testing docs into .claude/agents/ #2112

Uh oh!

justin808 commented Nov 23, 2025 •

edited by coderabbitai bot

Loading

Uh oh!

coderabbitai bot commented Nov 23, 2025 •

edited

Loading

Uh oh!

claude bot commented Nov 23, 2025

Uh oh!

coderabbitai bot left a comment

Uh oh!

claude bot commented Nov 23, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Reorganize PR testing docs into .claude/agents/ #2112

Reorganize PR testing docs into .claude/agents/ #2112

Uh oh!

Conversation

justin808 commented Nov 23, 2025 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Benefits

Pull Request checklist

Summary by CodeRabbit

Uh oh!

coderabbitai bot commented Nov 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Poem

Pre-merge checks and finishing touches

Uh oh!

claude bot commented Nov 23, 2025

Code Review - PR #2112

Summary

✅ What Looks Good

⚠️ Issues Found

🎯 Recommendations

📊 Overall Assessment

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

claude bot commented Nov 23, 2025

Code Review - PR #2112

Summary

✅ Strengths

✅ Verified

📋 Review Checklist

🎯 Recommendation

📝 Minor Note (optional enhancement)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

justin808 commented Nov 23, 2025 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Nov 23, 2025 •

edited

Loading