feat: extend ADR detection with additional directory paths and CLAUDE.md partial credit

[jwm4]

Summary

• Smarter path matching (ADR check: support additional directory paths (specs/, docs/design/, etc.) #414): splits ADR detection into two name sets.
  `docs_target_names` (broader, used under `docs/`) covers `architecture/`,
  `design/`, `specs/`, and the existing `adr/` family. `root_target_names`
  (narrower, used at repo root) excludes `architecture/` and `design/` to
  avoid false positives from common top-level directories.

• CLAUDE.md/AGENTS.md partial credit (architecture_decisions: credit ADR summaries in CLAUDE.md/AGENTS.md as a compliance path #392): when no inline ADR directory
  exists, the assessor awards 60/100 if `CLAUDE.md` or `AGENTS.md` contains
  an architecture/decisions section heading or a link to an external ADR/RFC
  repo. Full credit still requires an inline directory.

• Docs: `docs/attributes.md` entry for `architecture_decisions` expanded
  with a scoring table, recognized path list, and partial credit explanation.

Tests

31 tests in `tests/unit/test_doc_scoring_precision.py`, including 18 new ones
covering extended path variants (including case variants and hidden `.adr/`),
CLAUDE.md/AGENTS.md section detection, external link detection, and
inline-directory-takes-priority behavior.

Closes #414, #392

🤖 Generated with Claude Code under the supervision of Bill Murdock

[coderabbitai]

Warning

Rate limit exceeded

@jwm4 has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 58 minutes and 34 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: e814ee5b-a7a2-4f1d-97b2-212d58b5ceb8

📥 Commits

Reviewing files that changed from the base of the PR and between f6c5197 and b37eea9.

📒 Files selected for processing (3)

• docs/attributes.md
• src/agentready/assessors/documentation.py
• tests/unit/test_doc_scoring_precision.py

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/adr-extended-paths-and-claude-partial-credit

✨ Simplify code

• Create PR with simplified code
• Commit simplified code in branch feat/adr-extended-paths-and-claude-partial-credit

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

📈 Test Coverage Report

Branch	Coverage
This PR	73.3%
Main	72.9%
Diff	✅ +0.4%

---

Coverage calculated from unit tests only

[jwm4]

🤖 AgentReady Code Review

PR Status: 4 issues found (0 🔴 Critical, 2 🟡 Major, 2 🔵 Minor)
Score Impact: Partial-credit path inflates architecture_decisions by 60 points for repos with structural ## Architecture headings in CLAUDE.md

---

🟡 Major Issues - Manual Review Required

1. Partial-credit regex triggers on structural headings, not just ADR summaries

Attribute: architecture_decisions (Tier 3) - Assessment accuracy
Confidence: 95%
Location: documentation.py ~L729-731

Validated against two real repos assessed on the PR branch vs main:

Repo	CLAUDE.md content	main score	PR #419 score
mihai-dinculescu/tapo	## Architecture describing workspace members	0/fail	60/fail
authorizerdev/authorizer	## Architecture (Quick Reference) describing file layout	0/fail	60/fail

Neither repo has any ADRs. The false positive is triggered solely by the section_header regex matching ## Architecture structural headings. The ## Architecture convention is extremely common in CLAUDE.md files (it is the standard heading for describing codebase layout to agents), so this false positive will appear at scale.

The root issue is in _check_agent_file_adr_summary:

section_header = re.compile(
    r"^#{1,3}\s.*(architecture|decision|adr|rfc|design)",
    re.IGNORECASE | re.MULTILINE,
)

Suggested fix: require a more specific phrase rather than the bare word "architecture":

section_header = re.compile(
    r"^#{1,3}\s.*(architecture.decision|architectural.decision|adr|decision.record|rfc)",
    re.IGNORECASE | re.MULTILINE,
)

Or: require both section heading AND an external link before awarding partial credit (removing the has_section or ... short-circuit).

2. external_link pattern matches generic architecture reference pages

Attribute: architecture_decisions (Tier 3) - Assessment accuracy
Confidence: 85%
Location: documentation.py ~L733-735

external_link = re.compile(r"https?://\S*(adr|rfc|decision|architecture)\S*", re.IGNORECASE)

This matches any URL containing "architecture" - including https://aws.amazon.com/architecture/ or https://docs.microsoft.com/azure/architecture. A CLAUDE.md with an architecture keyword and a link to AWS reference docs would satisfy adr_keywords AND has_link and receive 60 points. Consider restricting to repository hosting URLs or requiring a more specific path pattern.

---

🔵 Minor Issues

3. Error message omits docs/design/ from "checked" path list

Confidence: 90%
Location: documentation.py ~L643

docs_target_names includes "design" so docs/design/ is scanned, but the error message says "checked docs/adr/, docs/architecture/, docs/specs/, specs/, and variants" - omitting docs/design/.

4. Missing test: keyword present, no section header, no link should score 0

Confidence: 85%
Location: test_doc_scoring_precision.py TestClaudeMdPartialCredit

The existing zero-score test uses content with no keyword matches at all, so it does not verify that the and has_link conjunction actually guards the gate. A test with "We follow an architecture-first approach.\n" (keyword present, no heading, no link) would close this gap.

---

Summary

• Real-world validation: 2 repos tested (mihai-dinculescu/tapo, authorizerdev/authorizer), both confirmed false positives for finding [P0] Create Automated Demo #1
• Rebase status: clean - no conflicts with main for documentation.py
• CI: all checks green; coverage 72.8% (-0.2% vs main, pre-existing project-wide gap)
• AgentReady Assessment: Run agentready assess . after fixes to verify score

---

Reviewed with Claude Code under the supervision of Bill Murdock.

[jwm4]

Thanks for the review. Here's how each issue was addressed:

#1 (Major) — Partial-credit gate too broad: Fixed. Changed the condition
from has_section OR (adr_keywords AND has_link) to has_section AND has_link.
A section heading alone is too common a false positive (confirmed by the tapo
and authorizer examples). Both conditions are now required.

#2 (Major) — External link regex matches generic URLs: Left as-is. With the
AND gate now in place, a generic architecture URL only triggers partial credit
when paired with a matching section heading — a combination unlikely to appear
without intentional ADR documentation. The compound condition does the heavy
lifting here.

#3 (Minor) — Error message missing docs/design/: Fixed.

#4 (Minor) — Missing keyword-only test: Fixed. Added
test_keyword_only_scores_zero and also split the existing section-only and
link-only cases into dedicated tests (test_section_only_scores_zero,
test_link_only_scores_zero) so all boundary conditions are explicitly covered.

---

Replied by Claude Code under the supervision of Bill Murdock.

[jwm4]

Re-ran the two false-positive repos against the updated branch (b37eea9):

Repo	main	PR (original)	PR (fixed)
mihai-dinculescu/tapo	0/fail	60/fail	0/fail
authorizerdev/authorizer	0/fail	60/fail	0/fail

Both false positives are resolved. The AND gate (has_section AND has_link) correctly requires both a matching section heading and an external link before awarding partial credit. Finding #1 is confirmed fixed.

Reviewed with Claude Code under the supervision of Bill Murdock.

[github-actions]

🎉 This PR is included in version 2.37.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀