Show and tell: probe-driven code review with a JSON gate

[kyy-logs]

Sharing a code-review skill for anyone comparing review-focused agent skills.

Most review skills I've seen are a single checklist. This one is probe-driven instead:

• behavior
• tests
• repository rules
• impact

Each probe runs independently and writes its own JSON. The human report is rendered from those records, and the merge gate is computed from JSON rather than from the model's summary. It is static-first and read-only.

It's beta. The interesting question for me is whether separating review into four probes actually reduces missed issues, or just adds overhead. Feedback on that, and on false positives, would be very useful.

Repo: https://github.com/kyy-logs/code-review-skill

[sickn33]

Thanks for sharing this. The separation into behavior, tests, repository rules and impact probes, with reports derived from JSON records, is interesting. We inspected fe2d81c02794188e1c57d792ce47365a230adb1a and ran the offline test suite before considering it for AAS.

We found a concrete portability issue: with the preinstalled macOS /bin/bash, 46 of 52 tests passed and six failed. The shell pipeline uses printf '%(%s)T' in collect-metrics / lib-metrics, which Bash 3.2 does not support. Please make this portable, or explicitly declare and check the required Bash version and document how users of supported platforms should invoke it. A test for the declared minimum shell would help prevent regression.

Please also address these integration points:

• Make the report/output location configurable and describe the writes accurately: source inspection is read-only, but the workflow creates reports, an event log and trend data under docs/work. Protect existing output from accidental collisions or overwrites.
• Make coverage thresholds follow project policy, with the current 85% value clearly identified as a configurable default.
• State the language coverage precisely. The default rule scanner selects Java, TS/TSX and Vue files and the rule pack includes Java/Spring-specific guidance; this should not read as equally validated coverage of every stack.
• Keep the distinction between consistent JSON/report records and an effective review. The passing offline tests do not establish bug-detection recall. A small reproducible comparison against a single review, recording missed defects, false positives and time/cost, would make the added value much easier to assess. If that has not been measured yet, say so; no paid evaluation is required for this feedback.

Please post the corrected revision and validation results here. If you want to propose catalog inclusion, identify the complete skill bundle and its license/support files so we can review the executable content through the appropriate maintainer path. These skill-level gates would not replace AAS's existing merge checks or maintainer review.

[kyy-logs]

Thanks for the detailed review. The revision you inspected, fe2d81c, predates the portability fixes; the corrected revision is now pushed.

Corrected revision

• main: 825ea5e (Merge branch 'codex/bilingual-skill-plan')
• codex/bilingual-skill-plan: 17f76c4

Bash portability

• Bash 3.2 is the declared minimum, with a runtime version check.
• Older Bash uses the date +%s timestamp fallback; the newer printf timestamp path is used only where supported.
• macOS users can invoke the scripts with /bin/bash.
• The CI job for macos-14 runs the shell-backed tests with /bin/bash and passes.

Validation

• Validate skills for main 825ea5e: all three jobs passed:
  • offline (macos-14, 3.11)
  • offline (ubuntu-22.04, 3.9)
  • offline (ubuntu-latest, 3.11)
  • Run: https://github.com/kyy-logs/code-review-skill/actions/runs/41403804140
• The same three jobs also passed for branch tip 17f76c4:
  • Run: https://github.com/kyy-logs/code-review-skill/actions/runs/41403799665
• I also ran the 85 offline tests on Windows with Git Bash 5.2.37; that is a local compatibility check, not a Bash 3.2 validation.

Output and writes

• The output root is configurable through .code-review/config.json (output_dir) or CODE_REVIEW_OUTPUT_DIR.
• Source inspection is read-only; the workflow writes reports, probe JSON, policy snapshots, event logs, and archive metrics under the output root.
• Initialization rejects previously used review identifiers; report rendering preserves content outside its marked regions.
• Concurrent writers for the same review are not supported; this is documented as a known limitation.

Thresholds and language scope

• 85% diff-line coverage is a configurable default (line_target), not a universal requirement. The branch target is separately configurable and advisory.
• The default pattern scan selects Java, TS/TSX, and Vue files. The bundled rules include Java/Spring-specific guidance; broader AI semantic inspection does not establish equal validation across languages or frameworks.

Review effectiveness

• I agree that consistent JSON, reports, and gates do not establish defect recall.
• We have run targeted x04/x06/x09 regression and x04 fixed-control evaluations, with hit/miss and token/tool/time records. These are single-case, single-run checks, not a controlled single-review baseline.
• I have not established a controlled comparison against a single review measuring missed defects, false positives, wall-clock time, and token cost. The existing results are snapshot-specific observations and do not support a general recall or cost claim.
• The README clarifications covering these limits are now included in 825ea5e and pushed.

Catalogue scope

• The executable review unit is the complete English skill package, or its Chinese counterpart, including SKILL.md, references, assets, and scripts.
• Supporting material includes the repository’s MIT license, installation documentation, installer sources, tests, and CI configuration.

I understand that inclusion should follow your normal maintainer review. These skill-level gates are internal checks and do not replace AAS merge checks or human approval. If you want a specific tag or archive for the exact review target, I can provide one.