Skip to content

feat(review): review_depth: lightweight annotation to skip preamble for low-risk steps#411

Merged
nhorton merged 4 commits into
mainfrom
fix/86-lightweight-review-depth
May 19, 2026
Merged

feat(review): review_depth: lightweight annotation to skip preamble for low-risk steps#411
nhorton merged 4 commits into
mainfrom
fix/86-lightweight-review-depth

Conversation

@sansari
Copy link
Copy Markdown
Contributor

@sansari sansari commented May 18, 2026

Summary

  • Adds review_depth: lightweight annotation to review blocks in job.yml step outputs and .deepreview rules
  • When set, the workflow's common_job_info preamble (## Job Context) is omitted from review instruction files — typically 300–800 tokens per review
  • Step inputs and review criteria are always included regardless of depth
  • Process-requirements reviews always use standard depth (never suppressed)

NOTE: This only adds the review_depth capability, it is utilized in a deepwork-frontend PR https://github.com/Unsupervisedcom/deepwork-frontend/pull/106

Implementation

File Change
parser.py ReviewBlock gains review_depth: str | None field
config.py ReviewRule and ReviewTask carry review_depth through the pipeline
matcher.py review_depth threaded from rule to task for all three strategies
quality_gate.py _build_preamble() conditionally omits Job Context when lightweight
job.schema.json review_depth enum ["lightweight"] added to review_block
deepreview_schema.json Same enum added for .deepreview rule review blocks; description clarifies this field is a no-op in .deepreview rules (no common_job_info to suppress there)

Test plan

  • New parser tests: test_from_dict_with_review_depth_lightweight, test_from_dict_review_depth_defaults_to_none
  • New quality gate tests: TestReviewDepthLightweight (3 tests) verifying Job Context is suppressed, standard depth includes it, and step inputs are always present
  • test_lightweight_review_depth_omits_common_job_info on build_string_output_review_tasks (string output path)
  • New matcher tests: test_all_changed_files_strategy_with_subsequent_rule, defensive path tests for _find_unchanged_matching_files
  • 1299 passing, coverage at 98.07%

Verification

To confirm omitting ## Job Context doesn't reduce review quality, two deepwork:reviewer agents were run in parallel against the same real analysis report — one with the full preamble (standard), one without (lightweight):

  • Standard: instructions included ## Job Context (business background for salman.io: nature writing pivot, newsletter open rate gap, digital garden CTR advantage)
  • Lightweight: same 5 review criteria, no job context block

Both returned PASS on all five criteria with the same findings and the same supporting evidence. The lightweight reviewer was marginally more critical on criterion 5 (missed opportunities), surfacing a minor gap the standard reviewer did not. This makes sense for reports that are self-contained: when the artifact under review carries its own context, the preamble is redundant. For reviews of opaque artifacts (raw code, schema files) that don't carry product context, standard depth remains appropriate.

Closes Unsupervisedcom/deepwork-frontend#86

🤖 Generated with Claude Code

sansari and others added 2 commits May 18, 2026 15:59
…low-risk steps

Adds a `review_depth: lightweight` annotation to review blocks in job.yml
step outputs and .deepreview rules. When set, the workflow's common_job_info
preamble (## Job Context) is omitted from review instruction files, reducing
token overhead for trivial or reversible intermediate steps. Step inputs and
review criteria are always included regardless of depth.

- Parser: ReviewBlock gains review_depth field
- Config: ReviewRule and ReviewTask carry review_depth through the pipeline
- Matcher: review_depth threaded from rule to task for all three strategies
- QualityGate: _build_preamble() conditionally omits Job Context when lightweight
- Schemas: review_depth enum ["lightweight"] added to job.schema.json and deepreview_schema.json
- Tests: 1299 passing, coverage at 98.07%

Closes #86

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@sansari sansari self-assigned this May 19, 2026
@sansari sansari requested a review from Copilot May 19, 2026 00:17
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an optional review_depth: lightweight annotation to review blocks in both job.yml step outputs and .deepreview rules. When set, the workflow's common_job_info preamble (## Job Context) is omitted from generated review instruction files to reduce token usage for low-risk/reversible intermediate steps. Step inputs and review criteria are always included, and process-requirements (PQA) reviews always use standard depth.

Changes:

  • Plumbs a new review_depth: str | None field from parsers (ReviewBlock, ReviewRule) through the matcher (ReviewTask) and into _build_preamble, which now skips ## Job Context when the value is "lightweight".
  • Adds a review_depth enum (["lightweight"]) to both job.schema.json and deepreview_schema.json.
  • Adds parser, matcher, and quality-gate tests covering lightweight behaviour, depth defaulting, and step-input retention.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/deepwork/jobs/parser.py Adds review_depth field to ReviewBlock and populates it from input dict
src/deepwork/review/config.py Adds review_depth field to ReviewRule and ReviewTask; parses it from .deepreview data
src/deepwork/review/matcher.py Threads review_depth from rule to task in all three strategies
src/deepwork/jobs/mcp/quality_gate.py _build_preamble takes review_depth; suppresses ## Job Context for lightweight; PQA preamble always standard
src/deepwork/jobs/job.schema.json Adds review_depth enum to review block schema
src/deepwork/schemas/deepreview_schema.json Same enum added to .deepreview review block schema
CHANGELOG.md Records the new review_depth: lightweight feature
tests/unit/jobs/test_parser.py Tests for parsing review_depth and defaulting to None
tests/unit/jobs/mcp/test_quality_gate.py Tests lightweight suppression, standard inclusion, step-input retention, and related coverage
tests/unit/review/test_matcher.py Tests for glob path-relative skip cases and all_changed_files + subsequent rule

sansari and others added 2 commits May 18, 2026 17:19
…-output test

- deepreview_schema.json: note that review_depth has no effect in .deepreview
  rules since they are not associated with a workflow and receive no common_job_info
- test_quality_gate.py: add test for build_string_output_review_tasks with
  review_depth: lightweight to confirm common_job_info is suppressed on string outputs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@sansari sansari marked this pull request as ready for review May 19, 2026 00:57
@sansari sansari marked this pull request as draft May 19, 2026 00:57
@sansari sansari marked this pull request as ready for review May 19, 2026 01:25
@sansari sansari requested review from ncrmro and nhorton May 19, 2026 01:26
@nhorton nhorton added this pull request to the merge queue May 19, 2026
Merged via the queue into main with commit 3ca7453 May 19, 2026
5 checks passed
@nhorton nhorton deleted the fix/86-lightweight-review-depth branch May 19, 2026 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants