Implement the ADR 002 deterministic-tiered review route (7.1-7.4, 7.6.1)#6
Conversation
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Sorry @leynos, your pull request is larger than the review limit of 150000 diff characters
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (7)
💤 Files with no reviewable changes (3)
Summary
Testing and follow-up
WalkthroughDakar now runs deterministic-flex-v1 reviews through CLI preparation, bounded Luna Flex finder packs, one Terra Flex audit, deterministic rendering, usage telemetry, budget admission, CLI-owned history recording, and expanded workflow, harness, documentation, and validation support. ChangesDakar Flex review
Sequence Diagram(s)sequenceDiagram
participant Operator
participant DakarCLI
participant ReviewState
participant ODW
participant LunaFlex
participant TerraFlex
Operator->>DakarCLI: Start branch review
DakarCLI->>ReviewState: Prepare range and state
ReviewState-->>DakarCLI: Return prepared review or skip
DakarCLI->>ODW: Run with prepared and tuning arguments
ODW->>LunaFlex: Dispatch admitted finder packs
LunaFlex-->>ODW: Return candidate findings
ODW->>TerraFlex: Submit compacted issue set
TerraFlex-->>ODW: Return audit verdicts
ODW-->>DakarCLI: Return deterministic report and ledger
DakarCLI->>ReviewState: Append completed review
Possibly related PRs
Suggested labels: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 16❌ Failed checks (1 warning, 15 inconclusive)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Sorry @leynos, your pull request is larger than the review limit of 150000 diff characters
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
|
@coderabbitai resume |
✅ Action performedReviews resumed. |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
|
@coderabbitai review |
✅ Action performedReview finished.
|
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
|
@coderabbitai resume |
✅ Action performedReviews resumed. |
|
@coderabbitai Have the following now been resolved? src/workflows/dakar-review/main.ts (1)
🤖 Detailed instructionsVerify each finding against current code. Fix only still-valid issues, skip the In |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '== Source symbols =='
ast-grep outline src/workflows/dakar-review/main.ts --items all --type function,const --match 'workflowMain|recordInput|ledger|REVIEW_MODELS'
echo '== Source provenance and ledger context =='
rg -n -C 12 'recordInput|REVIEW_MODELS|ledger\.push|LedgerEntry|attempts|completed|success' \
src/workflows/dakar-review/main.ts
echo '== Generated artefact provenance context =='
rg -n -C 12 'recordInput|REVIEW_MODELS|ledger\.push|estimatedWorstCaseUsd|attempts' \
workflows/dakar-review.js
echo '== Generation/freshness contract =='
rg -n -C 4 'workflow-freshness|build-workflow|dakar-review\.js' \
AGENTS.md Makefile scripts/build-workflow.mjs
echo '== PR-local state =='
git status --short
git diff --checkLength of output: 1143 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '== Source function map =='
ast-grep outline src/workflows/dakar-review/main.ts --items all --type function
echo '== Source provenance and ledger context =='
rg -n -C 14 'recordInput|REVIEW_MODELS|ledger\.push|estimatedWorstCaseUsd|attempts|models:' \
src/workflows/dakar-review/main.ts
echo '== Generated artefact provenance context =='
rg -n -C 14 'recordInput|REVIEW_MODELS|ledger\.push|estimatedWorstCaseUsd|attempts|models:' \
workflows/dakar-review.js
echo '== Generation/freshness contract =='
rg -n -C 4 'workflow-freshness|build-workflow|dakar-review\.js' \
AGENTS.md Makefile scripts/build-workflow.mjs
echo '== PR-local state =='
git status --short
git diff --checkLength of output: 48777
|
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
@coderabbitai review |
✅ Action performedReview finished.
|
✅ Action performedReview finished.
|
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Add the canonical docs/contents.md index and docs/repository-layout.md per the documentation style guide, and land ADR 002 on the deterministic tiered review-cost decision. Correct the ADR 002 title, which incorrectly duplicated ADR 001's number. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lead the migration guidance with the implemented pi adapters and name the hook that injects the Flex service tier. Retain the Codex CLI limitation and direct readers to the existing adapter-contract amendment.
Parse and validate the supported YAML subset before ODW starts, execute normalized custom gates from trusted policy, and slice path guidance to the finder packs whose evidence it governs. Keep unsupported keys auditable but inert, preserve configuration precedence and fail-closed accounting, and regenerate the workflow from its authored TypeScript sources.
Distinguish the CLI operations surrounding `odw run` from model-free report rendering inside the compiled workflow, and name the authored function and SARIF renderer that own that path.
|
@coderabbitai Have the following now been resolved? tests/workflow-orchestration.test.mjs (1)
🤖 Detailed instructionsVerify each finding against current code. Fix only still-valid issues, skip the In |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '== Relevant files =='
fd -a -t f . tests | rg '(^|/)(workflow-orchestration\.test\.mjs|workflow-rendering\.test\.mjs|run-workflow\.mjs)$' || true
echo '== Helper and test outlines =='
for file in \
tests/helpers/run-workflow.mjs \
tests/workflow-orchestration.test.mjs \
tests/workflow-rendering.test.mjs
do
if [ -f "$file" ]; then
echo "-- $file"
ast-grep outline "$file" --items all
else
echo "-- MISSING: $file"
fi
done
echo '== Shared helper implementation and call sites =='
rg -n -C 8 \
'export .*runWorkflow|function runWorkflow|AsyncFunction|readFileSync|replace\(|swallowFixtureFailure|parallel|pipeline|nullParallelSlots|agentCalls|sleepDelays|knobs' \
tests/helpers/run-workflow.mjs \
tests/workflow-orchestration.test.mjs \
tests/workflow-rendering.test.mjs \
2>/dev/null || true
echo '== Focused file contents =='
for file in \
tests/helpers/run-workflow.mjs \
tests/workflow-orchestration.test.mjs \
tests/workflow-rendering.test.mjs
do
if [ -f "$file" ]; then
echo "-- $file"
sed -n '1,190p' "$file"
fi
done
echo '== Focused diff and integrity =='
git diff --check
git diff -- \
tests/helpers/run-workflow.mjs \
tests/workflow-orchestration.test.mjs \
tests/workflow-rendering.test.mjs
git status --shortLength of output: 70079
Treat focused runtime-test execution as unverified here because this inspection environment does not execute repository code. |
|
@coderabbitai Have the following now been resolved? dakar/bin/dakar-review.mjs Make the CLI timeout cover the default retry envelope In the telemetry path inspected here, the default Flex configuration reports a 2,020-second worst-case review, but the CLI still stops following after 900 seconds. Under Flex backpressure where retries approach their configured timeouts, |
This comment was marked as resolved.
This comment was marked as resolved.
|
@coderabbitai resume |
✅ Action performedReviews resumed. |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Fail closed when trusted-base configuration cannot be read and normalise Git revision paths across platforms. Include adapter overhead in the Terra reservation and apply audit policy guidance against every candidate path, while keeping the displayed path list bounded. Update focused fixtures and the generated workflow artefact for the corrected accounting contract.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
|
@coderabbitai review |
✅ Action performedReview finished.
|
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Keep the production prompt surface aligned with the live Luna-to-Terra route by removing the unused per-candidate verifier helper and its test-only contract.
Describe the command root using the current checkout rather than committing a user-specific absolute path.
Stream review stdout to its result file and scan usage telemetry while stderr is teed, avoiding duplicate unbounded in-memory captures. Bound partial telemetry lines and retain non-sensitive malformed diagnostics. Show the full audit retry-chain reservation in the user-guide dry-run example.
|
@coderabbitai Have the following failed checks now been resolved? If further work is required, please provide an AI agent prompt for the remaining work to be done to address these failures. Do not treat warnings as optional or aspirational. Where a change is out of scope for this PR, propose a GitHub issue unless one exists already. (Treat documentation and validation coverage as in scope). ❌ Failed checks (3 warnings, 1 inconclusive)
|
|
@coderabbitai Have the following now been resolved? src/workflows/dakar-review/prompts.ts (1)
🤖 Detailed instructionsVerify each finding against current code. Fix only still-valid issues, skip the In |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '== Current PR checks =='
gh pr checks 6 --repo leynos/dakar || true
echo '== Telemetry parser, aggregation, and stderr retention =='
ast-grep outline scripts/live-review-harness.mjs --items all --type function
rg -n -C 8 'extractUsageLines|sumUsage|summarize|priceReportedUsage|stderrChunks|stderr\.on|DAKAR-USAGE|malformedTelemetry' \
scripts/live-review-harness.mjs tests/live-harness.test.mjs
echo '== Audit compaction implementation and property invariant =='
ast-grep outline src/workflows/dakar-review/candidates.ts --items all --type function
rg -n -C 7 'compactForAudit|auditCandidates\.length|min\(.*cap|deduplic|overCap|partition' \
src/workflows/dakar-review/candidates.ts \
tests/workflow-compact-audit.test.mjs \
tests/workflow-properties.test.mjs
echo '== API-key probe and reported-usage documentation contracts =='
rg -n -i -C 5 'curl|OPENAI_API_KEY|authorization|header-file|header file|reportedUsage|inputTokens|cachedInputTokens|outputTokens|cacheRead|cacheWrite|ledgerTotalReportedUsd|reportedUsd' \
docs/execplans/api-key-support.md docs/dakar-review-design.md
echo '== Docstring gate configuration and documentation on touched helpers =='
ast-grep outline scripts/check-docstrings.mjs --items all --type function
rg -n -C 5 'defaultSourcePatterns|threshold|80|coverage|undocumented|sourceFiles' \
scripts/check-docstrings.mjs Makefile package.json
rg -n -C 2 '/\*\*|export (async )?function |^(async )?function ' \
scripts/live-review-harness.mjs bin/dakar-review.mjs scripts/odw-config.mjs \
adapters/pi/extensions/flex-tier.ts src/workflows/dakar-review
echo '== Existing observability issue =='
gh issue view 2 --repo leynos/dakar --json number,title,state,url
echo '== Relevant diff integrity =='
git diff --check
git status --shortLength of output: 215962 🏁 Script executed: #!/bin/bash
set -euo pipefail
make docstrings
echo '== Verify reported-usage documentation references =='
rg -n -i -C 2 \
'ledger.*reportedUsd|reportedUsd.*ledger|ledgerTotalReportedUsd|inputTokens|cachedInputTokens|outputTokens|reportedUsage|cacheRead|cacheWrite' \
docs/dakar-review-design.md docs/execplans/api-key-support.mdLength of output: 682
Run the following in CI or a development environment with the repository toolchain installed, then provide the output: make docstrings
rg -n -i -C 2 \
'ledger.*reportedUsd|reportedUsd.*ledger|ledgerTotalReportedUsd|inputTokens|cachedInputTokens|outputTokens|reportedUsage|cacheRead|cacheWrite' \
docs/dakar-review-design.md docs/execplans/api-key-support.md
node --test tests/live-harness.test.mjs
node --test tests/workflow-compact-audit.test.mjs tests/workflow-properties.test.mjs
make workflow-freshness
make check
git diff --checkCorrect the remaining Keep observability out of this PR; existing issue |
This comment was marked as resolved.
This comment was marked as resolved.
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '== Prompt module outline =='
ast-grep outline src/workflows/dakar-review/prompts.ts --items all
echo '== Prompt helper definitions and references =='
rg -n -C 5 '\bverificationPrompt\b|\bauditPrompt\b|\btaskPrompt\b' \
src/workflows/dakar-review/prompts.ts \
src/workflows/dakar-review/main.ts \
tests \
workflows/dakar-review.js || true
echo '== Focused prompt source =='
sed -n '1,260p' src/workflows/dakar-review/prompts.ts
echo '== Prompt-test imports and relevant assertions =='
sed -n '1,280p' tests/workflow-prompts.test.mjs
echo '== Focused changes and integrity =='
git diff --check
git diff -- src/workflows/dakar-review/prompts.ts tests/workflow-prompts.test.mjs workflows/dakar-review.js
git status --short
echo '== Generation contract =='
rg -n -C 3 'build-workflow|workflow-freshness|workflows/dakar-review\.js' \
Makefile AGENTS.md scripts/build-workflow.mjs 2>/dev/null || trueLength of output: 41286
Treat focused test execution as unverified in this inspection environment, which cannot run repository code. |
|
@coderabbitai resume |
✅ Action performedReviews resumed. |
|
@coderabbitai Have the following now been resolved? src/workflows/dakar-review/main.ts (1)
Proposed fix- inputTokens: TERRA_MAX_INPUT_TOKENS,
+ inputTokens: TERRA_MAX_INPUT_TOKENS + ADAPTER_OVERHEAD_TOKENS,🤖 Detailed instructionsVerify each finding against current code. Fix only still-valid issues, skip the In src/workflows/dakar-review/prompts.ts (1)
Proposed fix- policyGuidanceBlock(context.policy, listedPaths), '',
+ policyGuidanceBlock(context.policy, candidatePaths), '',Add a regression case with more than 40 candidate paths and a policy instruction matching a later path. 🤖 Detailed instructionsVerify each finding against current code. Fix only still-valid issues, skip the In |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '== Target source outlines =='
ast-grep outline src/workflows/dakar-review/main.ts --items all --type function
ast-grep outline src/workflows/dakar-review/prompts.ts --items all --type function
ast-grep outline tests/workflow-prompts.test.mjs --items all --type function
echo '== Terra reservation accounting and related pricing/admission uses =='
rg -n -C 10 \
'RESERVED_AUDIT_USD|TERRA_MAX_INPUT_TOKENS|ADAPTER_OVERHEAD_TOKENS|reservedAudit(Chain)?Usd|estimateWorstCaseUsd' \
src/workflows/dakar-review/main.ts \
tests/workflow-pricing.test.mjs \
tests/workflow-dry-run.test.mjs \
tests/workflow-orchestration.test.mjs \
workflows/dakar-review.js
echo '== Audit-path display and policy-guidance wiring =='
rg -n -C 12 \
'candidatePaths|listedPaths|policyGuidanceBlock|auditPrompt|slice\(0, 40\)|40-path|40 candidate|beyond.*40|beyond the.*cap' \
src/workflows/dakar-review/prompts.ts \
tests/workflow-prompts.test.mjs \
workflows/dakar-review.js
echo '== Focused source/test excerpts =='
sed -n '175,220p' src/workflows/dakar-review/main.ts
sed -n '105,170p' src/workflows/dakar-review/prompts.ts
sed -n '1,300p' tests/workflow-prompts.test.mjs
echo '== Authored/generated and focused diff integrity =='
git diff --check
git diff -- \
src/workflows/dakar-review/main.ts \
src/workflows/dakar-review/prompts.ts \
tests/workflow-prompts.test.mjs \
workflows/dakar-review.js
git status --shortLength of output: 99498
|
2 participants
Summary
This branch implements
ADR 002:
the deterministic-tiered review route with hard cost admission control,
live-validated end to end. Deterministic host code absorbs every derivable
phase (configuration, range preparation, report rendering, history
recording — four model calls deleted); the remaining judgement runs on two
OpenAI Flex lanes (
gpt-5.6-lunafinder packs, onegpt-5.6-terraissue-set audit replacing the thirty-call per-candidate verifier fan-out)
behind a reserve-first admission controller with a versioned pricing table
and per-call ledger. Every live corpus review, including a 101-file
oversize probe, came in below USD 0.25 reported (range 0.021-0.115); a
seeded-defect review scored 3/3 recall with zero false positives.
Roadmap tasks: (7.1.1), (7.1.2), (7.1.3), (7.2.1), (7.2.2), (7.2.3),
(7.3.1), (7.3.2), (7.3.3), (7.3.4), (7.4.1), (7.4.2), (7.6.1)
Execplan:
docs/execplans/api-key-support.md
— implemented through milestones M0-M7 plus the M8 gate-enablement
addendum, with the live ledger, hand assessments, decision log, and
retrospective maintained throughout. The companion change adopting this
route in df12-build is
leynos/df12-build#64.
Notable mid-flight decision: ADR 002 originally specified Codex CLI Flex
adapters, but the M0 probe proved Codex never transmits
service_tier(three versions capture-tested; upstream issues openai/codex#26604,
openai/codex#31562, and openai/codex#2916 corroborate), so the adapters
use the
picoding agent with a Dakar-owned extension injecting the tierthrough pi's documented hook. The ADR carries the amendment.
Review walkthrough
ADR 002
and the
execplan
for the decision, evidence trail, and live-run ledger.
src/workflows/dakar-review/pricing.ts
(versioned table, cache-write-band worst case),
admission.ts
(reserve-first inequalities), and
retry.ts
(deterministic-jitter backoff, worst-case wall-clock budget).
src/workflows/dakar-review/main.ts
— phases Plan, Review, Audit; admission-gated Luna packs; the single
Terra audit with candidate re-pairing; downgrade, deferral, and the
zero-coverage fail-closed guard added after live evidence.
bin/dakar-review.mjs
— in-process prepare, skip, always-on recording through trusted roots,
reported-usage attachment, and the thirteen review-tuning flags.
adapters/pi/extensions/flex-tier.ts
and
adapters/pi/models.json
with the three
pi-*-flexadapters inodw.config.json.
scripts/live-review-harness.mjs
with the SHA-pinned corpus in
scripts/live-corpus.json.
users' guide
(including the CodeRabbit-format support matrix),
developers' guide,
design document
reconciliation, new
README,
contents,
and
repository layout.
Validation
make check: green at every commit (formatting, markdownlint, nixie,docstrings, typecheck, ODW dry run, workflow freshness, 230 tests,
spelling), run through scrutineer agents with teed logs.
is catalogued in the execplan.
reviews across tiers plus one seeded-defect review — 3/3 ground-truth
recall, zero false positives, correct severity grading; two audit
rejections on real PRs judged defensible by hand; one organic finding on
the oversize probe; admission refusals and pack truncation honestly
recorded; deferred and zero-coverage reviews never record the head.
surfaced a real fail-open defect there before that PR was opened.
Notes
legacy-route-finaltag preserves the pre-ADR-002 pipeline as thecomparison arm for roadmap task 7.5.3.
short-circuit, and the adjudicated legacy comparison; cost tunings
(raising
adapterOverheadTokenstoward the measured ~28k first-callcache write, restricting pi tools for finder calls) are recorded in the
execplan retrospective.
pionPATHandOPENAI_API_KEY; theworkflow itself remains dependency-free.
References
References