Repository navigation
REQ 120
The engine shall provide an offline plan for an authorized model-review workload using the same evidence selection, canonical request construction, and conservative budget accounting as inference, without contacting MCP, Ollama, source build tools, or VM infrastructure.
The authorized metadata-only diagnostic L2 under issue #246 reached its 90-second client deadline during inference after offline token accounting had passed. A reusable plan can expose request costs and limits before separately authorized live work, while making clear that these bounds do not predict CPU latency, cold-load cost, model metadata compatibility, or server completion. Issue #248 tracks this planning surface; it authorizes no live attempt or deadline change.
The explicit plan-only CLI validates the exact authorization, contract, stopped capture and public build proof, then reuses canonical evidence selection, build_review_request, and request_budget without starting MCP/Ollama clients or build/VM operations. Its JSON output contains only a closed plan schema, approved model digest and execution/lifecycle limits, authorized evidence field names and coverage counts, per-request canonical request/policy hashes and bounded input-byte/token/context/output/image costs, and an overall plan hash. It contains no prompts, responses, images, model prose, source data, private paths, or raw exception text. The output explicitly reports latency as unknown/not estimated, model metadata compatibility as unqueried, and server completion as not-started; it is not an assessment, success receipt, build proof, or capacity prediction. Expired privacy policy, unsupported accounting, budget overflow, or invalid evidence fails closed without truncation or external calls. Plan mode rejects inference-only/mutating output, checkpoint, resume, refusal-observation, endpoint, and MCP options. Tests prove request-hash parity with the canonical review builder, no external side effects, canary minimization, budget refusal, and expiration behavior.
TEST-121 executed pure planner and CLI regressions using synthetic/public fixtures and patched constructors that fail on any MCP/Ollama invocation. A plan's hash is compared with the canonical request hash used by the inference path. PR #249 passed all seven applicable checks and merged as 69db2c8616f976cdab207a78c715430204697dcb. No live run is required or authorized by this requirement.
High
scripts/tests/test_assisted_benchmark_review.py: test_offline_review_plan_matches_canonical_request_hashes_without_contents, test_offline_review_plan_refuses_expiry_budget_and_lifecycle_overflow, and image-plan minimization/cost refusal within test_approved_images_are_hash_verified_and_sent_only_to_local_vision_model; scripts/tests/test_assisted_benchmark_run.py: test_plan_only_cli_uses_no_mcp_ollama_or_output_side_effects, test_plan_only_cli_rejects_live_options_without_refusal_output, and test_plan_only_cancellation_does_not_claim_unknown_server_completion. Focused review/run/contract slice: 209 passed. Full local suites: .venv/bin/python -m pytest scripts/tests/ -q --tb=short (2,171 passed) and dotnet test tools/source-config/ProjectBuddy.SourceConfig.Tests --no-restore --verbosity minimal (108 passed). PR #249 passed all seven checks, including Windows and Ubuntu source-config jobs.
scripts/assisted_benchmark_review.py (select_evidence, build_review_request, request_budget, build_review_plan); scripts/assisted_benchmark_run.py (plan-only CLI path that never constructs McpStdioClient or LocalOllama); tools/assisted-benchmark/README.md; tests in scripts/tests/test_assisted_benchmark_review.py and scripts/tests/test_assisted_benchmark_run.py. Integrated through PR #249, implementation 66c394a394a32dea4115e7c63d427eaa94fbca95, merge 69db2c8616f976cdab207a78c715430204697dcb.
Implemented in develop through PR #249. Offline request/hash parity and no-client behavior verified; latency, model metadata compatibility, live assessment, and native-build qualification are not claimed.