Repository navigation
REQ 111
The engine shall accept an Ollama review only when its submitted evidence fits an explicit approved context budget and its response satisfies the complete-review contract.
A byte allowance is not a token budget, image processing consumes model-dependent context, and a normal stop marker alone cannot establish that the approved evidence was considered completely. Issue #232, under epic #229, applies to both source-config and assisted-benchmark review.
Each request records a versioned policy with positive numeric context and output-token limits, bounded text/image sizes and counts, and a model-compatible conservative accounting method for system prompt, evidence, output schema, images, and reserved output. Invalid limits, an unsupported accounting method, or an input that cannot be shown to fit causes rejection before that request transmits evidence. A deterministic reduction is permitted only when its policy is approved and its retained/omitted coverage is recorded; it cannot masquerade as review of the complete original input. Evidence is not silently byte-sliced or delegated to undocumented server truncation. Response model identity, completion/error metadata, JSON parsing, schema, evidence references, and decision consistency all pass before acceptance. Length-limited, malformed, contradictory, or otherwise incomplete output remains non-success even if the server reports a stop. A receipt binds policy/version, actual submitted evidence identity, limits, reduction/coverage outcome, and completion result without retaining unnecessary raw inputs. These checks establish an input/completion contract, not proof that a probabilistic model reasoned correctly about every token.
Boundary cases supporting TEST-113 integrated through PR #239, implementation af7e377e5e9efba985bd5075c9accb1d7a8bd8af, merge 45a7d71d00463fc22b913d95fbcd6412be72d7c4. Local suites passed 2,070 Python and 71 .NET tests; focused budget slices passed 108 Python and 60 generation cases. All seven upstream checks passed, including Ubuntu/Windows source-config jobs. The oversized-request reproducer failed before the fix. Exact-fit/overflow, invalid/mismatched policy, unsupported metadata, media size and completion-count cases are covered without live inference. Supported accounting is limited to declared gpt2 byte-level metadata and caller-reviewed digest-specific template/media bounds; unknown assumptions fail closed, and no truncation/reduction policy is implemented. No universal tokenizer, GPU capacity, model attestation, or reasoning-completeness proof is claimed.
SYS-REQ-029; source-config review REQ-108 and approved model identity REQ-109
High
scripts/tests/test_assisted_benchmark_review.py (test_request_cannot_silently_exceed_its_context_budget, test_context_budget_exact_boundary_and_one_token_over, test_invalid_budget_prevents_any_inference, test_unsupported_model_budget_metadata_prevents_inference, test_incomplete_or_over_budget_completion_is_not_an_assessment, test_image_payload_must_fit_the_explicit_media_budget); tools/source-config/ProjectBuddy.SourceConfig.Tests/GenerationTests.cs (OverBudgetEvidenceSendsNoInferenceAndWritesNoConfiguration, UnsupportedModelBudgetSendsNoInference, IncompleteCompletionLeavesNoConfiguration, ReviewBudgetBoundaryAndEvidenceAreExplicit); scripts/tests/test_assisted_benchmark_run.py (policy/request/model-limit receipt fields); .github/workflows/ci-assurance.yml. Reproduce with python3 -m pytest scripts/tests/ -q --tb=short and locked source-config restore plus dotnet test ProjectBuddy.SourceConfig.Tests --no-restore.
scripts/assisted_benchmark_contract.py (validate_review_budget, validate_authorization: digest-bound policy); scripts/assisted_benchmark_review.py (model_budget, request_budget, build_review_request, review_evidence: supported metadata, request preflight, completion counters and hashed coverage); scripts/assisted_benchmark_run.py (orchestrate: budget evidence); tools/source-config/ProjectBuddy.SourceConfig/OllamaReview.cs (ReviewBudget.Read, Validate, Check, CheckModel, OllamaReview.RunAsync, Parse); Program.cs (required --review-budget) and Generation.cs (approved policy and review evidence), under tools/source-config/ProjectBuddy.SourceConfig/.
Implemented in develop; supported policy/metadata boundaries regression- and CI-verified, no live capacity or universal tokenizer guarantee