Repository navigation
REQ 119
The engine shall preserve a requested minimized non-success observation of model-review refusal without replacing the original failure or exit code.
The metadata-only live attempt following issue #244 stopped with a deadline error but no safe phase/counter record, so it was not possible to distinguish preflight, model metadata, inference, or postflight exhaustion from the CLI output. Issue #246 adds diagnostic evidence rather than authorizing a retry, relaxed limits, or a passing benchmark claim.
An explicitly requested projectbuddy/assisted-review-refusal@1 record contains only host-defined phase, failure-kind and refused/cancelled outcome enums, original exit code, validated numeric lifecycle limits, bounded request/retry counters, finite elapsed time when available, possible in-flight state, a request hash when available, and explicit unknown/not-started server-completion status. The closed phase set covers input validation, orchestration/context and capture preparation, review/checkpoint preflight, identity preflight, model metadata, inference, identity postflight, and finalization. No raw prompt, response, model prose, private path, tool output, arbitrary exception message, or dynamically named failure category is retained. The observation is separate from any successful receipt/checkpoint, has complete: false, and cannot be consumed as success. Requested persistence uses private atomic non-overwriting custody (POSIX 0700 directory/0600 current-user regular file, or exact current-user-plus-SYSTEM access on supported Windows paths); unsupported/unsafe custody fails closed. Observation finalization failure emits only a fixed safe diagnostic and preserves the original exception/cancellation and exit code. Counts and phase are sampled without rerunning failed work, calling extra tools, expanding deadlines, or asserting server completion from residency alone. Successful receipts and existing model identity, budget, execution, privacy, lifecycle, and recovery contracts remain compatible.
TEST-120 records executed synthetic canary, fake-clock/transport, cancellation, private-writer and CLI regressions integrated through PR #247, implementation 537dc3025d278ae11da99298ef7cd09a9577ab32, merge fbd69b0ae6480ae97b725c325b4ad05fe5189ba1. Full local suites passed 2,166 Python and 108 .NET tests; all seven applicable PR gates passed, including Windows and Ubuntu source-config tests. The one separately authorized metadata-only diagnostic L2 preserved exit 2 with an inference-phase deadline observation, zero retries and unknown server completion; no final receipt or model verdict exists. This verifies refusal diagnostics, not successful review, native build or server termination. L1 is not relabelled as a pass; issue #248 tracks the separately scoped workload-planning gap.
SYS-REQ-029; complements REQ-113, REQ-114, REQ-115, and REQ-118
High
scripts/tests/test_assisted_benchmark_review.py: test_refusal_observation_covers_request_phases_without_exception_text, test_refusal_observation_retains_no_unvalidated_private_input, test_reused_observer_does_not_retain_a_prior_runs_counters; scripts/tests/test_assisted_benchmark_run.py: test_cli_refusal_observation_preserves_original_outcome_without_canaries, test_failed_refusal_persistence_does_not_replace_primary_deadline, test_success_does_not_write_a_refusal_observation, test_refusal_writer_is_private_non_overwriting_and_rejects_unsafe_data; tools/source-config/ProjectBuddy.SourceConfig.Tests/GenerationTests.cs: RefusalSnapshotIdentifiesEveryModelRequestPhase, RefusalObservationExcludesExceptionCanaryAndUsesPrivateCustody, ReusedRefusalObserverCannotRetainOldRequestCounters, CliDiagnosticsDoNotEchoPrivateInputPaths. Reproduce from source root with .venv/bin/python -m pytest scripts/tests/ -q --tb=short and dotnet restore tools/source-config/ProjectBuddy.SourceConfig.Tests --locked-mode followed by dotnet test tools/source-config/ProjectBuddy.SourceConfig.Tests --no-restore --verbosity minimal. Detailed live observations remain private; sanitized outcome/hash evidence belongs in issue #246, not raw captures or prompts in the wiki.
scripts/assisted_benchmark_observation.py: RefusalObserver.begin_review, mark, bind, snapshot, validate_observation, write_private_observation; scripts/assisted_benchmark_review.py: ReviewLifecycle.call, LifecycleTransport, _verify_model, review_evidence; scripts/assisted_benchmark_run.py: orchestrate, main. Under tools/source-config/ProjectBuddy.SourceConfig/: ReviewRefusalObservation.cs (ReviewRefusalObservation.Create, Validate, ReviewRefusalObserver.Begin, Mark, Snapshot), OllamaReview.cs (ReviewExecution.RequestAsync, Refusal, OllamaReview.RunAsync), Generation.cs (RunAsync, WritePrivateRefusal), and Program.cs (optional destination preflight and original-exit-preserving failure handling). Operator guides are tools/assisted-benchmark/README.md and tools/source-config/README.md.
Implemented in develop through PR #247; targeted offline regressions and private live refusal observation verified. L2 refused at inference without retry or final success receipt; model/native-build acceptance remains unproven.