Skip to content

fix(scrubber): refuse a scrub that could not run instead of reporting zero redactions - #87

Merged
abrichr merged 1 commit into
mainfrom
fix/scrub-unavailable-must-not-look-clean
Jul 28, 2026
Merged

fix(scrubber): refuse a scrub that could not run instead of reporting zero redactions#87
abrichr merged 1 commit into
mainfrom
fix/scrub-unavailable-must-not-look-clean

Conversation

@abrichr

@abrichr abrichr commented Jul 28, 2026

Copy link
Copy Markdown
Member

The defect

Scrubber.scrub_image returned [] in two different situations that the caller could not tell apart:

  • Presidio ran and redacted nothing.
  • Presidio was not available, so the raw screenshot was copied into .scrubbed/ unchanged.

_scrub_text_presidio had the same shape, silently downgrading to the BASIC regex pass while scrub_manifest.json still recorded scrub_level: "enhanced".

Why it is reachable on a stock install

openadapt-privacy is a hard dependency, but Presidio and spaCy are not in uv.lock, and openadapt_privacy.providers.presidio imports cleanly without them. It raises ModuleNotFoundError — a subclass of ImportError — only when asked to scrub. Verified against the repo's own locked environment:

$ python -c "from openadapt_privacy.providers.presidio import PresidioScrubbingProvider; \
             PresidioScrubbingProvider().scrub_image(img)"
ModuleNotFoundError: No module named 'spacy'

So openadapt-desktop scrub <capture_id> --level enhanced produced:

  • a .scrubbed/screenshots/ tree of unredacted screenshots,
  • scrub_manifest.json claiming scrub_level: "enhanced" with total_redactions: 0,
  • review_status.json of pending_review,
  • and a CAPTURED -> SCRUBBED transition — one operator approval away from REVIEWED, which engine/review.py treats as cleared for egress.

The engine is fail-closed everywhere else (engine/policy.py, engine/runner_loop.py, engine/portal/ingress.py). This was the exception.

The fix

"Could not run" is now representable, not merely logged:

Change File
ScrubbingUnavailableError carrying the level and the underlying cause engine/scrubber.py
_require_provider() constructs the provider and scrubs a probe string — importing the module is not evidence that it works engine/scrubber.py
scrub_capture preflights before creating any output, so a refused scrub leaves no partial directory that reads as a completed one engine/scrubber.py
scrub_image no longer writes output_path when redaction did not run engine/scrubber.py
_scrub_text_presidio refuses instead of returning regex output under an enhanced label engine/scrubber.py
Dispatcher returns {"ok": false, "error": ...}, audits scrub_refused, and leaves the capture CAPTURED engine/dispatch.py
CLI prints the refusal and exits 1 engine/cli.py

BASIC level is unchanged. Copying screenshots without image redaction is what that level declares, and the manifest records it — a documented, representable degradation, not this defect class.

Tests, and the mutation check

New: tests/test_engine/test_scrubber.py::TestScrubUnavailableIsNotCleanResult (5 cases) and tests/test_engine/test_dispatch.py::TestScrubCaptureRefusal.

Every one was confirmed to FAIL against the pre-fix behaviour. Restoring the old except ImportError: shutil.copy2(...); return [] (leaving the tests untouched):

FAILED tests/test_engine/test_scrubber.py::TestScrubUnavailableIsNotCleanResult::test_scrub_image_refuses_and_writes_no_output[ScrubLevel.STANDARD]
FAILED tests/test_engine/test_scrubber.py::TestScrubUnavailableIsNotCleanResult::test_scrub_image_refuses_and_writes_no_output[ScrubLevel.ENHANCED]
FAILED tests/test_engine/test_scrubber.py::TestScrubUnavailableIsNotCleanResult::test_scrub_image_refuses_when_provider_cannot_run
FAILED tests/test_engine/test_scrubber.py::TestScrubUnavailableIsNotCleanResult::test_scrub_capture_refuses_before_writing_anything
FAILED tests/test_engine/test_scrubber.py::TestScrubUnavailableIsNotCleanResult::test_scrub_text_does_not_silently_downgrade_to_regex
FAILED tests/test_engine/test_dispatch.py::TestScrubCaptureRefusal::test_unavailable_scrubber_reports_failure_and_holds_state
6 failed in 0.14s

with the defect's own signature in the dispatcher case:

>       assert result["ok"] is False
E       assert True is False

Local verification

uv run ruff check engine/ tests/ scripts/ — clean.
uv run pytest tests/ -q — 752 passed, 6 skipped.

Notes

  • No release fires from this merge: release.yml is workflow_dispatch only, and native-release.yml is tag/release: published only.
  • No overlap with open PR feat(portal): answer halts on a phone without any customer ingress #85 (engine/portal/service.py, engine/private_flow_config.py).
  • The desktop UI currently calls scrub_capture without a level, so it uses BASIC and is unaffected. It also discards the result; surfacing the refusal in RecordReview.tsx is a separate, smaller follow-up and is deliberately not in this PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NyCHrzA1psrKMFfroYbzaM

… zero redactions

`Scrubber.scrub_image` caught `ImportError`, copied the raw screenshot into
the `.scrubbed/` directory, and returned `[]` -- the exact value it returns
when Presidio ran and redacted nothing. `_scrub_text_presidio` did the same,
silently downgrading to the BASIC regex pass while the manifest still recorded
`scrub_level: "enhanced"`.

This is reachable on a stock install, not a hypothetical. `openadapt-privacy`
is a hard dependency but Presidio and spaCy are not in the lock, and
`openadapt_privacy.providers.presidio` imports cleanly without them -- it
raises `ModuleNotFoundError` (an `ImportError`) only when asked to scrub. So
`openadapt-desktop scrub <id> --level enhanced` wrote a `.scrubbed/` tree of
unredacted screenshots, a `scrub_manifest.json` claiming enhanced-level
scrubbing with `total_redactions: 0`, a `review_status.json` of
`pending_review`, and advanced the capture to SCRUBBED -- one operator
approval away from egress.

The fix makes "could not run" representable rather than logged:

- `ScrubbingUnavailableError` carries the level and the underlying cause.
- `_require_provider()` constructs the provider AND scrubs a probe string,
  because importing the module is not evidence that it works.
- `scrub_capture` preflights the provider before creating any output, so a
  refused scrub leaves no partial directory that reads as a completed one.
- `scrub_image` no longer writes `output_path` when redaction did not run: an
  unredacted screenshot must never appear inside `.scrubbed/`.
- `_scrub_text_presidio` refuses instead of returning regex output under an
  `enhanced` label.
- The dispatcher returns `{"ok": false, "error": ...}`, audits `scrub_refused`,
  and leaves the capture in CAPTURED, which the review state machine already
  blocks from every outbound path. The CLI prints the refusal and exits 1.

BASIC level is unchanged: copying screenshots without image redaction is what
that level declares, and the manifest records it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NyCHrzA1psrKMFfroYbzaM
@abrichr
abrichr merged commit d918b63 into main Jul 28, 2026
15 checks passed
@abrichr
abrichr deleted the fix/scrub-unavailable-must-not-look-clean branch July 28, 2026 03:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant