Skip to content

feat(portal): add the runner-local mobile decision portal and QR pairing - #75

Merged
abrichr merged 2 commits into
mainfrom
feat/mobile-decision-portal-pairing
Jul 27, 2026
Merged

feat(portal): add the runner-local mobile decision portal and QR pairing#75
abrichr merged 2 commits into
mainfrom
feat/mobile-decision-portal-pairing

Conversation

@abrichr

@abrichr abrichr commented Jul 27, 2026

Copy link
Copy Markdown
Member

Implements step 4 of the one-week MVP in .private/MOBILE_ATTENDED_DECISION_PORTAL_2026_07_26.md: Desktop's half of the mobile attended-decision loop — portal lifecycle, customer ingress configuration, one-use QR device pairing, generic OS notifications, and the local task shell.

Desktop owns transport. It adds no decision model: the question, the evidence, the allowed actions, the revalidation, and the outcome all come from openadapt-flow's attended console and are relayed verbatim.

What actually shipped before this PR

I verified the current state across four repos before writing code (per the mid-task correction), reading origin/main directly rather than the stale Desktop checkout.

  • Flow already has the decision half on origin/main: console/human_decisions.py (PR #272, #275) projects a signed HumanDecisionTaskV1, and console/app.py serves /api/attention, /api/attention/{run_id}, /api/attention/{run_id}/actions/{action_id}, /api/attention/notification, and /api/runs/{run_id}/artifact.
  • Types has the wire contract (openadapt_types/human_decision.py, 0.6.0). This PR does not touch it and does not depend on it — the portal relays Flow's JSON without decoding it, which keeps Desktop out of the concurrent agent's lane.
  • Cloud has a responsive dashboard queue and a PHI-free notification outbox, but no QR/device pairing, no PWA/service worker, and no runner-local portal URL anywhere. Its card literally tells the operator to "Open it on the customer runner" — and until now there was nothing on the runner to open.
  • Desktop had none of this lane. git grep -i 'portal\|qr\|mobile\|attended' on origin/main returns only unrelated XDG-portal notes; tauri-plugin-notification was registered in src-tauri/src/main.rs:80 and never used.

So the lane was genuinely open, and it is precisely the missing link between Cloud's "go to the runner" and Flow's local evidence.

Pairing security properties, and how each is enforced

All in engine/portal/pairing.py, enforced in the store rather than the UI.

Property Enforcement
The QR carries no execution capability The link is {origin}/pair#c={secret} and nothing else. The secret rides in the fragment, which browsers never transmit, so it cannot reach a proxy access log or a referrer header. The phone never receives Flow's console bearer token — it gets a portal session token instead.
Single use, atomically claim() does find → validate → compare-and-swap under one threading.Lock. A 12-thread barrier test asserts exactly one winner and eleven already_claimed refusals.
Five-minute expiry, server-side PAIRING_TTL_S = 300 evaluated from time.monotonic() on every access. A test freezes datetime backwards and shows the refusal still fires, so a wall-clock change cannot extend a code and no UI countdown is involved.
Runner-bound Every record and session carries the runner id and the exact ingress origin it was minted for.
Phishing resistance Claiming yields a session that is unusable (202 pending_approval) until the operator approves. Both devices display the same six-character code from an alphabet with no 0/O/1/I/L.
No credential at rest in the clear Only SHA-256 digests are stored; every comparison is hmac.compare_digest. Portal secrets use oapp_, deliberately distinct from Cloud's oap_ local-bridge prefix — a test asserts neither surface's regex accepts the other's secret.
Superseded codes Showing a new code cancels the previous pending one, so an abandoned QR on a prior screen cannot be scanned later.

Loopback default and ingress opt-in

engine/portal/ingress.py — every widening step is explicit and fails closed:

  • Default binds 127.0.0.1 and advertises a loopback URL. reachable_from_phone is False and the pairing screen says so rather than minting a dead link.
  • customer_ingress still binds loopback unless you name a specific address; the expected shape is your own reverse proxy beside the runner.
  • Refused in every mode: wildcard bind (0.0.0.0, ::, *, empty), plaintext origin, origin with path/query/credentials, hostname-as-bind-address, and customer_ingress without both portal_public_origin and portal_ingress_acknowledged.
  • Any invalid combination raises and the portal does not start.

There is no test-only wide bind. tests/test_portal/test_end_to_end.py binds a real socket using the shipped loopback configuration and drives the whole loop over HTTP.

No protected content in notifications

engine/portal/notifications.py is structural, not editorial: it reads one integer from Flow's endpoint and renders the body from a fixed template. No upstream string is ever forwarded, so there is no path by which a question, value, identifier, or application name can reach a lock screen. assert_generic_notification refuses any widened payload, and is called in the portal service, again in the dispatcher before emit, and again in src/lib/attentionNotification.ts before the plugin. Tests feed hostile payloads (MRN 4417092, Coverage: active, the full question text) through every layer and assert they cannot appear.

No service-worker caching of protected routes

engine/portal/shell/sw.js precaches one Object.freezed literal list; there is no after-the-fact cache write; and the fetch handler returns before respondWith for anything not in that list. Because the guard is an allowlist, a newly added protected route is excluded by default. tests/test_portal/test_shell.py parses the worker source and asserts each clause, plus that the precache list contains no /api/ path and is a subset of the server's shell assets. Protected responses are additionally Cache-Control: no-store, no-cache, must-revalidate, private with Pragma: no-cache.

It is a responsive PWA (viewport-fit, safe-area insets, thumb-reachable action bar). No iOS or Android project was added; a test asserts that.

Acceptance tests from the design this satisfies

  • an action outside allowed_actions refuses — the shell renders only task.allowed_actions and the relay is a closed route allowlist; Flow remains authoritative
  • two operators cannot both decide the same task — one-use claim; concurrent-claim test
  • raw evidence responses are no-store, authorization-scoped, and never cached by the PWA/service worker
  • no notification contains protected content
  • loss of network after tap shows pending/uncertain, never success — the shell shows "Submitted. Waiting for this computer to check the live screen…", relays the runner's exact status, and on 5xx/no-response says the result is uncertain and not to answer again
  • an accepted tap is never rendered as verified (test asserts VERIFIED never appears in the shell)

Refusal-matrix cases (expired/forged/wrong-tenant/wrong-pause/wrong-bundle/wrong-event-sequence, idempotency, staleness, uncertain delivery) are Flow's, unchanged and reached through the relay.

What is stubbed, and what must land elsewhere

Nothing in the decision path is stubbed — the seam is real and calls Flow's shipped routes. Two things block end-to-end operation, both outside this diff:

  1. Packaging (needs a human). pyproject.toml pins openadapt-flow==1.23.0 in the build extra, which predates console/human_decisions.py (added in 1.24.0), and does not request the console extra — so fastapi/uvicorn are absent from the frozen sidecar and openadapt-flow console exits with an install hint. Freezing uvicorn also needs PyInstaller collection flags in scripts/build_frozen_engine.py. I deliberately did not change the signed-installer runtime pin here: it alters every installer and cannot be validated without the native matrix. The portal works today against a dev install (pip install 'openadapt-flow[console]>=1.24.0').
  2. A Flow interface for the console capability. serve() generates its bearer token inside the function and only prints it; there is no injection point. engine/portal/service.py::_parse_console_banner reads the exact banner line with a strict regex that fails loud. A narrow --capability-file option in Flow would replace this properly.

Also worth flagging from the survey: Flow's console middleware refuses any request whose Host is not 127.0.0.1/localhost and any Origin that is not http://127.0.0.1:<port> (console/app.py:89-116). A phone therefore cannot use the console even behind a customer reverse proxy without Host/Origin normalization — which is exactly what engine/portal/flow_client.py does, and why the relay exists rather than a direct proxy.

Gates

  • uv run ruff check engine/ tests/ scripts/ — clean
  • uv run pytest tests/673 passed, 2 skipped (100 new)
  • npm run test:ui — 37 passed (11 new); npm run build — clean
  • Wheel build verified to include engine/portal/shell/*; check_release_consistency.py clean; uv lock --check clean
  • New dependency segno (BSD-3-Clause, pure Python, no runtime deps) renders the QR locally — a pairing link is never sent anywhere to be encoded

🤖 Generated with Claude Code

https://claude.ai/code/session_01NyCHrzA1psrKMFfroYbzaM

Desktop's half of the mobile attended-decision loop: portal lifecycle,
customer ingress configuration, one-use QR device pairing, generic OS
notifications, and the local task shell. Decision semantics stay in
openadapt-flow; this relays them and adds no authority of its own.

The portal binds loopback by default and fails closed on any incomplete
ingress configuration. The QR carries only a one-use, five-minute,
runner-bound pairing secret in the URL fragment, claimable exactly once
under one lock, and unusable until the operator matches a short code shown
on both devices. Protected task, decision, and evidence responses are
no-store, and the shell's service worker precaches one frozen literal list
with no after-the-fact cache write.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NyCHrzA1psrKMFfroYbzaM
An independent review of the portal found that several documented security
properties were not actually enforced. Each is fixed with a regression test.

1. The confirmation code defended nothing against the stated threat. It was
   derived from the pairing and shown on Desktop, then handed to whoever
   claimed the secret -- so an attacker who photographed the QR and claimed it
   first was shown the exact code the operator's screen was displaying, and
   "codes match" confirmed the attacker. The code is now minted per claim,
   returned only to the claiming device, kept here as a digest, and typed back
   by the operator, with bounded attempts that cancel the pairing on
   exhaustion.

2. The five-minute deadline stopped applying once a code was claimed: the
   sweep expired only pending records, so a claim could be approved hours
   later into a full twelve-hour session, and a new QR did not retire it.
   Claimed records now expire, approval re-checks the deadline itself, a new
   QR retires every unapproved predecessor, and both clocks are consulted so
   neither a wall-clock change nor a suspended machine can extend a lifetime.

3. The HTTP handler set no socket timeout, so every read blocked forever and
   idle connections could pin all worker threads. It now has one, refuses an
   oversized or malformed body before reading rather than silently emptying
   it, and never raises out of the handler.

4. Console startup read a pipe with a blocking readline while holding the
   service lock, so a console that started and said nothing wedged every
   portal command permanently; stderr was never drained, which would block the
   console once its pipe filled. Startup now happens outside the lock with
   both pipes drained by reader threads and a sentinel on early exit.

Also hardened: shell assets are served network-first so a fix in app.js can
reach an already-paired phone; the QR is returned as an inert data URI instead
of raw SVG markup for the Desktop window to inject; only raster image types are
relayed; upstream 5xx bodies are replaced with a fixed message; artifact
identifiers reject traversal like run identifiers already did; and the relay
re-validates identifiers so its boundary cannot drift from the HTTP surface's.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NyCHrzA1psrKMFfroYbzaM
@abrichr

abrichr commented Jul 27, 2026

Copy link
Copy Markdown
Member Author

Update: adversarial security review, and four real defects fixed

I ran an independent adversarial review of this diff against the stated threat model (hostile network, photographed QR, protected evidence must not leak or be cached). It found that several documented properties were not actually enforced. All four are now fixed with regression tests; the review's remaining categories came back clean.

1. The confirmation code defended nothing against the actual threat

The original code was derived from the pairing in create() and then returned to whoever claimed the secret. An attacker who photographs the QR from across the room and claims it first was therefore handed the exact code the operator's screen was displaying — and "approve only if the codes match" would have confirmed the attacker perfectly. approve() took only a pairing_id, so nothing the attacker lacked was ever demanded. The property in the module docstring was simply false.

Fixed by reversing the direction: the code is now minted at claim time, returned only to the claiming device, kept here only as a digest, and the operator types back what their phone is showing. A remote attacker's phone displays a code the operator cannot see, so approval fails. Attempts are bounded (3) and exhausting them cancels the pairing outright. pairing_status() no longer exposes any code.

This deviates from the design doc's literal "matching code on both devices" wording while serving its stated purpose ("reducing remote QR phishing"). It is the numeric-comparison-vs-passkey-entry distinction, and entry is the one that actually resists a copied QR.

2. The five-minute expiry stopped applying once a code was claimed

_expire_locked only expired pending records, and approve() never re-checked the deadline — so a pairing claimed at T+299s could be approved hours later into a full 12-hour session, with pairing_status cheerfully reporting claimed and a clamped 0s. Worse, create() cancelled only pending records, so a stolen-and-claimed code survived the operator displaying a fresh QR and sat latent waiting for a mis-click.

Fixed: claimed records expire too; approve() re-checks the deadline itself and reports expired rather than not_claimed; a new QR retires every unapproved predecessor and revokes its session. Expiry now consults both clocks — monotonic so a wall-clock change cannot extend a deadline, and wall time because CLOCK_MONOTONIC stalls across suspend and a slept laptop would otherwise carry a session well past 12 hours.

3. No socket timeout anywhere — slowloris

_Handler set no timeout, and neither does ThreadingHTTPServer, so socketserver never called settimeout(): every read blocked forever. With daemon_threads = True, a few thousand idle sockets pin every thread and FD. Fixed with an explicit timeout, an explicit protocol_version, a Content-Length parse that refuses instead of raising out of the handler, and an oversized body refused before reading (which also made the previously-dead 413 branches reachable).

4. Console startup could wedge the portal permanently

_start_console called a blocking readline() while holding the service lock, so CONSOLE_START_TIMEOUT_S was decorative — a console that started and printed nothing blocked forever, deadlocking status(), stop(), and every portal_* command with no recovery. stderr was never drained either, so the console would block once its pipe filled. Fixed: startup happens outside the lock, both pipes are drained by reader threads for the process's lifetime, a sentinel fires on early exit so failure is immediate, and terminate() is followed by wait().

Also hardened

  • Shell assets are served network-first (cached copy as offline fallback only). Cache-first pinned whatever app.js was installed the first time a phone paired — browsers only reinstall a worker when sw.js itself changes bytes, so a security fix could never have reached an already-paired device. The structural no-post-fetch-write guarantee is unchanged and still asserted.
  • The QR is returned as an inert data:image/png URI instead of raw SVG for the Desktop window to inject — it encodes the pairing secret, so a raw-HTML sink there was a secret-exfiltration primitive.
  • Only image/png|jpeg|webp are relayed; image/svg+xml is refused as an active document.
  • Upstream 5xx bodies are replaced with a fixed message (a traceback or deployment path must not reach a phone); 4xx refusals pass through intact because they are operator-actionable.
  • Artifact identifiers reject .., matching what run identifiers already did.
  • The relay re-validates run/action identifiers so its boundary cannot drift from the HTTP surface's.
  • The evidence blob URL is revoked after load instead of leaking for the document's life.

Review categories that came back clean

Route authorization ordering (no path reaches the relay without an approved session); percent-decoding consistency (the route regexes exclude %, so %2e%2e/%2f 404 rather than bypass); no-store on every protected response including all refusal and 500 paths; CORS (no Access-Control-Allow-Origin is ever emitted, and the credential is an Authorization header with credentials: "omit", so the absence of an origin check on GET is safe); secret handling (log_request and log_error both delegate to the overridden log_message, so there is genuinely no access log — I verified this against the CPython source); eviction flooding (eviction only runs behind a valid secret or an operator action); the claim critical section (genuinely one uninterrupted find → validate → compare-and-swap); and app.js XSS (every interpolation passes through esc(), every attribute is double-quoted, no href/src/on* sink, and CSP has no unsafe-inline).

Gates

686 passed, 2 skipped (113 portal tests), ruff clean, frontend 37 passed + build clean, gitleaks clean, all 14 CI checks green on the previous head.

@abrichr
abrichr merged commit 3e51a34 into main Jul 27, 2026
15 checks passed
abrichr added a commit that referenced this pull request Jul 27, 2026
… to a deployment target (#78)

* fix(packaging): freeze the attended console the decision portal needs

The mobile decision portal (#75) shipped dead in every signed installer.
The frozen sidecar pinned `openadapt-flow==1.23.0`, which predates
`openadapt_flow/console/human_decisions.py`, and requested no extras, so
fastapi/uvicorn were absent and `openadapt-flow console` exited with an
install hint instead of a capability banner.

Verified against the built artifact rather than the pin:

- 1.23.0 `console --attend` exits with "the operator console needs fastapi,
  uvicorn"; its `/api/attention/{run_id}` returns a bare attention item with
  no `task`/`presentation`, which the portal shell cannot render.
- `openadapt-flow[console]==1.25.0` requires `openadapt-types>=0.6.0,<0.7.0`,
  which conflicted with Desktop's exact `openadapt-types==0.5.0`. The 0.6.1
  control-overlay v2 schemas and enums are byte-identical to 0.5.0, so the
  regenerated projection changes only its provenance header.
- 1.25.0 moved Playwright from a core Flow dependency to its `browser` extra.
  Without that extra the frozen driver became build-only and the artifact
  boundary gate correctly refused the archive.
- PyInstaller does not reach uvicorn's run-time string imports
  (`uvicorn.loops.*`, `uvicorn.protocols.*`, `uvicorn.lifespan.*`) or Flow's
  lazily imported console package.
- Flow prints the console's one-time capability banner with a plain `print`.
  A frozen executable block-buffers stdout, so the banner never reached the
  pipe the portal reads and a serving console still looked dead.
  `PYTHONUNBUFFERED` does not reach a PyInstaller-frozen interpreter;
  `--python-option u` does.

The frozen-notice closure now resolves the same extras the installer freezes,
so fastapi/starlette/uvicorn are audited runtime packages with mandatory
notices instead of build-only imports, and one shared parser refuses a Flow
pin that drops a required extra.

`scripts/smoke_test_frozen_flow.py` proves the result behaviourally on every
sidecar build: it starts the frozen attended console, parses the banner with
the portal's own parser, and drives `/api/session`,
`/api/attention/notification`, `/api/attention`, `/api/attention/{run_id}`,
and an unauthenticated probe through `engine.portal.flow_client`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NyCHrzA1psrKMFfroYbzaM

* fix(portal): bind the console to a deployment target and bound its startup

`portal_start` could not complete. Two portal-side defects, plus the open
security question about the staged deployment config, are resolved here.

**The staged config is not session-lived, and per-run re-staging would be
theatre.** `data_dir/deployment.json` carries reusable credentials
(`rdp_password`, `rdp_username`, `rdp_domain`, `agent_token`, `agent_tls_pin`)
alongside PHI-capable selectors; `private_flow_config` already treats every
`password`/`token`/`secret` key as sensitive. That rules out an hours-long
window. But in the pinned `openadapt-flow==1.25.0`,
`_attended_service_from_args` resolves `--config` eagerly through
`load_deployment` *before* it yields, and `AttendedActionService` is built from
the parsed `DeploymentConfig` and never sees the path again -- so re-writing the
file per run would not change one byte of what the console executes with.

The lifetime is therefore the console's startup: staged 0600, console spawned,
file removed the instant the capability banner arrives. `serve()` prints that
banner strictly downstream of the config load in the same `with` statement, so
the banner is a happens-after proof, not a timing guess. Measured window on the
frozen binary: 4.99s. This bounds what duration can actually bound -- a
same-user backup, file-sync client, crash reporter, or support bundle sweeping
the copy. It does not pretend to stop same-user code execution, which already
has the operator's own file and the console's in-memory copy.

Removal is guaranteed by `stage_private_yaml`'s `finally` on every exit path
including a console that never announces itself and an unparseable config. A
`finally` cannot survive SIGKILL, so start also sweeps `.deployment-*.yaml`
older than `STALE_STAGING_AGE_S`; the age bound means a concurrent start can
never have its own in-use file deleted.

Also fixed:

* `PortalService` passed `--allow-actions` with no deployment target, which
  Flow deliberately refuses. It now passes `--config <staged>`. A missing or
  unparseable config refuses before anything is spawned, naming the file.
* `start` called `session` immediately after the banner, racing uvicorn's bind.
  `_await_console_session` polls to a real deadline and gives up at once if the
  console exited, rather than burning it.

Behavioural evidence from the extended smoke, on the built artifact:
`console_accepts_staged_config`, `console_read_staged_config` (Flow echoes a
marker existing only inside the staged file), `console_attended_session`,
`console_serves_without_staged_config` (every route driven with the file
already deleted), `console_attended_banner_seconds: 4.992`, and
`console_refuses_unbound_mutations` still true -- the no-target refusal is
preserved, not worked around. Flow requires an injected `WindowClient` for this
backend on Linux by design, so the headless leg proves the config read rather
than pretending an attended session can attach.

Nothing unpinned, loosened, or vendored; the `--collect-all openadapt_flow`
refusal, the artifact gate, and `bundled_flow_pin()` are untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NyCHrzA1psrKMFfroYbzaM

* test(portal): classify the attended-session host from Flow's own refusal

The frozen smoke previously decided whether an attended session could attach
from a hardcoded platform tuple. That is guesswork on two of the three CI legs,
and it can hide a real defect: a Windows build missing a frozen module would be
indistinguishable from an unsupported host.

It now attempts the attended start everywhere and classifies the outcome from
Flow's own message. Flow implements a window-scoped replay client on macOS
(Quartz) and Windows (Win32) and refuses elsewhere by design, so Linux records
`console_attended_session: "no-host-window-client"`. Any other startup failure
still fails the smoke.

Every platform, Linux included, continues to prove the frozen binary read the
staged config's bytes via the echoed `backend.kind` marker.

Re-verified on the rebuilt macOS arm64 artifact: console_attended_session true,
console_serves_without_staged_config true, console_attended_banner_seconds
4.992, console_refuses_unbound_mutations true.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NyCHrzA1psrKMFfroYbzaM

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant