Repository navigation
ADR 0081 Explicit Consent Before Cloud Ai
Accepted (2026-11-10). Extends ADR 0033 (AI assist and the provider resolution), ADR 0035 (automatic analysis) and ADR 0075 (chat control).
MowDeck sends user data to an AI model in three features:
- AI assist ("recover", "find dock"): up to four camera pictures, which can show people, children, animals and the neighbours' property (the system prompt even asks the model to flag them), a map sketch with the names of the user's areas, and the scene context (position, activity, error code, signal).
- Automatic analysis on errors (ADR 0035): the same, started by the service when a mower reports a recoverable error. Nobody acts at the moment the data is sent; the user opted in once, under a text that did not name the third party.
- The chat assistant (ADR 0075): the typed text, the mower's name, the area names, the activity and the time, through Vision's structured answer endpoint.
When the model is a cloud model (OpenAI, Anthropic, Google, Mistral and so on, the user's own or the installation's), that is personal data going to a third party. Apple's App Review Guideline 5.1.2(i) asks for disclosure and explicit permission before personal data is shared with a third party, "including with third-party AI", and the GDPR asks for the same transparency. Until now the only hint was a note on the settings page that appeared when a user configured their own cloud provider. A user of the installation's model never saw it, and nothing asked before the first picture left.
The native iOS and Android apps render the same web UI from the server (ADR 0036), so one change in the web app covers all clients.
Requirements:
- Consent is explicit (a button, not a pre-ticked box), informed (what, to whom, that it leaves the installation, that it is optional, how to withdraw) and withdrawable.
- It is enforced on the server, not only in the UI: a script, the assistant add-on or a stale client must not be able to bypass it.
- One rule in one place. No feature may be able to forget it.
- Nothing changes for local models, the demo and the provider test, which do not send the user's data to a third party.
The domain (vision.domain.consent, pure, no I/O) derives a Destination (kind, provider label, family, model) from the effective provider:
-
simulated: the rule based fake model (fake/...: the public demo, development). Nothing leaves the process. -
local: the user's or the operator's own hardware. -
cloud: everything else, including every model nobody knows. Unknown means cloud.
Who decides where a request goes differs, so there are two classifications, and the domain exposes them as two functions that cannot be mixed up:
-
A user's own provider (
classify_own) is classified by a heuristic, because the person whose data it is chose the endpoint.fake/...is simulated. A family flaggedlocal(domain/families.py: Ollama, and the "OpenAI-compatible endpoint") is local when its base URL points into a private network (loopback, LAN, a single-label host such ashttp://ollama:11434); Ollama without a base URL is local too (its default base is on the LAN). Because the prefixopenai/is shared by OpenAI's cloud and by any OpenAI-compatible endpoint,openai/<model>is local only with a private base URL. A "local" family whose base URL is public (a hosted Ollama, a public proxy) is cloud. -
The installation's model (
classify_installation) is never classified by guessing. The user did not choose it and cannot see where the operator's gateway forwards to:openai/...withhttp://litellm:4000may be a private address in front of a cloud provider. The only thing that makes it local is the operator's explicit settingMOWDECK_VISION_DEFAULT_LOCAL=true(VISION_DEFAULT_LOCALin Compose and Helm, defaultfalse); without it the installation's model is cloud whatever its prefix or base URL.fake/...stays simulated either way.
The classification is exposed, never guessed by the client: GET /vision/status (hosted_default_kind), GET /vision/providers (hosted_destination) and GET /vision/me/provider (destination, the one in effect now).
A Consent is the time a user agreed and the destination it was given for. A destination is identified by provider family, model and endpoint: the normalised host (and the port when it is not the scheme's default) of the effective base URL, or default for the provider's own endpoint; for the installation's model it is the host of the operator's configured base URL. The endpoint is part of the recipient: the same model string at another address is another third party. Two pure functions decide:
-
consent_required(destination): onlycloudrequires consent. -
covers(consent, destination): yes when provider family, model and endpoint all match. A changed provider, model or endpoint needs a new consent, because the third party or the model it runs changed. The dialog names the host when it is not the provider's default endpoint.
Consent is keyed by user and account (a sub-user has their own; _ids(principal) takes both from the verified token, so a request can only read, grant or withdraw its own), stored in user_preferences (revision 0008: consent_at, consent_kind, consent_provider, consent_family, consent_model, consent_endpoint, all null without consent) behind the existing Store port. A user who withdraws loses it entirely.
ConsentGate.check(principal, destination) (vision.application.consent) raises ConsentRequired when consent is required and not covered. ProviderService.resolve() is the one place that decides which model a request uses (own provider, else the installation default, else vision-provider-missing); it calls the gate before it returns the configuration. Every path that sends user data to a model goes through it, so none can forget it:
| Path | Where it resolves | Refused before |
|---|---|---|
Manual AI assist, recover and recover/upload
|
VisionService._start_round |
any picture is captured, any image analysed, any proposal created |
| Retry of a proposal | VisionService._start_round |
the same |
| Automatic analysis | VisionService.request_auto |
the photo is asked for |
| Chat assistant (structured answer) | ProviderService.structured |
the question is sent, any usage accounted |
The provider test is exempt: it sends a synthetic image the service generated itself and never resolves. Simulated and local destinations never require consent, so the demo and local models fall out of the same rule, with no special case in the UI or the routes.
ConsentRequired is a domain error. The API adapter maps it to 403 urn:mowdeck:problem:vision-consent-required; the destination travels in errors (parameter: destination, code: the kind, detail: provider and model), the way feature-not-in-plan carries its feature. The assistant service does not learn any rule: it recognises Vision's problem type in the 403 of the structured call and answers its own 403 assistant-consent-required, which the web app treats the same way.
Enabling it (PUT /vision/me/preferences, auto_analysis: true) needs a covering consent when the destination is cloud (ConsentService.require_current; nothing is saved otherwise). At sending time AutoAnalysis.trigger reaches the gate through request_auto, with the principal of the opted-in user whose provider and quota the analysis uses, so it is that user's consent that counts; an opted-in user without a covering consent is skipped, and the skip is recorded like the others (Skip.NO_CONSENT, logged with the user), never silent. Withdrawing a consent switches the user's automatic analysis off. The gate runs at the start of the request (request_auto, _start_round) and once more right before the model is asked (VisionService._analyse): a consent withdrawn while the photo is taken or the light is switched on ends the proposal as failed ("consent ... withdrawn") with nothing sent. The only remaining window is a request already on its way to the provider (milliseconds to the model's answer), which cannot be recalled.
Nobody has a consent record after the migration, so everyone is asked on first use. Users who already switched automatic analysis on are not grandfathered: their opt-in was given under a text that did not name the third party, so it pauses (no_consent) until they agree in the app. A what's-new entry says so, and the AI provider settings show the paused state.
| Route | What |
|---|---|
GET /vision/me/consent |
required, the current destination, granted_at, granted_for, whether it covers the destination |
PUT /vision/me/consent |
Agree to the provider in effect now (replaces an earlier consent; records nothing for a destination that needs none) |
DELETE /vision/me/consent |
Withdraw; automatic analysis goes off |
One client module (apps/web/src/lib/consent/consent.svelte.ts: the three calls, the dialog state and guard(scope, run)) and one dialog (ConsentSheet.svelte, mounted once in the root layout like the step-up sheet). guard asks first, runs the request that sends data, and when the service still answers with the consent problem (the provider changed meanwhile) asks again and repeats once. A refusal ("not now") returns a marker and the feature stays unused without an error. The features only wrap their request: AssistStore.start and retry, ChatControl.ask, and the AI provider settings (saving an enabled provider, switching automatic analysis on). The settings also show the state (provider, model, date, withdraw), for users of the installation's model as well. The texts are German and English and name what is sent: for AI assist and the automatic analysis the camera pictures (people and neighbouring property may be visible), a map sketch with the area names and the mower status; for the chat the typed text and the mower and area names.
Saving a provider asks for consent right after it is saved, because the classification of the new provider is the service's, not the form's; declining keeps it saved, and every feature asks again before it sends anything.
- UI only (a checkbox in the settings, a dialog in the app): rejected. A script or the assistant add-on could still send data, and a stale client could skip the dialog.
- Check in each of the four callers: rejected. A fifth caller would forget it. The provider resolution is the one place all of them share.
- One consent for "AI" regardless of provider: rejected. The third party is what the user agreed to; switching from a provider to another one, or to a different model, changes who gets the pictures.
-
Classify by the model string alone: rejected for
openai/: it is both OpenAI's cloud and every local OpenAI-compatible server. -
Guess the installation's model as local from a private base URL or the
ollama/prefix: rejected. The user cannot check it and the operator's gateway may forward to a cloud provider; only the operator's explicitVISION_DEFAULT_LOCALcounts. - Bind the consent to family and model only: rejected. The same model string at another endpoint is another recipient.
- Grandfather automatic analysis opt-ins: rejected, see above.
- Nothing leaves the installation to a cloud model without a recorded, specific agreement. The check is on the server, so every client (web, apps, API scripts, the assistant) is covered, and a user who agreed under one provider is asked again for another.
- Operators with a local installation model set
VISION_DEFAULT_LOCAL=trueto spare their users the dialog; forgetting it only costs one extra click per user (the safe side). - A consent for a provider the user later replaces stays on record until withdrawn; it simply does not cover the new destination.
- Existing automatic analyses pause until the user agrees; the settings page says so. The cost is a quiet period after the upgrade for users who relied on it.
- The privacy policy template (
apps/web/static/legal/privacy.md) gets an "AI features" section describing the data flow and the consent; operators fill in their provider. - The classification of a user's own provider by base URL treats a private-network host as local: the user chose that endpoint and knows what is behind it. A public DNS name that resolves to a private address counts as cloud (the safe side). The installation's model gets no such guess.
- A user who points the same model string at another host is asked again, and the dialog shows the host.
Mirrored from docs/adr/0081-explicit-consent-before-cloud-ai.md at commit 486d902bc6ed1a38fa761c7017c1080b4ee2cd02. Edit the file in the repository, not this wiki page.
Mirrored from the docs/ directory at commit 486d902bc6ed1a38fa761c7017c1080b4ee2cd02. Edit the files in the repository, not this wiki.
MowDeck documentation
- Home
- Phases
- Architecture
-
Decisions (ADR)
- Architecture decisions
- ADR 0001: OpenFGA instead of self-built authorization
- ADR 0002: Self-built Identity service in FastAPI instead of Keycloak or Zitadel
- ADR 0003: SvelteKit for the web app
- ADR 0004: Monorepo with a uv workspace
- ADR 0005: AGPL-3.0 for services and apps, Apache-2.0 for libraries
- ADR 0006: PostgreSQL for all persistence, no SQLite
- ADR 0007: NATS JetStream as event bus
- ADR 0008: Traefik as gateway
- ADR 0009: Command and event envelopes, subject naming and de-duplication
- ADR 0010: Python 3.13 for all services and libraries
- ADR 0011: Separate GPL-3.0 package for the PyMammotion adapter
- ADR 0012: Mower control interface and safety rules
- ADR 0013: Device-Gateway registry, sharding and per-mower concurrency
- ADR 0014: Authorization model, tuple ownership and the mowdeck-authz client
- ADR 0015: Tokens, sessions and key rotation
- ADR 0016: Fleet commands and authorization per command kind
- ADR 0017: Telemetry storage and retention
- ADR 0018: Realtime WebSocket protocol
- ADR 0019: Mammotion adapter transports, manual drive and secrets
- ADR 0020: Web app architecture
- ADR 0021: Browser authentication flow
- ADR 0022: Notifications: rules, recipients and delivery
- ADR 0023: Map rendering and projection
- ADR 0024: Scheduler: recurrence, rain pause and command issuing
- ADR 0025: Home Assistant integration as a pure API client
- ADR 0026: Manual drive channel over the Realtime WebSocket
- ADR 0027: Health history and theft alarm
- ADR 0028: Entitlements: plans as data and enforcement
- ADR 0029: Helm chart and release pipeline
- ADR 0030: Mowing patterns as plug-ins and the Planner
- ADR 0031: Billing with a Stripe adapter
- ADR 0032: Live camera over Agora, joined directly by the browser
- ADR 0033: Vision-assisted recovery with human approval
- ADR 0034: Linked cloud accounts and device discovery
- ADR 0035: Server-side camera frames and automatic AI analysis on errors
- ADR 0036: Native app shells with Capacitor, loading the web app from the server
- ADR 0037: Camera panorama calibration in the browser, with the AI supplying points only
- ADR 0038: Dual link per mower and firmware updates
- ADR 0039: Spot mowing by MowDeck-driven paths
- ADR 0040: Link heartbeat for online status
- ADR 0041: Spot runs from the dock: undock, transit and return
- ADR 0042: Drive calibration
- ADR 0043: Turn styles
- ADR 0044: Drive view with camera
- ADR 0045: Camera relay
- ADR 0046: Device settings as capabilities
- ADR 0047: The RTK base station as a Fleet device
- ADR 0048: Maintenance commands
- ADR 0049: Manual driving over the Mammotion cloud
- ADR 0050: Passkeys and the app login hand-off
- ADR 0051: Sign in with GitHub
- ADR 0052: Sign in with Apple
- ADR 0053: Retention and downsampling of the position track
- ADR 0054: Aerial photo under the mower map
- ADR 0055: Deploy maintenance window
- ADR 0056: Camera photo in push notifications for errors and theft alarms
- ADR 0057: Step-up re-authentication for critical actions
- ADR 0058: Theft tracking mode
- ADR 0059: iOS widgets and the mowing Live Activity
- ADR 0060: Apple Watch app
- ADR 0061: Mowing angle rotation
- ADR 0062: Setup tour for new users
- ADR 0063: Matter server (mowers in Apple Home and Google Home)
- ADR 0064: Personal access tokens and the public API documentation
- ADR 0065: Security center and new-login alerts
- ADR 0066: Cloud provider plugins and adapters for more vendors
- ADR 0067: Error code lexicon, provider independent
- ADR 0068: Public demo mode
- ADR 0069: Provider conformance kit and provider package template
- ADR 0070: Virtual zones for mowers without a vendor map
- ADR 0071: Capability matrix generated from the adapters
- ADR 0072: API explorer in the web app, for the public demo
- ADR 0073: Guided tour with coach marks
- ADR 0074: NATS authentication and per-service subject permissions
- ADR 0075: Chat control as an optional assistant add-on
- ADR 0076: Demo sandboxes, one account per visitor
- ADR 0077: Liveness, readiness and routing
- ADR 0078: Batched state ingestion
- ADR 0079: Fast demo sandbox claim and release
- ADR 0080: GitHub Codespaces dev container with the full Compose stack
- ADR 0081: Explicit consent before camera pictures or text go to a cloud AI provider
- ADR 0082: Deleting your own account inside the app
- ADR 0083: Payments ledger and operator dashboard
- ADR 0084: Area-clear confirmation on the native surfaces
-
Development
- Getting started
- Service template
- Shared libraries
- Cloud providers: adding a vendor
- Adding a provider: template and conformance kit
- Adding an error lexicon for a vendor
- Design: identity and authorization
- Design: fleet, telemetry, realtime
- Design: web app
- Web app
- Home Assistant integration
- Camera source bridge protocol
- Security assurance case
- Fuzzing and property-based tests
- CI and pull request scope
- Reproducible builds
- GitHub wiki mirror
-
Operations
- Operating the Compose stack
- Operating MowDeck on Kubernetes (Helm)
- Internal TLS
- Building the Bluetooth bridge (ESPHome on a XIAO ESP32-S3)
- Releases
- Vendor library updates
- Billing operations
- Telemetry: retention and position downsampling
- iPhone and Android apps
- Passkeys
- Sign in with GitHub
- Aerial photo under the map
- Sign in with Apple
- Theft tracking mode
- Matter server
- Chat control (assistant add-on)
- Source code offer (AGPL-3.0 section 13)
- Public demo
- Load testing
- Spikes
-
Guides
- Guides
- ESP32 bridge: purchase and setup
- Signing in with a passkey
- Signing in with GitHub
- Error codes in plain words
- Aerial photo under the map
- Signing in with Apple
- Confirming critical actions
- Your security page
- Deleting your account or yourself
- iPhone widgets and the mowing Live Activity
- Apple Watch
- The setup tour
- Apple Home and Google Home (Matter)
- Chat control
- Worx Landroid, Kress and LandXcape mowers
- Scripting MowDeck
- Capability matrix
- API reference
- Original plan (German)