FUG-120: native iOS exposure bridge (getUserMedia can't set exposure on WebKit) - #91
Open
issuefleet[bot] wants to merge 1 commit into
Open
FUG-120: native iOS exposure bridge (getUserMedia can't set exposure on WebKit)#91issuefleet[bot] wants to merge 1 commit into
issuefleet[bot] wants to merge 1 commit into
Conversation
…on WebKit) The capture path is getUserMedia + MediaStreamTrack.applyConstraints, and exposureControl.ts locks exposure DOWN so auto-exposure stops blowing the LEDs out. But WebKit implements none of the MediaStream Image-Capture exposure extensions (exposureMode/exposureTime/exposureCompensation/iso): on iOS getCapabilities() returns nothing, planExposure() finds no control, and the manual slider + auto servo are silent no-ops — the reported bug. Fix: a first-party Capacitor plugin @splanc/exposure-bridge (AVFoundation), same SPM shape as the wss/speech bridges. Exposure is a property of the physical AVCaptureDevice, not of a capture session, and WebKit's getUserMedia opens the back wide-angle camera — the same device singleton — so configuring it with setExposureModeCustom(duration:iso:) applies to the frames the WKWebView is already rendering (no second capture session). - ExposureBridgePlugin.swift: capabilities/setExposure/clearExposure. Mirrors planExposure semantics — target 0..1 maps shortest→longest, ISO pinned to the sensor minimum, maxExposureMs (Nyquist / manual ceiling) caps the longest exposure using the device's real activeFormat range. - xr/nativeExposure.ts binds it via the injected Capacitor global (registerNativePlugin, no @capacitor/core import), gated on isIosNative(). - MediaStreamCaptureSource.setExposure routes to the native bridge on iOS, falling back to the web applyConstraints path elsewhere; stop() restores continuous auto-exposure. No UI change — the existing slider/servo just start working on iOS. - Scaffolding parity: Package.swift/package.json/BUILD.bazel/.gitignore, link dep in web/package.json + pnpm-lock, doc §4.6, CI/BUILD comments. No new Info.plist string (camera is already granted via getUserMedia). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The exposure slider (Advanced ▸ Manual override, while mapping) does nothing on iOS.
The capture path is
getUserMedia+MediaStreamTrack.applyConstraints(web/src/xr/mediaStreamCapture.ts), andexposureControl.tsdeliberately locks camera exposure down so auto-exposure stops blowing the LEDs out in the dark. But WebKit implements none of the MediaStream Image-Capture exposure extensions (exposureMode/exposureTime/exposureCompensation/iso): on iOStrack.getCapabilities()reports no exposure controls, soplanExposure()finds nothing to set and both the manual slider and the auto servo are silent no-ops. This is exactly the "the webview may not expose this" case the ticket anticipated.Fix — a native AVFoundation bridge
Exposure is a property of the physical
AVCaptureDevice, not of a capture session. WebKit'sgetUserMediaopens the back wide-angle camera (the only device type itsAVVideoCaptureSourceuses) — the same shared device singleton. So a new first-party Capacitor plugin@splanc/exposure-bridge(same SPM shape as the wss/speech bridges) configures that device withsetExposureModeCustom(duration:iso:), and the lock applies to the frames the WKWebView is already rendering. No second capture session, no ownership of WebKit's.Semantics mirror the existing
planExposureexactly, so behaviour is uniform across platforms:targetin[0,1]maps0= shortest exposure (darkest, least LED bloom) →1= longest.maxExposureMs(the Nyquist cap for the servo, the user's ceiling for manual override) caps the longest exposure — computed against the device's realactiveFormatduration range.Changes
web/native-plugins/exposure-bridge/—ExposureBridgePlugin.swift(capabilities/setExposure/clearExposure) +Package.swift/package.json/BUILD.bazel/.gitignore.web/src/xr/nativeExposure.ts— binds the plugin via the injected Capacitor global (registerNativePlugin, no@capacitor/coreimport), gated onisIosNative().web/src/xr/mediaStreamCapture.ts—setExposure()routes to the native bridge on iOS, falling back to the webapplyConstraintspath everywhere else;stop()restores continuous auto-exposure.web/package.json+pnpm-lock.yaml, doc §4.6 indocs/design/ios-support.md, comment touch-ups inweb/BUILD.bazeland the macOS CI workflow.No UI change — the existing manual-override slider and exposure servo just start working on iOS. No new Info.plist string — camera access is already granted via
getUserMedia, and reading/writing device exposure needs no extra entitlement. Non-iOS (browser PWA, Android) behaviour is unchanged:nativeExposureAvailable()is false and the old path runs verbatim.Testing
bazel build //web:web_tests_js(full app + testtsc),//web:dist(Vite/Rolldown PWA bundle), and//web:exposureControl_test///web:exposure_testall pass. Pre-commit hooks (prettier, buildifier, …) clean.native: manual exposure …ms, iso …). The?exposure=servoauto path should likewise track scene brightness.🤖 Generated with Claude Code
Closes-Linear: FUG-120 (https://linear.app/fughilli/issue/FUG-120/ios-exposure-control-doesnt-work)