How to avoid wide-lens backfacing camera on new phones?

[jan-ivar]

From a stackoverflow question, newer phones have multiple back cameras. How would an app distinguish between them, specifically: avoid or pick the often unsuitable wide-lens camera?

While getUserMedia does support device selection using constraints, I see no reliable difference in constrainable properties between a wide-lens camera and its regular counterpart.

Proposal: A new focalLength constraint.

This would be the distance between sensor and lens (I'm no photographer or device expert, but this seems to often be an inherent property of the lens, e.g. on the Samsung S10).

This would be different from the existing (and similar-sounding) focusDistance, which is distance from lens to object.

[DoDoENT]

In general, some web apps might want to perform real-time image processing of the camera stream. Sometimes, the app would need to use the camera with autofocus capability. Currently, it is not possible to specify that (usually a wide-lens camera does not support autofocus, but on some devices it does).

The focusDistance constraint should also cover this case, as something similar already exists in Android Camera API, where the minimum focus distance of 0 indicates a fixed-focus camera. Also, iOS has also very similar property. However, at the moment no browser implements this constraint.

Regarding the focalLength, the appropriate android property would probably be available focal lengths, while on iOS it would require some calculation.

However, would it be possible to implement that feature also in desktop browsers?

[alvestrand]

Focal length is not an useful value, since its numeric value depends on the size of the sensor, which is information totally opaque to the user (a 35 mm wide sensor, the size of old-fashioned film, will have a "medium wide angle" focal length of 35 mm; a 9 mm wide sensor will have the same angle of view with a 9 mm lens).

We should use angle of view as the selection parameter. Surprisingly, ImageCapture, which incorporates a "zoom" control, hasn't solved this problem.

[jan-ivar]

@alvestrand Sounds desirable, but can we rely on this information being available from drivers? For drivers that report (only) focal length, are you suggesting user agents guess the size of sensors and convert to angle?

[jan-ivar]

From https://lists.w3.org/Archives/Public/public-webrtc/2020Feb/0073.html

Proposal A: focalLength
...
Decision: Rough consensus to go forward with proposal A but in millimeters.

[sonntag-philipp]

Are there any updates on this topic?

Is there some way I could help? This specification would be really useful, because it seems to be a big problem on many devices, especially Samsung as I noticed so far.

[jasonmayes]

+1 to an update on this if possible. I have had a number of folk ask me about this too when out at events and currently unable to provide any reasonable way to do this.

The use case here was for a company in the real estate industry where they need access to the wide angle camera. They cant / dont want to use the standard cam as customers want the wide angle views in the tours, but currently are unable to get access to it via the web APIs.

Even better would be if one could access the stabilized camera feed if the phone supports video stabilization too for example. Right now this is only previewable at the camera app level (my guess is that this is a software level thing implemented by the default OS camera app, but would be amazing if that were exposed too) to save having to reimplement that feature.

Also to note: The reason they can not use a pre-recorded video in this use case is that they want to use TensorFlow.js machine learning models on the live video stream to show the user where to place the camera (AR like experience) to capture the best tour possible of a given house etc so need to do all in real time via the live view vs a recorded video. So this is a tool to assist real estate folk to take the best videos possible so needs to be done in the moment.

[polaroidkidd]

@jasonmayes I've spent quite some time trying to find workaround to get this to work (the other way around though, we don't want wide-angle cameras and some phones select them as default when requesting a video device) and I've come up empty handed.

Usually it isn't that of a big deal since users are usually provided with a "switch camera" button. However, since firefox requests permissions per device, switching the camera will result in an extra permission dialog.

You can either:

1. Iterate through all cameras and repeatedly ask for permission (I personally don't like this since it comes off as "give-me-access-to-everything"
2. Ensure that you have toasts informing the user to switch the camera from wide-angle to standard-angle and live with the fact that they'll have to give permissions again.

currently are unable to get access to it via the web APIs

You mean that even when iterating over the video-devices you can't get access to the wide-angle camera?

[piechoo]

Is there any progress with this issue? It would be great, if we could choose correct lens not by just guessing

[polaroidkidd]

I think the big issue hampering progress on this issue is the lack of data available when requesting a camera from a device as the device might not make the information required to differentiate between wide angle and normal lense available to the browser. Can you request wide angle lenses when developing natively on android or iOS?

[ivancuric]

What I've done so far is to query the torch capability of the camera.
On all the Android devices I've tested it's always on the wide-angle (default) one that has it, not the ultrawide and the telephoto one.

Since iOS doesn't have expose torch capabilities, I have to fall back to querying the standardized camera names, which is a hassle since the strings are localized, and the localization dictionary isn't publicly available.

[RomanKiryanov]

The requested feature is very desired for our application. We, as also mentioned above, are interested in selecting a simple camera (not telephoto, not ultrawide) for the AR experience in web. So we are eager to help.

I would like to share some hints towards navigating the iOS back camera labels: we found these library https://unpkg.com/browse/scandit-sdk@4.6.1/src/lib/cameraAccess.ts, where you can see all (popular) localizations of back camera label + the logic of selecting the simple camera by selecting the "shortest" label. Not sure if it works correctly for all languages, especially Chinese.

[suzukieng]

I think the big issue hampering progress on this issue is the lack of data available when requesting a camera from a device as the device might not make the information required to differentiate between wide angle and normal lense available to the browser. Can you request wide angle lenses when developing natively on android or iOS?

On iOS, you select an explicit device type (enum), e.g. builtInWideAngleCamera, so it's kind of specific to Apple devices.
https://developer.apple.com/documentation/avfoundation/avcapturedevice/devicetype

On Android, using camerax API, there are "camera characteristics" which can be used to filter out undesirable cameras (https://developer.android.com/reference/android/hardware/camera2/CameraCharacteristics), such as supported auto-focus modes or focal length range – the list is quite extensive. I have no experience on how well this works in practice, I guess it would be up to the device vendor to provide all these values correctly.

[ivancuric]

EDIT:
This was related to a comment that was deleted in the meantime.

I am sceptical of this approach:

1. While the name is prefixed by "camera2", 0 only indicates that this is the first enumerated camera in the array, which is vendor-arbitrary. A front facing camera can also be indexed at 0, as can a telephoto lens.
2. The "facing back/front" part is locale-specific so it's not something you can test in production. This also duplicate information as you've already queried the facingMode.
3. Forgive me if I'm wrong as it might just be a language barrier issue, but this sounds like a chatGPT response based on how verbosely it breaks down a trivial snippet of code, while also missing the nuances.

[fbueckle]

Also wanted to chime in to say that such a feature would be very welcome. We had our fair share of issues with Samsung devices and resorted to always using the last element of the returned list of devices in order to choose correct camera (based on anecdotal evidence) but on the new iPhone 15 Max Pro this fails and hence results in the use of a wide-lens camera. From an UI/UX perspective it is also not reasonable to let the user manually select the proper back camera. For example the iPhone 15 Max Pro has SIX potential back cameras!