Weird Wide Angle effect with AR on some phones #303

Whystler · 2021-06-24T20:58:08Z

Do you want to request a feature or report a bug?
BUG

What is the current behavior?
On a Samsung Galaxy 10+ phone, the Aframe AR experience, which uses a marker, presents a skewed and unusual look from the camera and does not show the content on the marker. For example, I placed the marker on the seat cushions of a couch, and pointed the phone at it, and the view on the phone's camera shows the top of the back cushion of the couch. When maneuvering the phone oddly enough to locate the marker, nothing appears on the marker.

Furthermore, on the Samsung, it is noticeable that the image is skewed. If you maneuver the phone so that you are looking at someone's face, their face is slightly skewed and stretched.

On a Google Pixel 3, and other phones, everything works properly. The view is normal, and the marker scans to show the content.

If the current behavior is a bug, please provide the steps to reproduce.

Initiate a website with a marker-based Aframe AR experience on a Samsung Galaxy 10+ phone.
Immediately you should notice a difference in the camera view on the phone.

In this experience, if you scan the marker, a grid image shows up with an overlay UI. Using the overlay UI to pinpoint a cell on the grid, you can change the grid's cell colour to green by tapping on the screen. This is a test example for a game, which is not complete.

Here is the url for the experience: https://unphased.neocities.org/restaurant/test05a.html

And here is the url for the marker image: https://unphased.neocities.org/restaurant/pattern-marker-test01gb.png

Furthermore, here is a picture showing the Samsung Galaxy 10+ issue on the screen vs what you see on a Google Pixel 3:
https://unphased.neocities.org/phone-issue-explained01.png

Please mention other relevant information such as the browser version, Operating System and Device Name

This was tested on the Google Chrome browser and the Firefox browser, using WIndows 10. The issue presented itself using both browsers. The devices tested were the Samsung Galazy 10+, which presented the issue, and also on the Google Pixel 3, and the Galaxy A20 which did not present the issue.

What is the expected behavior?

The expected behaviour is that the view seen on the AR website should be identical if not similar to the view one normally sees when taking a picture on the phone.

If this is a feature request, what is motivation or use case for changing the behavior?

This is not a feature request. It is a bug. I expect it would be difficult for anyone with a Samsung Galaxy 10+ phone to be able to experience marker based Aframe AR . Interestingly, a few months ago , someone explained to me online that one of my experiences appeared stretched. Since it was just the one person, I didn't give it much credence at the time as I was distracted by other things I had to do.

EXTRA INFO, as discussed on the AR topic on the Aframe Slack channel:

It was mentioned that the Samsung Galaxy 10+ phone actually has three back cameras. The first has a wide angle lens, the middle has a standard lens, and the third has a telephoto lens. I verified this to be true.

As a result, I expect that the Aframe AR is choosing the wrong back facing camera, and that in this case is it showing a portion, somehow, of the wide angle lensed camera. Why not the whole thing, I do not know.

I hope this helps!

kalwalt · 2021-06-27T20:46:34Z

Hi @Whystler probably, but i can not be sure, because i have not that device for testing, the camera choosen by the app cause the issue can you check the width and height of the screen in pixels?

Whystler · 2021-06-27T21:03:59Z

Hello @kalwalt

It happens for sure on these two phones:

Galaxy Note 10+
https://en.wikipedia.org/wiki/Samsung_Galaxy_Note_10

Galaxy S10e:
https://en.wikipedia.org/wiki/Samsung_Galaxy_S_series#Samsung_Galaxy_S10

Is that the information you need?

espresso95 · 2021-11-08T00:47:35Z

I am encountering the same problem, on my S21 Ultra, it is choosing the wide angle lens and it is warping my AR experience. I will try deep dive this issue.

alejo4373 · 2021-11-08T01:28:32Z

Could you try to pinch in, as if zooming out?. I had a similar issue in my Samsung S8 and when I pinch in it zooms out and shows the entire image of the camera.

ArielJurkowski · 2021-11-21T14:53:21Z

I've ran to this issue too and I found out that the logic behind selecting the main back camera isn't specific enough and some devices give you the wide lens camera. My Samsung has the wide camera listed in the device list before the normal camera and that's why I'm getting the error.

I've found this piece of hacky code by Googling, actually was thinking of doing this abomination myself before I found it.

They have multiple "back" keywords in an array in multiple languages and they filter out video device labels based on those keywords. Then they sort the left devices alphabetically (ex. "camera 0 back" will be before "camera 4 back", this fixes my Samsung issue).

Here's my code, I've edited it directly in my local arjs js file.

const backCameraKeywords = [
    "rear",
    "back",
    "rück",
    "arrière",
    "trasera",
    "trás",
    "traseira",
    "posteriore",
    "后面",
    "後面",
    "背面",
    "后置", // alternative
    "後置", // alternative
    "背置", // alternative
    "задней",
    "الخلفية",
    "후",
    "arka",
    "achterzijde",
	"หลัง",
    "baksidan",
    "bagside",
    "sau",
    "bak",
    "tylny",
    "takakamera",
    "belakang",
    "אחורית",
    "πίσω",
    "spate",
    "hátsó",
    "zadní",
    "darrere",
    "zadná",
    "задня",
    "stražnja",
    "belakang",
	"बैक"
];

// get available devices
navigator.mediaDevices.enumerateDevices().then(function(devices) {

	const mainBackCamera = devices
		.filter(x => x.kind === "videoinput")
		.filter(camera => {
			const lowercaseLabel = camera.label.toLowerCase();
			return backCameraKeywords.some(keyword => lowercaseLabel.includes(keyword));
		})
		.sort((camera1, camera2) => camera1.label.localeCompare(camera2.label))[0];

	var userMediaConstraints = {
		audio: false,
		video: {
			facingMode: 'environment',
			focusMode: "continuous",
			width: {
				ideal: _this.parameters.sourceWidth,
				// min: 1024,
				// max: 1920
			},
			height: {
				ideal: _this.parameters.sourceHeight,
				// min: 776,
				// max: 1080
			}
		}
	};

	const deviceId = _this.parameters.deviceId || (mainBackCamera ? mainBackCamera.deviceId : null);

	if (deviceId) {
		userMediaConstraints.video.deviceId = {
			exact: deviceId
		};
	}

I couldn't get userMediaConstraints specific enough for the normal lens to trigger, so I've used the hack to possibly get the main back camera. If that doesn't yield any results, then I just use the old existing logic. CTRL+F for navigator.mediaDevices.enumerateDevices() in your arjs file to find the section with the old logic.

darde · 2021-12-20T01:23:17Z

Hello @kalwalt

It happens for sure on these two phones:

Galaxy Note 10+ https://en.wikipedia.org/wiki/Samsung_Galaxy_Note_10

Galaxy S10e: https://en.wikipedia.org/wiki/Samsung_Galaxy_S_series#Samsung_Galaxy_S10

Is that the information you need?

It also occurs for the Samsung Galaxy A51
https://en.wikipedia.org/wiki/Samsung_Galaxy_A51

darde · 2021-12-21T19:28:12Z

I've ran to this issue too and I found out that the logic behind selecting the main back camera isn't specific enough and some devices give you the wide lens camera. My Samsung has the wide camera listed in the device list before the normal camera and that's why I'm getting the error.

I've found this piece of hacky code by Googling, actually was thinking of doing this abomination myself before I found it.

They have multiple "back" keywords in an array in multiple languages and they filter out video device labels based on those keywords. Then they sort the left devices alphabetically (ex. "camera 0 back" will be before "camera 4 back", this fixes my Samsung issue).

Here's my code, I've edited it directly in my local arjs js file.
const backCameraKeywords = [
    "rear",
    "back",
    "rück",
    "arrière",
    "trasera",
    "trás",
    "traseira",
    "posteriore",
    "后面",
    "後面",
    "背面",
    "后置", // alternative
    "後置", // alternative
    "背置", // alternative
    "задней",
    "الخلفية",
    "후",
    "arka",
    "achterzijde",
	"หลัง",
    "baksidan",
    "bagside",
    "sau",
    "bak",
    "tylny",
    "takakamera",
    "belakang",
    "אחורית",
    "πίσω",
    "spate",
    "hátsó",
    "zadní",
    "darrere",
    "zadná",
    "задня",
    "stražnja",
    "belakang",
	"बैक"
];

// get available devices
navigator.mediaDevices.enumerateDevices().then(function(devices) {

	const mainBackCamera = devices
		.filter(x => x.kind === "videoinput")
		.filter(camera => {
			const lowercaseLabel = camera.label.toLowerCase();
			return backCameraKeywords.some(keyword => lowercaseLabel.includes(keyword));
		})
		.sort((camera1, camera2) => camera1.label.localeCompare(camera2.label))[0];

	var userMediaConstraints = {
		audio: false,
		video: {
			facingMode: 'environment',
			focusMode: "continuous",
			width: {
				ideal: _this.parameters.sourceWidth,
				// min: 1024,
				// max: 1920
			},
			height: {
				ideal: _this.parameters.sourceHeight,
				// min: 776,
				// max: 1080
			}
		}
	};

	const deviceId = _this.parameters.deviceId || (mainBackCamera ? mainBackCamera.deviceId : null);

	if (deviceId) {
		userMediaConstraints.video.deviceId = {
			exact: deviceId
		};
	}
I couldn't get userMediaConstraints specific enough for the normal lens to trigger, so I've used the hack to possibly get the main back camera. If that doesn't yield any results, then I just use the old existing logic. CTRL+F for navigator.mediaDevices.enumerateDevices() in your arjs file to find the section with the old logic.

Hey, Bro, thanks for this smart solution. It worked for me. However, I'm facing another issue related to the focus distance. It seems that the focus is a bit zoomed in. Is there any way to control the focus distance?

ArielJurkowski · 2021-12-21T21:05:39Z

Try to remove focusMode constrain from my code, I might have left it there by accident when testing. Or set it to 'none'.

Check this link out, there is info about two focus related constraints and their all possible values: focusMode and focusDistance. Yoink.

I didnt have this issue, so please check yourself... Bro 😉 Its your best bet, but it might not work, the camera API is crap (on purpose for privacy reasons, but still... ).

MarkEEaton · 2023-08-04T21:50:33Z

(This is a delete and redraft)

@ArielJurkowski What file from the github repo do you run as your "local arjs js file"?

ArielJurkowski · 2023-08-05T05:50:25Z

@MarkEEaton

I needed a quick and dirty solution, so I just downloaded and edited the final js file directly. I used the 3.3.3 version of aframe-ar-nft.js (here). It seems the new master version has the code with navigator.mediaDevices.enumerateDevices() minified. That wasn't the case with version 3.3.3, but I think my solution should still work if you edit the js file properly.

MarkEEaton · 2023-08-05T19:44:31Z

@ArielJurkowski thanks for this! I got it working locally. I had to downgrade aframe to version 1.2.0. I will test it on some more devices next week.

intermosh · 2023-10-03T16:54:47Z

Thank u very much, this made it for me. At least in the device I'm testing. Had to downgrade to 1.2.0 as well.
I will test on more devices and share feedback.

jamess922 · 2024-11-04T01:40:21Z

@ArielJurkowski
Hi，ArielJurkowski ，I wanted to try your solution on the Arjs Hiro demo, but found that the two arjs js files were completely different（aframe-ar.js for Marker Based and aframe-ar-nft.js for Image Tracking）. Have you tested Hiro's demo? How should I modify the code?

kalwalt added the bug Something isn't working label Jun 27, 2021

jonomatusky mentioned this issue Feb 2, 2022

Wrong Camera Detection - Multiple camera phone zoomed view issue hiukim/mind-ar-js#105

Closed

mateusmb mentioned this issue Jun 9, 2022

Fix the zoom issue on some devices mateusmb/ar-image-tracking#4

Open

Sov3rain mentioned this issue Aug 1, 2022

fix: too much zoom on multi camera devices for tracking hiukim/mind-ar-js#203

Closed

b2renger mentioned this issue Jan 21, 2024

Camera selection on mobile device b2renger/Irreverent#1

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Weird Wide Angle effect with AR on some phones #303

Weird Wide Angle effect with AR on some phones #303

Whystler commented Jun 24, 2021 •

edited

Loading

kalwalt commented Jun 27, 2021

Whystler commented Jun 27, 2021 •

edited

Loading

espresso95 commented Nov 8, 2021

alejo4373 commented Nov 8, 2021

ArielJurkowski commented Nov 21, 2021 •

edited by kalwalt

Loading

darde commented Dec 20, 2021

darde commented Dec 21, 2021

ArielJurkowski commented Dec 21, 2021 •

edited

Loading

MarkEEaton commented Aug 4, 2023

ArielJurkowski commented Aug 5, 2023 •

edited

Loading

MarkEEaton commented Aug 5, 2023

intermosh commented Oct 3, 2023

jamess922 commented Nov 4, 2024

Weird Wide Angle effect with AR on some phones #303

Weird Wide Angle effect with AR on some phones #303

Comments

Whystler commented Jun 24, 2021 • edited Loading

kalwalt commented Jun 27, 2021

Whystler commented Jun 27, 2021 • edited Loading

espresso95 commented Nov 8, 2021

alejo4373 commented Nov 8, 2021

ArielJurkowski commented Nov 21, 2021 • edited by kalwalt Loading

darde commented Dec 20, 2021

darde commented Dec 21, 2021

ArielJurkowski commented Dec 21, 2021 • edited Loading

MarkEEaton commented Aug 4, 2023

ArielJurkowski commented Aug 5, 2023 • edited Loading

MarkEEaton commented Aug 5, 2023

intermosh commented Oct 3, 2023

jamess922 commented Nov 4, 2024

Whystler commented Jun 24, 2021 •

edited

Loading

Whystler commented Jun 27, 2021 •

edited

Loading

ArielJurkowski commented Nov 21, 2021 •

edited by kalwalt

Loading

ArielJurkowski commented Dec 21, 2021 •

edited

Loading

ArielJurkowski commented Aug 5, 2023 •

edited

Loading