Stretched face on Android Chrome

[marcusx2]

If you try out this example on your Android Chrome, your face will be stretched vertically.

I already created a bug report on the threejs github, but one of the repo managers asked that the issue should be handled here instead...

[xavierjs]

Hi,

Does the issue still occur with the latest commit?

[marcusx2]

Unfortunately, no. In the JeelizResizer.js, in the apply_sizeCanvas function, the css width and height inside _if (isApplyCSS){ overwrites what you have in the styleFullScreen.css though. I tried removing that if but it didn't solve the issue either.

On another note, the issue doesn't happen on iOS Safari. It seems to be a problem only on Android.

[xavierjs]

I don’t have an android phone to test and indeed there is no issue with iOS. Can you try tinkering with the css to see if it solves the issue ? You can also mail me a device with the issue.

[marcusx2]

I don’t have an android phone to test and indeed there is no issue with iOS. Can you try tinkering with the css to see if it solves the issue ? You can also mail me a device with the issue.

I'm using a Samsung S8. I tried tinkering with the CSS to no avail. However here's some information that might help. The following CSS makes an HTLMVideo element fullscreen while keeping the aspect ratio (the transform scale is just to flip horizontally). It works on Android and iPhone.

_videoElement.setAttribute('style', 'width: 100vw;height: 100vh;object-fit: cover;position: absolute;top: 0;left: 0; transform: scale(-1, 1);');

I tried using the css above on the styleFullScreen.css but it didn't work though. It's like the canvas ignores the style or it doesn't work for some other reason. Maybe some other math that screws up the style or something...

[marcusx2]

@xavierjs I found out what the issue is and how to fix it, but not the proper solution. The problem is the apply_sizeCanvas function and the _overSamplingFactor, which results to 4. If I hardcode 2 instead(3 also doesn't work), the webcam feed shows up properly.

function apply_sizeCanvas(width, height){
    _whCanvasPx = [
      Math.round(2 * width),//use 2 instead of 4 (overSamplingFactor)
      Math.round(2 * height)//use 2 instead of 4 (overSamplingFactor)
    ];

	console.log(_overSamplingFactor);
    // set canvas resolution:
    _domCanvas.setAttribute('width',  _whCanvasPx[0]);
    _domCanvas.setAttribute('height', _whCanvasPx[1]);
	
	console.log('_whCanvasPx[0]: ' + _whCanvasPx[0] + ' _whCanvasPx[1]: ' + _whCanvasPx[1]);
	console.log('width: ' + width + ' height: ' + height);

    // canvas display size:
    if (_isApplyCSS){
      _domCanvas.style.width = width.toString() + 'px';
      _domCanvas.style.height = height.toString() + 'px';
    }
  }

So, it seems that different devices support different oversampling factors, or maybe it's just android (or even, just some Androids). If you detect that it's Android and set the oversampling factor to 2, it will solve the issue. Ideally the code should be able to figure out what is the max sampling factor the device supports though.

EDIT

I can see that the overSamplingFactor is set to the device's pixel ratio, which is 4 on my Android Chrome. I tried hardcoding the overSamplingFactor to 4 on my Chrome for Windows as well to see if the problem also happened (on my windows chrome the device pixel ratio is only 1.25), but the problem didn't happen. However, on my iPhone Safari the device pixel ratio is 3, and when I tried to force it to be 4, the face didn't become stretched, but the app blew up with an error and closed. So it's safe to say that 4 for the oversampling factor is not a number mobile devices like lol. I guess if you always set the oversamplingfactor to devicePixelRatio/2, it will solve the problem, although not ideal.

[xavierjs]

Hi,

Thank you for these precisions. It is really weird that it comes from the oversamplingfactor.
If it is the case it really means that there is an internal browser error.
Are you sure that the issue is not intead:

• at the initialization, JeelizResizer will set the canvas resolution as window.innerWidth * window.innerHeight. There is the URL bar
• then the URL bar is hidden by the browser. window.innerHeight should increase. But for some reason no window.resize event is triggered. The canvas resolution stays the same (wrong value for the height). But the canvas is stretched in fullscreen

[xavierjs]

I have pushed a potential fix. It forces canvas resizing 500 milliseconds after init in JeelizResizer here:

jeelizFaceFilter/helpers/JeelizResizer.js

Line 262 in b133c79

window.setTimeout(on_windowResize, 500);

Can you try it? And if it does not work increase the 500 timeout to 2000 for example and wait 2 seconds?

Thanx a lot!

[marcusx2]

Unfortunately, it did not work. I tried with 2 seconds as well. I also made a few more discoveries. It seems that even if I set the oversampling factor to 2, I still get a small distortion on Android Chrome. However, the issue does not happen on Firefox for Android. Don't know about other browsers, but it seems that this problem is browser specific. It just looks like on some browsers the canvas attribute width/height needs to be the same as the css width/height, otherwise it distorts. Maybe there's a fix to this, but it looks like it's one of those CSS issues where it works on one browser, but doesn't on the other, and you need to find a common CSS that works across all browsers, which can be very difficult.

[xavierjs]

Hi
This is really weird. Can you try with the latest commit? I blocked the page display scale using a specific META tag.
Currently the canvas attribute width and height (i.e. the resolution) is strictly proportional to canvas CSS attribute (the proportionality factor is the devicePixelRatio), so I don't understand what can go wrong.

[marcusx2]

The meta tag code finally fixed the issue, without the need of this window.setTimeout(on_windowResize, 500); (doesn't do anything, can be removed).

Currently the canvas attribute width and height (i.e. the resolution) is strictly proportional to canvas CSS attribute (the proportionality factor is the devicePixelRatio), so I don't understand what can go wrong.

Yes, this is true. I don't understand why it stretches either, but the meta tag solved the issue. Make sure to also test on all the iPhone browsers as well!

One question, does the meta tag prevent the apply_sizeCanvas from doing its job though? So the oversampling factor won't do anything?

[xavierjs]

It does not change the apply_sizeCanvas issue. Maybe there were some zoom in or out with different values on X and Y axis. It is really weird... Thank you for your help, I have added the fix to another demo.