All materials render black on some mobile devices

[Supernuija]

Testing on various mobile devices (phones and pads) indicates that depending on the drivers, GPU:s and browser versions three.js may render all objects pitch-black. Mobile device having the same GPU/processor may behave differently depending on the phone model and year; older may work while newer doesn't. The same device may operate correctly with Chrome but not with Firefox etc.    

The solution to this would be using material precision: 'mediump' on all mobile devices as a default value. Solving this has first been suggested in 2018 by @Mugen87 mrdoob/three.js#14570. These examples are from another thread, and I can confirm that this is an issue. Testing various devices using these two tests below confirmed it. 
 
This will show a white cube on all devices: 
 var material = newTHREE.MeshPhongMaterial( { precision: 'mediump' } ); 
https://jsfiddle.net/f2Lommf5/6721/embedded/result
 
This will not show a white cube on some devices, rendering it pitch-black 
 var material = new THREE.MeshPhongMaterial( { precision: 'highp' } ); 
https://jsfiddle.net/f2Lommf5/6722/embedded/result
 
IHMO is very important that 3D graphics operate flawlessly by default on all devices capable of running them, despite the bad drivers, operating systems, or browser versions.
 
Is anyone interested to investigate a solution to this? I am willing to test and confirm how possible solution works with various devices. 

[X-Ryl669]

Your links don't work.

[Supernuija]

There was an invisible space in the address, now fixed.
 

[X-Ryl669]

I'm not a developer, so I can't assert what's correct, but IMHO, this is a per-device issue, at worst, a ThreeJS issue, not a end user software issue. If the device does not support highp but claims it does, there's nothing that can be done to solve this.
Switching to mediump everywhere breaks almost all materials rendering and this decision should be done by ThreeJS and not here. You probably should ask them to have a fallback path when the device is not working with the higher precision.

[kovacsv]

That's a good idea to detect somehow if the device can't handle highp precision. In this case it can be modified here, because it can be set as a material parameter. The big question is how to detect such devices.

[Supernuija]

I was thinking that some kind of simple check could be applied, where known colour object would be drawn on specific location (like the white cube) and then checked if the result on that pixel area is coloured black then parameter would be changed to mediump. How it is done codingwise, is up to someone who knows better. A simple check before loading the actual model.

[Supernuija]

One quick test could be just changing https://threejs.org/docs/#api/en/renderers/WebGLRenderer.precision - value from "highp" >> "mediump" and seeing how everything looks and operates. This issue has been recognised by three.js but it hasn't been fixed yet. It seems that this setting will not affect desktop, that uses highp no matter what the setting is. That's why even testing it is quite difficult, because it can't be emulated with desktop.

Do you think that could try that setting for a day or two on your server and I could test if the problem is solved by that, and how it affects to the image quality and accessibility (all materials turning black issue) in various mobile devices?

[kovacsv]

As I understand the documentation the default value is highp if the device supports it. So setting it to mediump will force it to mediump even if the device supports highp. I don't think it's a good idea.

The problem is with the devices that say they support highp, but the support is buggy. So I think only a predetection should work, but I don't know if it's feasible.

[Supernuija]

Three.js is considering changing the default value to mediump but this requires first a shader fix regarding a minor lighting error using mediump which may take forever before anyone takes on it (issue opened already 2018), so we are back into square one. They were not eager to include a pre-test for figuring out whether the driver is buggy or not. As said, no one really knows how many devices on market have this render all materials black issue. Fact is that no manufacturer will care about any older phone drivers and especially regarding this kind of very trivial fix.

What I was asking from you was that if you could change the value on your server (or some other test server if you have any) for a quick test so that it forces everything to mediump (notice that desktop is not affected by it, because it automatically always override this value to highp), so I could test on my end how it looks with mobile devices in reality using various devices: the ones that are capable to display highp and the ones that are not but display all materials in black atm, phones and tablets, android and ios.

[kovacsv]

Here you have a version with precision set to mediump: http://kovacsv.hu/mediump/

Please let us know the results!

[Supernuija]

Thanks a lot for the super quick action, I will give it a go and report as soon as I get all tests done!

[Supernuija]

Just got all tests run and it was exactly as I suspected =) Mediump works fine with all devices and highp doesn't.

Here are all test results, so you can see yourself, all browsers are the latest available for those devices.
https://drive.google.com/drive/folders/1DK5tLlnrlmSfWL1g5Nl8yejsEKAWFb4n?usp=sharing

There are now two test links on my server you can try yourself with different devices. At least I couldn't see any kind of difference in quality with these two settings displaying the same model. You can try these test sets and see yourself.
If you have some models that display differently with those two settings I would be eager to see and test them. If there is no visual difference in quality, I would strongly recommend changing the default value precision to guarantee the compatibility with all kind of devices.

https://quique.fi/talotesti.html - this one is with your site's normal highp setting
https://quique.fi/talotesti2.html - this one is with your test site's mediump setting

[kovacsv]

Thanks for the check! The question is how can we make sure that it doesn't affect desktop at all. I've checked the three.js source code, and it generates the shaders with mediump even on desktop if you specify it that way. But maybe MeshPhongMaterial (which is used) doesn't affected by this change. So to do this fix I'd like to be completely sure that it won't affect desktop at all.

[Supernuija]

In my link where the test results are, in root folder there are two files that have both highp and mediump renderings on Desktop.
They are identical with all other renderings where materials are displayed. As a matter of fact one can't see any difference between any of those files, no matter what. That's why I would suggest two alternative approaches to this issue.

1. Keep the mediump test site still open and ask for other users to test their designs to see if there is any visible difference between the test site rendering and normal site rendering, and in the other users find the same results as I did (no difference) then change mediump to be the default value for all devices (desktop and mobile)

2. add an extra switch to your site that can be inserted to the model loading string that specifies desired precision level to use mediump in case one wants to make sure that their models will display correctly on every mobile device without the risk of rendering materials black.

I am confident that this render black issue will not go away and also that three.js team likes rather to develop all kind of more exciting new visual features than to solve something that is not actually their fault but someone elses (manufacturers and operators not making drivers correctly obeying standards, lacking proper testing and QA)

For the end user this ia still a problem that prevents totally using the viewer in the worst case and there is nothing they can do about it. So it's 100% game stopper.

[Supernuija]

@kovacsv How did you like my 2 proposals mentioned above?

[kovacsv]

I've decided that the next release will have mediump as default.

Reasons behind the decision:

• My internal test system didn't show any differences on desktop with mediump.
• As far as I understand MeshPhongMaterial - which is used - doesn't have problems with mediump.
• It seems that desktop browsers will just treat mediump as highp.
• It can be reverted any time if there are issues.

[kovacsv]

Released in 0.7.10 version.

[Supernuija]

Thanks for making that change! I am sure that this change will make your excellent 3DViewer more popular on the long run, when all models will display materials, no matter what kind of mobile devices are used, and how well HW/SW manufacturers have done their work. =)

[kovacsv]

Reopen, because the fix caused a new bug: #75

[kovacsv]

@Supernuija, I had to revert back the changes, because it caused other issues in devices that had no problems before. Now I think the only way is to detect wrong hardware somehow. I've created a small experiment that - in theory - can detect the case when materials are black.

Could you please try this link on devices where the error happens?
http://kovacsv.hu/other/invalid_driver_detector.html

It should write "Invalid driver detected" on the devices where the problem occurs, "Everything is OK" otherwise.

[Supernuija]

I can confirm that your detection works perfectly!

In every mobile device/browser combination it gave the correct answer. Applying this automatic detection to change highp to mediump will solve this issue for good.

Three.js should investigate thoroughly your detection method to figure out how to fix their end permanently, too.

Awesome work, you really know your stuff !! When do you think you could apply this fix into your release?

[kovacsv]

I've added the detection, @Supernuija if you have time for a final check, please check this url:
https://kovacsv.hu/mediump/

[Supernuija]

That detection works perfectly!

I did test it with all devices/browsers that are known to fail, and also compared model loading time perf between current version and this test version, and couldn't see any difference.

That version is ready to be applied imho, thanks for your excellent work to solve this issue!!

Your detection method should also be included to three.js, so that this already 3 year old bug would move permanently into history books on worldwide scale.

[kovacsv]

Thanks for the checking, I'll release it today. For the record, the checking code is not something I proud of. It just generates a red plane, and check if it's really red. A more elegant solution would be great, but at the moment it does the job.

[kovacsv]

Fixed in 0.7.12 version.

[Supernuija]

Great that you already released the fixed version!

For the time being your solution is the best on the market ; ) - and any working solution is better than no solution at all.

As I have mentioned before, in these kind of cases where different parties like HW and SW manufacturers just blame each other for something that doesn't work, and users suffer the consequences without any possibility to do anything about it, any working solutions are worth of gold.

I perfectly understand why three.js is not eager solving this issue, and rather wish that it would go into history when ageing devices eventually vanish from the market at some point.

I am not sure if this is related to the same thing, but just recently someone had this issue: pmndrs/react-three-fiber#1388 - which might indicate that this problem persists even in new devices, if it has to do with the same bug.

The fact is that new devices will come to market all the time, and this webgl side is still so new that QA side regarding it's functionality drags million miles behind. We can't control how well the others do their work, just our own end. Sometimes patching might be the only reasonable thing to do.