Can't View Model in AR from Cordova App Persistent Storage

[lewispipeandpiper]

My Code

ar.handlebars

        <model-viewer id="ar_view" src="" ar ar-modes="webxr scene-viewer quick-look" environment-image="neutral" poster="img/logo.webp" shadow-intensity="1" camera-controls xr-environment>

            <button id="ar-failure">
                AR is not tracking!
            </button>

            <div id="ar-prompt">
                <img src="img/hand.png">
            </div>

            <button slot="ar-button" class="button-general" id="ar-button">View in AR</button>

            <div id="ar_toolbar" role="navigation">
                <div class="left">
                    <a class="button-general" id="change-model-button"/>Change View</a>
                </div>
            </div>

        </model-viewer>

ar.js

  if( app.PHONEGAP ) {
			if( typeof(app.CACHE.TANKS[this.index].file) == "undefined") {
				// Move into .gltf persistant storage
				settings = { fileName: identifier + '_Astronaut.gltf', fileLocation: 'assets/' + identifier + '_Astronaut.gltf', index: this.index, type: 'gltf' };
				app.FILE.move_gltf(settings);
				
				// Move into .bin persistant storage
				settings = { fileName: identifier + '_Astronaut.bin', fileLocation: 'assets/' + identifier + '_Astronaut.bin', index: this.index, type: 'bin' };
				app.FILE.move_bin(settings);
			}

			// Show model from storage
			setTimeout(function() { 
				app.DOM.content.find("model-viewer").attr("src", cordova.file.dataDirectory + identifier + '_Astronaut.gltf');
			}, 100);
		}
		else {
			app.DOM.content.find("model-viewer").attr("src", 'assets/' + identifier + '_Astronaut.gltf');
		}
	},

Description

I'm trying to load a model from persistent storage within a Cordova application. I can get it as far as loading the model in the normal viewer, but when trying to load the model into AR, it causes SceneViewer to crash and say "Couldn't load object". I've tried just about every path and route, but am very confused as I am able to view the model in the normal 3D viewer just fine with the same path from persistent storage.

When trying to use the cordova path cdvfile://localhost/files/1350SLBCHEMICAL_Astronaut.gltf, SceneViewer simply just crashes whenever it is trying to detect an area for placing the model (which is normally immediately). The same also happens with file:///data/user/0/uk.co.tuffaar.app/files/1350SLBCHEMICAL_Astronaut.gltf.

I have tried to debug from the SceneViewer, but when using chrome://inspect, it does not show up, so I am unable to see any error messages. No error messages show when viewing the app either.

Is this actually possible to do, or is this a limitation of SceneViewer?

As an alternative, I have tried hosting the files on a web server, so that they get cached on device, which all works fine for the 3D viewer (even offline). But the limitation of SceneViewer not allowing offline usage has killed off this idea too.

For this app it is important to be able to use both 3D and AR offline after having downloaded the files one time.

Version

• model-viewer: https://unpkg.com/@google/model-viewer/dist/model-viewer.min.js

OS

• Android
• iOS
• Linux
• MacOS
• Windows

AR

• WebXR
• SceneViewer
• QuickLook

[elalish]

Have you tried WebXR (our AR default)? It's our default precisely because it is more reliable and faster than scene viewer (and does not redownload the model, so your case should definitely work). Scene Viewer is a separate app, hence why chrome can't tell you anything about it. Likewise, if you need a cookie or some kind of authentication to download the file, that context won't be accessible from Scene Viewer.

[lewispipeandpiper]

<model-viewer id="ar_view" src="TEST/1350SLBCHEMICAL_Astronaut.gltf" ar ar-modes="webxr" environment-image="neutral" poster="img/logo.webp" shadow-intensity="1" camera-controls xr-environment>
When trying to just use ar without modes, like the example scene from the docs, the "View in AR" button disappears as though AR is not supported without SceneViewer.

Both of the methods above still work with the normal 3D view as expected.

Even when I get the AR button to show using what's below, it still opens with sceneViewer.

<model-viewer id="ar_view" src="TEST/1350SLBCHEMICAL_Astronaut.gltf" ar ar-modes="webxr" environment-image="neutral" poster="img/logo.webp" shadow-intensity="1" camera-controls xr-environment>

[elalish]

For the WebXR version, do you see any info in the JS console? One common mistake is that WebXR requires HTTPS, so any HTTP site will fail. Also if it's in an iframe you need to set the xr permission.

[LTeather]

For the WebXR version, do you see any info in the JS console? One common mistake is that WebXR requires HTTPS, so any HTTP site will fail. Also if it's in an iframe you need to set the xr permission.

Cordova runs as though it is a native app and I believe only uses the http protocol. So using WebXR may not be possible.

[lewispipeandpiper]

For the WebXR version, do you see any info in the JS console? One common mistake is that WebXR requires HTTPS, so any HTTP site will fail. Also if it's in an iframe you need to set the xr permission.

Cordova runs as though it is a native app and I believe only uses the http protocol. So using WebXR may not be possible.

I was wrong in my last post so ignore that.

I am now at a point where the hosted model and the app itself is now running on https. Though the "view in AR" button is still not appearing.

The console shows no errors or information when loading or interacting with ModelViewer.

The app is running through https://localhost/index.html (Cordova App) on device.

        <model-viewer id="ar_view" src="https://website.com/downloads/1350SLBCHEMICAL_Astronaut.gltf" ar ar-modes="webxr" environment-image="neutral" poster="img/logo.webp" shadow-intensity="1" camera-controls xr-environment>

            <button id="ar-failure">
                AR is not tracking!
            </button>

            <div id="ar-prompt">
                <img src="img/hand.png">
            </div>

            <button slot="ar-button" class="button-general" id="ar-button">View in AR</button>

            <div id="ar_toolbar" role="navigation">
                <div class="left">
                    <a class="button-general" id="change-model-button"/>Change View</a>
                </div>
            </div>

        </model-viewer>

This results in the AR button not being shown.

This behaviour of having the "View in AR" button hidden happens on Both an iPhone 12, and a mid-range android device.

I have also tried the adding the polyfill from the examples incase of a dependency missing, but this also did not help.

[elalish]

Well, WebXR isn't supported on iPhones yet, so the button should be hidden there (add "quick-look" to your ar-modes to get iOS support). What is your mid-range Android device? If you remove your custom button, does the default AR button still fail to show up? Are you sure this isn't running inside of an iframe or webView or something? I don't know anything about how Cordova works.

[lewispipeandpiper]

The Android Device is a Motorola G30 running latest version of Android and fully updated apps.

Cordova is a way of building a web application for a mobile device. The way it is structured is still like a normal website, and the ModelViewer DOM element is not within an iFrame. My code would work the same way had it been on a regular website.

Is there any reason other than the device not being supported that the button would not appear? I have tried removing the custom button, and still no button appears. When the scene-viewer option is given, the button shows which leads me to believe that WebXR is not supported on the device? Though this has been tested on a Galaxy S21, and still had the same results.

[elalish]

Ah, I think it is in fact embedded in a WebView: https://cordova.apache.org/docs/en/10.x/guide/hybrid/webviews/

WebXR is not supported in WebView, only in the newer WebLayer, so that's likely the issue. So Scene Viewer is a better choice in this case; do you get the same unsupported error in a vanilla website on this phone? You can try copying this example: https://modelviewer.dev/examples/augmentedreality/#sceneViewer. You could also try some of our models to see if it's simply a Scene Viewer error with that particular model.

[lewispipeandpiper]

Ah, I think it is in fact embedded in a WebView: https://cordova.apache.org/docs/en/10.x/guide/hybrid/webviews/

WebXR is not supported in WebView, only in the newer WebLayer, so that's likely the issue. So Scene Viewer is a better choice in this case; do you get the same unsupported error in a vanilla website on this phone? You can try copying this example: https://modelviewer.dev/examples/augmentedreality/#sceneViewer. You could also try some of our models to see if it's simply a Scene Viewer error with that particular model.

Okay, makes sense why it wasn't working despite my best efforts. Thanks for identifying that.

When viewing the site in browser, WebXR works fine, but when in an iFrame on the app it does not. Am I missing something as it still does not work despite me allowing spatial tracking as per the docs.

<iframe src="https://pipeandpiper.co.uk/dl/index.html" title="Model Test" allow="xr-spatial-tracking *" frameborder="0" style="width: 100%; height: 100%"></iframe>

I have also tried putting the * in '*', to no avail.

only scene-viewer works. Am I missing something? Reading the docs, I believe I have everything.

[elalish]

Hmm, it does look like you've put the right permissions on the iframe. Can you paste a repro link (with just webXR mode so we can see the button doesn't show)? @klausw Do you have any thoughts on why this iframe would fail to allow WebXR?

[lewispipeandpiper]

Hmm, it does look like you've put the right permissions on the iframe. Can you paste a repro link (with just webXR mode so we can see the button doesn't show)? @klausw Do you have any thoughts on why this iframe would fail to allow WebXR?

Is it not at all possible to serve SceneViewer a model from a locally hosted device/location?

I can access https://localhost/assets/1350SLBCHEMICAL_Astronaut.gltf through Chrome inspect just fine, so I would have thought that SceneViewer would have been able to pick it up also. Being sent through https I thought this would have been fine.

the exact error SceneViewer gives me is:
Couldn't load object
Looks like there is something wrong with this object.

I have checked my model, and it has no errors, and have also tried it with gltf & glb formats.

This is my last idea as I really need a solution where models are able to be viewed in AR offline within a Cordova application. I don't want to use Wikitude due to the enormous costs involved.

[elalish]

Can you link the model here so we can repro? If you copy models from modelviewer.dev, do those fail in the same way?

[lewispipeandpiper]

Can you link the model here so we can repro? If you copy models from modelviewer.dev, do those fail in the same way?

This is using the low poly Astronaut from ModelViewer. But it happens with any model that I use.

[elalish]

This is my last idea as I really need a solution where models are able to be viewed in AR offline within a Cordova application. I don't want to use Wikitude due to the enormous costs involved.

@tpsiaki I vaguely recall a restriction on local serving in Scene Viewer. Do you know a way to accomplish this?

[teacoat]

I am also having issues with the AR button in capacitor, @lewispipeandpiper did you find a solution?

[lewispipeandpiper]

I am also having issues with the AR button in capacitor, @lewispipeandpiper did you find a solution?

Sadly my only solution was for the app to go to the browser on a webpage which we had set-up to go straight into AR. It's not a pretty solution, but it was the only one that was found.

I haven't found an alternative either, ModelViewer seems to be it really for what's good, cheap (free), and available.

ModelViewer seems to have a lot of trouble with being in any sort of WebView on mobile, and really struggles when you want to use AR. For just 3D models, it is completely fine.

[elalish]

Indeed, QuickLook is not supported (yet) on iOS WebViews, so there's nothing we can do about it. But it is supposedly coming (@grorg at Apple). WebXR is also not yet supported on WebViews, but hopefully soon.

[teacoat]

@lewispipeandpiper clever solution, will work in the interim for sure! Thank you

[teacoat]

I am also having issues with the AR button in capacitor, @lewispipeandpiper did you find a solution?

Sadly my only solution was for the app to go to the browser on a webpage which we had set-up to go straight into AR. It's not a pretty solution, but it was the only one that was found.

I haven't found an alternative either, ModelViewer seems to be it really for what's good, cheap (free), and available.

ModelViewer seems to have a lot of trouble with being in any sort of WebView on mobile, and really struggles when you want to use AR. For just 3D models, it is completely fine.

Do you have an example of how you would set it up to go straight to AR mode?