Cordova mobile app compatibility

[benallfree]

Description

This PR provides experimental Cordova mobile app compatibility.

[dmarcos]

Any way to detect cordova implicitly?

[dmarcos]

'cordova' in window ?

[benallfree]

@dmarcos Ready for another review, but please do not merge yet. Still testing in conjunction with AR.js.

[dmarcos]

What version of Chrome is Cordova shipping with? Can someone build apks for Android that use WebXR?

[benallfree]

I will have to check. I know for iOS I need to use some polyfills that make native calls but mimic web APIs like getUserMedia. I think it also needs a WebVR polyfill. I’m not extremely familiar with the VR support on Android but I’ll check.

Does the answer alter anything needed for this PR?

[dmarcos]

I’m not educated on Cordova. If someone wants to package a WebXR experience for Rift / SteamVR / Quest we would need a way to enter VR automatically. It seems you are focused on Magic Window

[benallfree]

iOS WebKit has some missing APIs that even Mobile Safari allows. So for embedded WebKit like Cordova uses, polyfills are needed even though the browser is capable. Kind of frustrating but that’s the status currently.

I’ll keep researching though.

[benallfree]

I am curious: Has any research been done on A-Frame + React Native? supermedium/aframe-react#35

[dmarcos]

I am curious: Has any research been done on A-Frame + React Native? supermedium/aframe-react#35

I haven't heard of any examples. In order for A-Frame to work on React Native a WebGL backend and DeviceMotion APIs would be needed. Not sure if that's possible at the moment.

What is the status of this PR?

[benallfree]

What is the status of this PR?

I am working on getting some A-Frame demos working in a sample Cordova project living here: https://github.com/benallfree/arjs-mobile-app-samples

My ultimate goal is to get AR.js working too, but right now it appears that there are a few rendering issues with A-Frame that I am looking into.

[benallfree]

@dmarcos For example, the Hello WebVR looks like this on the iPhone simulator running Cordova with an embedded WebView:

[benallfree]

@dmarcos This is roughly what I need to do in Cordova (dynamically load A-Frame) because WebKit is not available/initialized until after Cordova's deviceready event: https://glitch.com/~wide-puppy-po6d3fgfy

That demo works on Mobile Safari, but does not work when embedded in Cordova.

[dmarcos]

Make sure you enable antialias

[dmarcos]

Notice that ar.js imposes additional requirements on top of A-Frame like access to the camera. Making A-Frame hello-world example work is the first step as you are doing.

[benallfree]

We are getting closer. This is running in Cordova on a physical iPhone device using device motion to pan around. The iOS simulator chokes badly.

You can see there are a few bordering issues and the zoom appears wrong but it's getting close. I will keep you updated in this thread :)

PS - the frame rate of the GIF makes it look choppy, but the real experience is very smooth.

[dmarcos]

Great work and thanks for the update! Much appreciated.

[benallfree]

It looks like these lines are getting executed but Browserify is not injecting the <script> tags:

if (utils.device.isBrowserEnvironment || window.cordova) {
  require('./style/aframe.css');
  require('./style/rStats.css');
}

Looking into it further. Any ideas?

[dmarcos]

It looks like these lines are getting executed but Browserify is not injecting the <script> tags:

if (utils.device.isBrowserEnvironment || window.cordova) {
  require('./style/aframe.css');
  require('./style/rStats.css');
}

Looking into it further. Any ideas?

Yes, those are executed by browserify at build time. What's the problem with those styles in Cordova?

[benallfree]

@dmarcos You've persuaded me to a very surgical approach.

The only remaining issue is that the A-Frame canvas seems to take full screen control regardless of the parent container size, but the VR button is relative to the parent container size.

The CSS mod fixes that, but I believe it might be an appropriate mod for non-Cordova as well. see glitch

[dmarcos]

Oh wow 😀 This is really surgical. Well done! You no longer need an a-frame fork with these changes?

[benallfree]

We will not need a separate aframe fork or even a specialized Cordova plugin at this time.

The 'offline' requirement caused a lot of complexity, thank you for resisting it.

I still think it would be wise to consider making A-Frame's bootstrap more configurable by refactoring the asset base URL and polyfillConfig settings, but that can be another project for another time.

Please let me know what you think of the CSS issue. I don't think it belongs in this PR but there definitely appears to be an unintended side effect if the is nested in a

.

[benallfree]

Ok this one is ready to be merged & closed.

Thank you for all the assistance. I created a separate issue (#4517) for the CSS thing I found.

[dmarcos]

Thanks for the patience. This work is super appreciated.

Do you know what version of Chromium ships with? WebXR ships in Chrome since m79. We've seen people asking for ways to package A-Frame apps and distribute them on Steam / Oculus Store. I think we're pretty close to get something usable but not a Cordova expert

[benallfree]

Interesting, do you have anyone you can put me in touch with?

I don't have an Android device so I haven't done any Android testing with this, just iOS.

This is definitely just a first effort, I expect we will find out lots of little details once someone tries to ship an app :)

[dmarcos]

I believe @arpu expressed interest in the past on packaging A-Frame apps (sorry if my memory is not accurate).

[dmarcos]

Thanks for this. Sorry it took so long to merge.