How do we use XXX WebGL library with HoloJS?

[trusktr]

Suppose I like Babylon, or PlayCanvas, or Three.js, or my own custom WebGL engine.

How do I use it in HoloJS?

I noticed HoloJS has it's own shaders, which makes me think I have to use HoloJS instead of one of those other libraries, as an alternative to those.

[Almost-Done]

ThreeJS already works with HoloJS - one of the sample apps is a ThreeJS app. If there's interest we can provide a BabylonJS sample as well.

HoloJS does not have it's own shaders. HoloJS only provides a WebGL implementation with auto-stereo rendering on the HoloLens. The issue you mentioned is related to how HoloJS re-writes the shaders submitted by ThreeJS, BabylonJS etc. to make them auto-stereo on the fly: only the first shader is modified while the rest are left unchanged. This causes problems with materials in ThreeJS since each material has it's own shader.

[trusktr]

So I'm interested in doing graphical rendering with my own lib (infamous), which has some architectural similarities to A-Frame: namely my lib and A-Frame both define custom elements which can be manipulated, and which are used to declaratively define 3D scenes. Both libs read changes to the DOM elements and render into WebGL based on that.

I have a feeling this won't work with HoloJS because it doesn't have a DOM AFAIK.

I'm thinking of making an option in my lib to turn off the declarative DOM API, leaving the imperative JavaScript API available. At that point, then maybe HoloJS will be able to modify my shaders and hopefully it'll just work?

[trusktr]

What @sjando suggested in #24 I think is a good idea: to expose an API that is replica of WebVR, so that WebVR apps are easy to run in HoloJS too.

Here's a basic example, showing the that WebVR apps decide how to render each eye.

It's more involved for the app to render each eye, but nice because libs like A-Frame can abstract it away, and will work in the web AND HoloJS (assuming HoloJS can accommodate some form of DOM).

---

Another idea is to just make a full-view Edge instance (WebVR work in it). This would probably be easier than you having to maintain a separate VR implementation, when Edge already has it, and has the WebVR API, and it would mean much better compatibility with existing WebVR apps.

[Almost-Done]

To get web standard stereo rendering on the HoloLens, we picked auto-stereo through shader manipulation as the quicker way. WebVR is the other option and they can coexist.

For your case, if you can work around the lack of a full DOM, auto-stereo should work with the single shader caveat. We can provide some help if you run into issues. The DOM could be implemented fully in script (parts of it are already stubbed in script) but its a tall task to get to a level where JQuery runs.

[trusktr]

So far I only have a single shader. I suppose I can boolean uniform all the features, then turning them on/off can basically make different shaders. I'm relatively new to GL though. From what I've heard, the GLSL compiler actually compiles different outputs behind the scenes for each possible permutation of code paths based on the logic in a shader, so for example a shader with a single conditional statement actually results in two compiled outputs. Do you know if something like this is true?

[trusktr]

Just got Windows 10 at work. Going to try infamous on this soon. I'm thinking to make a single mondo shader with conditionals (possible put together with glslify), and for other environments dynamically construct the shaders without conditionals.

What requirements must the shader have, for higher chance of working in HoloJS?

[Almost-Done]

Multiple shaders are now supported with auto-stereo.

If you run into issues with your shaders not working with auto-stereo, please let us know and we'll investigate.

I'll open a new issue to enable rendering from scripts without the auto-stereo option.

[ranveeraggarwal]

ThreeJS already works with HoloJS - one of the sample apps is a ThreeJS app. If there's interest we can provide a BabylonJS sample as well.

It would be great if you can provide a BabylonJS sample. We have an existing VR application on BabylonJS and we plan to use HoloJS to port it to HoloLens. Can you help us out?