Permalink
Browse files

more on htmlmixer post

  • Loading branch information...
1 parent 74683eb commit 4c05b44da32ba6b1ee77d705981f3c1228cf03c4 @jeromeetienne committed Apr 23, 2013
@@ -6,7 +6,7 @@ comments: true
categories:
---
-Wouldn't that be cool if we were able to mix normal Web page in our webgl? To interact with them as we usually do? to viewthem, to click on them... To scroll, to hover or even to type in input tag.
+Wouldn't that be cool if we were able to mix normal Web page in our webgl? To interact with them as we usually do? to view them, to click on them... To scroll, to hover or even to type in input tag.
Oh yeah it would be so great! We, webgl people, are currently an isolated Island in the web world. Being able to mix with normal page would give us access to so much interactive content.
In this post, we gonna see how to do exactly this: how to seamlessly mix normal DOM elements in our webgl scene. They will actually appear as part of the 3d scene. Don't worry it is surprisingly easy with three.js.
@@ -19,11 +19,16 @@ In this post, we gonna see how to do exactly this: how to seamlessly mix normal
## Demo of a youtube browser mixed in WebGL
+{% img right /data/2013-04-23-closing-the-gap-between-html-and-webgl/navigation-bar-small.png %}
+
First let's see the result in action. Here is a demo I did to show all the videos I do for this blog. [Try it out](http://jeromeetienne.github.io/videobrowser4learningthreejs)!
-It shows a 3d scene with a tv set and three characters sitting on grass. The key point is on the tvset screen. This is an actual YouTube player. Not me emulating it, this is the real thing! You can access it anytime from the blog navigation bar.
-(See screenshot on the right)
+It shows a 3d scene with a tv set and three characters sitting on grass.
-Pretty cool no? Now let's see how to do this
+The key point is on the tvset screen. This is an actual YouTube player. Not me emulating it, this is the real thing! You can access it anytime from the blog navigation bar as
+you can see on the right.
+This
+[demo](http://jeromeetienne.github.io/videobrowser4learningthreejs)
+is pretty cool no? Now let's see how to do this.
## Let's Get Started
DOM elements are all about flat 2d rectangles. In
@@ -35,53 +40,68 @@ So let's try to map a
to a
[THREE.PlaneGeometry](https://github.com/mrdoob/three.js/blob/master/src/extras/geometries/PlaneGeometry.js).
Their position and rotation must match.
+The goal is to make them appear as one thus the viewer can't distinguish them.
So first, how to orientate a dom element, would you ask?
-Simple we gonna use a feature introduced by HTML5,
+Simple we gonna use a feature introduced by HTML5, called
[css 3D transformation](http://www.w3.org/TR/css3-transforms/).
Here are some [tutorials](http://www.html5rocks.com/en/tutorials/3d/css/)
[about](http://desandro.github.io/3dtransforms/)
[it](https://developer.mozilla.org/en-US/docs/CSS/Tutorials/Using_CSS_transforms).
css3d is done for this exact purpose, to position and rotate a DOM element in 3d.
+{% img right /data/2013-04-23-closing-the-gap-between-html-and-webgl/webgl-plane-small.png %}
+
+(TODO redo sreenshot with black grid)
+
Good News! three.js can already render things using this technology.
- It is called [css3drenderer](https://github.com/mrdoob/three.js/blob/master/examples/js/renderers/CSS3DRenderer.js).
+ It is called [THREE.CSS3DRenderer](https://github.com/mrdoob/three.js/blob/master/examples/js/renderers/CSS3DRenderer.js).
See [various](http://threejs.org/examples/css3d_molecules.html)
-[examples](http://threejs.org/examples/css3d_periodictable.html)
-[using](http://threejs.org/examples/css3d_youtube.html)
-[it](http://threejs.org/examples/css3d_panorama.html)
+[examples](http://threejs.org/examples/css3d_periodictable.html)
+of
+[its](http://threejs.org/examples/css3d_youtube.html)
+[usage](http://threejs.org/examples/css3d_panorama.html)
in three.js ```/examples```.
Now we need to put the same plane on both renderers, WebGL Renderer and CSS3D Renderer.
Here is the code for the plane in
-[WebglRenderer](https://github.com/mrdoob/three.js/blob/master/src/renderers/WebGLRenderer.js)
+[WebGLRenderer](https://github.com/mrdoob/three.js/blob/master/src/renderers/WebGLRenderer.js)
, wireframe with segments to see thru.
```javascript
+// create the plane mesh
var material = new THREE.MeshBasicMaterial({ wireframe: true });
var geometry = new THREE.PlaneGeometry();
var planeMesh= new THREE.Mesh( geometry, material );
+// add it to the WebGL scene
+glScene.add(planeMesh);
```
-Now that we got the plane in place, here is the code for the domelement in css 3d.
-Notice how we reference the same position and rotation as the ```planeMesh```
+Now that we got the plane in place, here is the code for the DOM element in css 3d.
+Notice how we reference the same position and rotation as the ```planeMesh```, thus
+they will move together.
```javascript
// create the dom Element
var element = document.createElement( 'img' );
element.src = 'textures/sprites/ball.png';
// create the object3d for this element
var cssObject = new THREE.CSS3DObject( element );
+// we reference the same position and rotation
cssObject.position = planeMesh.position;
cssObject.rotation = planeMesh.rotation;
// add it to the css scene
cssScene.add(cssObject);
```
+(TODO put a screenshot of the css object with an image behind the webgl plane)
+
All seems to go well.
-We got the same plane in css and webgl. now we need to see the rotated dom element behind the webgl plane
-So let's use webgl renderer and css3d renderer together.
-We use css on the page to put cssRenderer exactly behind webglRendererer.
-Thus they look the same to the viewer. Something along this line
+We got the same plane in css and webgl. Now we need to see the dom element behind the webgl plane.
+To do this, let's use webgl renderer and css3d renderer together on the same page.
+
+We use stylesheet to put css renderer exactly behind the webgl one.
+Thus they look the same to the viewer, as you can see on the right.
+Use the following line to obtain the same result.
```javascript
var cssRenderer = new THREE.CSS3DRenderer();
@@ -90,28 +110,31 @@ cssRenderer.domElement.style.position = 'absolute';
cssRenderer.domElement.style.top = 0;
```
-#### meta
-* (Screenshot of each steps)
-* lack of screenshot .... unclear
-* to review
+(TODO two screenshots with torus in normal material, one in front of the grid, one behind the grid)
-## Blending them together
We are in good shape but not yet done. We still need to make both react as if they were one.
-What happen if we add a 3d object in front of webgl three.plane? As you can see on the right, It looks Ok.
-What if we put this object behind it? Hmm not so good. The object is behind the Plane, good.
-But it is in front of the rotated dom element. It should appear as if it were behind but it doesnt.
-Why that? It is due to the webgl z-buffer.
-It displays our torus because it thinks the torus is the closest to the camera.
-It is not aware that our THREE.PlaneGeometry acts as a see thru to make our css3d visible.
-So that nothing behind thethe THREE.PlaneGeometry should be displayed.
-How to fix this, would you ask? We gonna use a tricky part of Webgl: the blending.
-
-When you are doing blending, we use a blend function to combine the colors from both the existing and the new fragments
-to make an entirely new fragments.
-What is it ? it is the way to dertermine the colors of a pixel (fragment in technical term) when you add a new pixel.
-So when doing blending, we use a blend function to combine the colors from both the existing and the new fragments
-to make an entirely new fragment.
-It is a weird beast using several WebGL calls and many equations. The total number of possibilities is scary :) An complete explanation of blending is way out of thescope of this post. For more detail, see
+What happens if we add a torus 3d object in front of webgl plane? As you can see on the c
+left, it looks Ok.
+What if we put this object behind it? Hmm not so good. As you can see on the right, the object
+is behind the Plane,
+but it is in front of the dom element. It should appear as if the torus were behind but it doesnt.
+Why that? It is due to the webgl
+[z-buffer](http://en.wikipedia.org/wiki/Z-buffering).
+
+It displays our torus because it thinks the torus is closer to the camera than the DOM element.
+It is not aware that our webgl plane should act as a see-thru to make our css3d visible.
+So nothing behind our webgl plane should be displayed.
+How to fix this, you would ask? We gonna use a tricky part of webgl: the blending.
+
+## Blending them together
+
+What is blending ? It is the way to determine the color of a pixel when you add a
+new pixel (fragment in technical terms).
+So when doing blending, we use a blend function to combine the colors from both the
+existing and the new fragments to make an entirely new fragment.
+
+It is a weird beast using several WebGL calls and many equations. The total number of possibilities is scary :)
+A complete explanation of blending is way out of scope of this post. For more detail, see
["WebGL Beginner's Guide"](http://www.amazon.com/WebGL-Beginners-Guide-Diego-Cantor/dp/184969172X)
from
[Brandon Jones](http://blog.tojicode.com/), a great book to start with raw WebGL.
@@ -120,8 +143,8 @@ To get a feel of blending, you can play with them in
The one which interest us is called ```THREE.NoBlending```.
When drawing the face, it will completely ignore the color below and set it to the color of the face.
-So if we put our face color to black aka 0, 0, 0 and opacity to 0, we gonna obtained what we want.
-The plane will act as a see thru to the dom element below. Here is how to init your material.
+So if we put our face color to black aka ```(0, 0, 0)``` and opacity to ```0```, we gonna obtained what we want.
+The plane will act as a see-thru to the dom element below. Here is how you initialize your material.
```javascript
var material = new THREE.MeshBasicMaterial();
@@ -131,13 +154,49 @@ material.blending = THREE.NoBlending;
// any mesh using this material will act as a see-thru to the css renderer
```
-* (Provide Screenshot for each step and alternative)
-* Then we are done ! We got a actual dom element seamlessly integrated in our webgl scene! Let's pet our back, i think this is an important step!
-* (Here put the paragraph in Webgl)
-
-
-
-
+Then we are done ! We got a actual dom element seamlessly integrated in our webgl scene! Let's pet our back, i think this is an important step!
+
+## HTML And WebGL Sitting Together In A Tree ?
+Well, not quite unfortunatly...
+WebGL is 3d inside a [canvas element](http://en.wikipedia.org/wiki/Canvas_element)
+and a canvas is a black box from the html page point of view.
+You can't bind [DOM events](http://en.wikipedia.org/wiki/DOM_events) inside canvas.
+You can't have
+[stylesheet](http://en.wikipedia.org/wiki/Style_sheet_\(web_development\))
+to change canvas content.
+You can't put dom elements inside your canvas.
+Those two don't talk to each other.
+
+### What Are The Limitations ?
+Unfortunatly it isn't all pink, WebGL and HTML aren't really merged.
+This is only a nice trick. It has some limitations.
+For example, the dom element is rotated using [css 3d](http://example.com/TODO).
+This is a fairly new technology.
+So you may hit bugs.
+
+Moreover, it only appears as a part of 3d... but this remains plain DOM.
+So it doesnt benefit from webgl specific display.
+For example, it is impossible to get
+[post processing](http://example.com/TODO)
+on the dom element.
+Indeed, this technic is applied in 2d on the rendered scene and the DOM element is not in it.
+Additionally the dom element won't share the lighting as the rest of your webgl scene.
+Nevertheless, [css shader](http://example.com/TODO) allows you to apply shader on normal DOM element,
+so it may be possible to make a coherent lighting.
+The web is so beautifull nowadays!
+
+### Conclusion
+I love this new trick.
+I've been trying to make webgl easier for while now.
+My strategy has been to make it closer to what webdevs know today,
+copying [jQuery API on top of three.js](http://jeromeetienne.github.com/tquery/),
+emulating [dom events inside webgl scene](http://learningthreejs.com/blog/2012/01/17/dom-events-in-3d-space/)
+or even making [3d text act as web links](http://learningthreejs.com/blog/2012/02/27/linkify-tquery-extension/).
+To integrate actual web pages inside webgl scene definitly matches this vibe!
+
+Congratulations guys! You can now mix html pages with your webgl content. I encourage you to experience with this new way to interact with webgl 3d.
+
+That's all for today, have fun :)
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 4c05b44

Please sign in to comment.