Raytracer / Pathtracer for Three.js

[zz85]

This has been an idea stuck in my head, but I'm trying to resist the temptation not to fall (and hurt myself) in another rabbit hole so I would just like to throw this out for discussion/comments.

There are already several ray/path tracers running on webgl out there like http://www.iquilezles.org/apps/shadertoy/ and @evanw's http://madebyevan.com/webgl-path-tracing/ and subblue's fractallabs using the 2 triangles method http://www.iquilezles.org/www/material/nvscene2008/rwwtt.pdf.

The idea is that to perhaps port or write from scratch a simple raytracer to THREE.RaytraceRenderer which would take three.js scene objects to rendered in webgl fragment shader.

Would anyone have an estimate how complex a scene (many planes/quads/objects) would be these techniques support. In shadertoy, Kinderpainter seems to run pretty fast, but droid seems to be much more gpu intensive.

Not sure if there's any real use for this, esp if it slower like doing pathtracing but I guess it could be a fun experiment or something to use with my renderflies project.

[mrdoob]

That's the aim of the current structure. Anyone should be able to code their own renderer. Having a RaytraceRenderer would be awesome, but most of the raytracing stuff use basic/procedural shapes. Raytracing a custom geometries may prove a bit slow?

[zz85]

hi mrdoob, do you think its better to extend WebGLRender for this (like
Anaglyph and CrosseyedWebGLRenderer) or write it as a new renderer like dom,
canvas, svg?

[mrdoob]

I think it would be better as a new renderer.

[zz85]

yeah make more sense, especially when you need to recompile the shaders. there's too many things going way above my head now, but i'll update if i can make any progress...

[evanw]

Hmm, this looks interesting. I was going to write something like this if WebCL ever came out, which I think makes this a whole lot easier to do. For larger scenes it makes sense to use an acceleration data structure like a kd tree or a 3d grid. You might also want to try supporting a voxel grid primitive, which could be stored as a signed distance field and traced using sphere tracing.

[zz85]

i just found out that @prutsdomj has done some interesting related raymarching distance fields stuff with threejs. see https://github.com/prutsdomj/WebGL-RayMarch and http://nopjia.blogspot.com/

this approaches works more with primitives, and such a renderer for three.js scene might be limited in some ways. for more compatibility, one may use the approach in another paper that supports polygons found in one of the referenced issues.

now that mrdoob is working on his software renderer, its quite tempting to attempt such a mesh up too, but i'll hold doing this until i've more confidence of pulling this off.

[nopjia]

The sphere tracing technique is useful/efficient for distance fields, and there's a lot of cool images you can make with distance functions. (For example: http://blog.hvidtfeldts.net/index.php/2011/06/distance-estimated-3d-fractals-part-i/)

If you'd like support a triangle mesh, you'd have to go with traditional ray tracing. Since fragment shaders is what we're talking about, a data structure like kd-tree would be quite tricky to implement -- actually I'm not sure if it'd be possible on the current GLSL version or if it'd help performance. Doing something simpler like bounding volumes would help.

[zz85]

@prutsdomj

thanks for the inputs. actually the paper in mind which uses ray-triangle intersection is this, although i'm not sure how good will that be

just another question - how difficult do you think is it to mesh up simple RayMarching renderer for threejs using what you have? Perhaps sometime like

var scene = new THREE.Scene();
scene.add( new THREE.Sphere() );
scene.add( new THREE.Cube() );
scene.add( new THREE.Plane() );
// etc

renderer = new THREE.RaymarchingRenderer();
renderer.render( scene, camera );

[metaleap]

Eh what is a "raymarching renderer" -- it's all shader code no? ;) You wanna "compile" a scene-graph of spheres and cubes to simple GLSL vec3(posx, posy, posz) declarations?...

[metaleap]

Hey @zz85 that paper you just linked 2 weeks ago is interesting, especially this part:

"Our current implementation of the grid traversal code does not map well to the vertex program instruction set, and is thus quite inefficient. Since grid traversal is so similar to rasterization, it might be possible to modify the rasterizer to walk through the grid."

Anyone got any rough ideas how traversal would map to rasterization? Way I see it, in old-school rasterization we need to send lots of vertices (or generate them from geometry shader), they are vertex-shaded and fragment-shaded. Guess you could see the hardware-based z-culling as "traversal" and in the frag shader just output the "node ID" the vertex represents as color. Since modern hardware can supposedly handle millions of "triangles" (if they represent nodes in a kd-tree, octree or other structure we can make them 1px big to minimize redundant frag shader invocations) -- could be worth a shot. Just wondering if anyone ever played with this or sees any issues with that idea?

[zz85]

@metaleap The idea was originally to have a drop in RaymarchingRenderer, which renders a three.js scene, which produces a higher quality (hopefully) render. For example, CanvasRenderer can be replaced WebGLRenderer or even a CSS3Renderer. Or perhaps follow the Anaglyph, Crosseyed, ParallaxBarrier, Ascii *Effects route.

[skrat]

Raymarching renderer is possible through type checking, and having a translator for every (not really, just some, like cube, sphere, plane) geometry type, to a GLSL fragments (depth map function and it's calls). With this you can renderer using full screen quad and technique described by Iñigo Quílez. Object can animate and be updated, but adding or removing would require shader recompile.

It might be useful for offline lightmap renderer, with GI or AO. That also goes for classical ray tracing.

[mrdoob]

Isn't it possible to store the scene graph info on a float texture and use that in the raymarcher?

[skrat]

I tried that, although with arrays, and I experienced many issues with GLSL (and GL ES), namely constant size loops and many other things that you're not allowed to do in a loop, like accessing array/texture with other index than the one from loop etc. You can check a simple raymarcher here https://bitbucket.org/skrat/shadowbox/src , I might be taking another look at it, although I think a path tracer with GI, in JS, running on CPU (perhaps in web worker) might be far more practical.