node.js server rendering with node-webgl and/or headless-gl #7085

Closed
bsergean opened this Issue Sep 2, 2015 · 19 comments

Projects

None yet

5 participants

@bsergean
bsergean commented Sep 2, 2015

I have been trying (with great pain ... but finally some success) to use node + three.js to render some models on a Linux server box.

The only workable approach I found was to use the fork of three.js that @kabbi did back in March. I'm using Xvfb on the server on Linux, and @kabbi fork which is using node-webgl to create GL context from node. @kabbi did not have to modify too many files to make this works, mostly adding some cast to WebGLRenderer.js, make some of the LoadUtils code work differently. The nodes header.js file is used to create a shim/wrapper so that the global document (not accessible from node) is using the fake one that node-webgl provides.

https://github.com/kabbi/three.js/blob/d97012f8234796b9e0291e6f6e965e293d950099/src/nodejs/header.js

Here is the commit that had node / node-webgl support -> kabbi@d97012f

(the kind of "cast" I'm talking about, mostly around some uniforms)

                case '1iv':
-                   _gl.uniform1iv( location, value );
+                   _gl.uniform1iv( location, new Int32Array(value) );
                    break;

I would like to have people advices on how to merge this back into the main repo, whether this is feasible or not, and on how to move forward. Any thought, anyone ?

Also, my current main interest is to do offscreen rendering, so a library like https://github.com/stackgl/headless-gl is actually making more sense for my use case. I tried to create a context from headless-gl and pass it to the WebGLRenderer with no luck (getting a white image with my test-case).

BTW for my test cases I tried to use node-canvas to render text into textures, but I got some crashes with node-canvas and ended up using 3D Fonts (TextGeometry) which work well.

Thanks for any help (and for this awesome library). Once one experiences the speed and ease of doing OpenGL in a browser it's very hard to go back to C++ ;)

@bhouston
Contributor
bhouston commented Sep 3, 2015

Is there any way to fix node-webgl rather than modifying ThreeJS?

Alternatively, could you just patch _gl.uniform1iv function and replace it with a new one that does this conversion? Something like this:

var originalUniform1iv = _gl.uniform1iv;
_gl.uniform1iv = function(..) {...  originalUniform1iv.call( _gl, ... ); }

I think you could do after creating the WebGLRenderer but before actually rendering?

That does the conversion if necessary? I'm not sure if _gl is a protected object or not. But it would be best to not pollute ThreeJS core with workarounds if one can work around them outside of ThreeJS.

BTW I know Ken Museth over at DA. :)

@bsergean
bsergean commented Sep 3, 2015

Interesting though on "monkey patching" node-webgl. That's something to try: when trying to plug in headless-gl I was passing the gl-object to three.js / I don't know if the gl object from node-webgl is private or not but I'll see that. If it goes well I'll report back (doing something else now, trying to use web-workers to load a big model without blocking the UI).

I'd like to use stock three.js as much as I can (ideally fetched form npm). The other thing I've had some trouble with is using three with browserify, but today I tried again and it looks like THREE = require('three') just works ... which is great.

Funny that you know Ken Museth, 3D is such a small world; I have only been to Siggraph once and it had a high-school reunion feeling, github must be close to that ;)

@bsergean
bsergean commented Sep 8, 2015

I got three.js + headless-gl to work nicely together. My only problem is the precision GL_ES directive which make the shader compile step fails on Desktop OpenGL. I created a pull request to help with that.

@bhouston
Contributor
bhouston commented Sep 8, 2015

@bsergean Sweet. What WebGL extensions does headless gl support? I guess there is a list somewhere?

Maybe you should contribute a ThreeJS example or something. :)

@bsergean
bsergean commented Sep 8, 2015

Unfortunately headless-gl does not support any extensions at this point, but the three.js code is playing nicely with the lack of extension support and the fall-back work well, at least for my case. The main problem I've had was that I need to render fonts, and I was originally using THREE.sprite for that purpose + a canvas object. Unfortunately the node canvas module was crashing for me, so I used a 3D font instead (TextGeometry) which works great.

I'm gonna create an issue on headless-gl about the lack of extension support and what it would take to add.

And yes, adding an example would be a good thing to do, I'll do that. Should I put it in the examples folder ?

@bhouston
Contributor
bhouston commented Sep 9, 2015

You has to run it by @mrdoob, but I think that node.js is becoming a major platform for JavaScript development. We could actually create a root "node" folder and put the npm generator in it as well as an examples folder. Then we could start to build up the node capabilities of ThreeJS in a focused fashion. @WestLangley also has a JavaScript convolution generator that could go in the node folder.

@bsergean
bsergean commented Sep 9, 2015

Ok. I have created some documentation and a sample script to get a .png of a cube (I'm sure we can make a much more impressive demo but it shows my lack of creativity + and keep the sample simple ;) ) out of three.js + headless-gl.

It's in a gist, and should work for anyone hopefully. I tested it on my Mac. Not sure where to take this from there, but it's a working sample so it's a good start.

https://gist.github.com/bsergean/6780d7cc0cabb1b4d6c8

@whatisor

Hello,
I used headless-gl with threejs. It can run well some case but:

  • Texturing not work
  • Antialiasing not work.
@whatisor

Finally, I get Texture working 👍
But Antialiasing is no idea :(

@bsergean

Here is a gist that explains how I got texturing to work.

https://gist.github.com/bsergean/08be90a2f21205062ccc

I should try to create a similar one to explain how I got antialiasing to work (It's explained in one of my comment in stackgl/headless-gl#30 (comment)).

@whatisor

Hello,
Thank for your pointing.
For antialias, I tried this :
var gl = require('gl')(width, height, { preserveDrawingBuffer: true,antialias:true })
But it does not work, although I check you have antialias option in source code.
What should I need more to enable it?
Thank you. (not sure I can understand all your discussion of old issue)

@bsergean

Hey @whatisor / here is a gist that should cover everything (in coffee-script, but porting to javascript should be trivial and you can use coffee -c to help with it). A bunch of extra code / files are required, but until the root issue is fixed in headless-gl this is my way of doing it.

https://gist.github.com/bsergean/0d79ce3c7384cf6d1bb6

The antialias: true is ignored at this point.

@jackts
jackts commented Jan 8, 2016

Hi @bsergean,
I am still heavily researching on your topic and trying different examples. No luck to make it work for my use case, as I am not that advanced in JavaScript. I found couple different implementations which might work:
1st is modified softwarerender:
http://ox86.tumblr.com/post/129341221225/look-ma-no-canvas-server-side-3d-rendering-with

2nd is node-canvas backed By Cairo (Svg) very good example.
http://build-failed.blogspot.com/2014/01/generating-server-side-tile-maps-with.html?m=1

3d : to use jsdom, node-canvas, and phantom js, but my main concern is performance of server GPU with 1000 clients or so using it at the same time

My use case: I want to render on server and output jpg, Svg, png, obj, or gif or 3d scene with (vertices only) to showcase the result to the customer and to hide the rendering logic and models from the user(so they can't steal it).

@bsergean
bsergean commented Jan 8, 2016

Hi @jackts / the only output I have produced from node is a straight rgba buffer which is read back from GL, using headless-gl (https://github.com/stackgl/headless-gl), and then converted to an image format (I use png).

BTW, you don't have to render on the server to obfuscate code (~ prevent from stealing in your phrasing); you would just have to post-process your javascript with tools like browserify to make a single js bundle, that you can obfuscate with uglify.js.

You need a very good reason not to use natives clients; a server based approach has advantages but also drawbacks, such as scalability. If native clients can execute your code you don't need a big server, and it will be so much faster than streaming a video which is essentially what you would have to do with a node implementation. Lastly if you are trying to make an interactive experience the latency will make the experience horrible.

My use case for using node.js + three.js was offline reporting, with zero timing constraint.

@tomasparks

I was about to ask about this:
my project is going to create imposters, terrain textures, skyboxs and 3d models severer side

  • 3D models: are going to procedurally generated and some are going to send to a custom LODer
  • imposters: some of the 3D model are going have imposter made for them
  • terrain textures: are going to procedurally generated using a quad tree system eg: google map style
    some of these could be create client-side, but I dont trust the client to upload the correct data to the server
@whatisor

@bsergean : Thank you for your help.
However, in theory, because nodejs gl base in native desktop gl, it should be better quality.
However, as I see with same code, its quality is worse.
I guess reason is FXAA is not enough quality in comparsion with webgl on browser.
How do you think?

@bhouston
Contributor

@whatisor, can you share screenshots? I believe that by default WebGL uses hardware-based MSAA. OpenGL on the desktop is able to do MSAA as well, although I am unsure if it is enabled by default.

@bsergean

@whatisor / I think that this issue is not the good one to discuss the anti-aliasing problem; the anti-aliasing problem is a headless-gl only problem, which is tracked here -> stackgl/headless-gl#30. Desktop OpenGL knows how to anti-alias as @bhouston pointed out

@whatisor, that gist explains how to workaround the headless-gl problem, by doing anti-aliasing "yourself" -> https://gist.github.com/bsergean/0d79ce3c7384cf6d1bb6 / if you need this bug to be fixed at this point your best bet is to dig into this gist to understand how to do anti-aliasing yourself.

I'm going to close this issue (which I filed some time ago...), since it is now documented on how to use three.js + headless.gl, even if there are some bugs left.

@bsergean bsergean closed this Jan 20, 2016
@bhouston
Contributor

If you want to setup a MSAA buffer, I can share this code for anyone to use that does that -- it is used in our Exocortex Fury renderer product and its been tested on both Linux and Windows on a ton of GPUs over the past 6 years:

EStatus GPUParticleRenderer::createOffscreenMultisampleBuffer() {

    // http://www.opengl.org/wiki/GL_EXT_framebuffer_multisample

    int maxSamples;
    glGetIntegerv(GL_MAX_SAMPLES_EXT, &maxSamples);

    int samples = _frameSpec.getNumMultiSamples();
    if ( samples > maxSamples )
    {
        char szBuffer[1024];
        sprintf_s( szBuffer, 1024, "The requested number of multisamples (%i) is greater than the maximum supported (%i)", samples, maxSamples );
        return EStatus::FailureCode( szBuffer );
    }

    glGenFramebuffersEXT(1, &multiSampleTarget.fboId);
    glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, multiSampleTarget.fboId);

    glGenRenderbuffersEXT(1, &multiSampleTarget.colorBufferId);
    glBindRenderbufferEXT(GL_RENDERBUFFER_EXT, multiSampleTarget.colorBufferId);

    GL( glRenderbufferStorageMultisampleEXT(GL_RENDERBUFFER_EXT, samples, GL_RGBA32F_ARB, _frameSpec.getImageSize().x, _frameSpec.getImageSize().y) );

    glGenRenderbuffersEXT(1, &multiSampleTarget.depthBufferId);
    glBindRenderbufferEXT(GL_RENDERBUFFER_EXT, multiSampleTarget.depthBufferId);

    GL( glRenderbufferStorageMultisampleEXT(GL_RENDERBUFFER_EXT, samples, GL_DEPTH_COMPONENT24, _frameSpec.getImageSize().x, _frameSpec.getImageSize().y) );

    glFramebufferRenderbufferEXT(GL_FRAMEBUFFER_EXT, GL_COLOR_ATTACHMENT0_EXT, GL_RENDERBUFFER_EXT, multiSampleTarget.colorBufferId);
    glFramebufferRenderbufferEXT(GL_FRAMEBUFFER_EXT, GL_DEPTH_ATTACHMENT_EXT, GL_RENDERBUFFER_EXT, multiSampleTarget.depthBufferId);

    multiSampleTarget._init = true;     // can now be deleted!

    char szErrorMessage[1024];
    if( glExtCheckFramebufferStatus(szErrorMessage) != 1 )
    {
        char szBuffer[2048];
        sprintf_s( szBuffer, 1024, "Unable to allocate framebuffers because: %s", szErrorMessage );
        return EStatus::FailureCode( szBuffer );
    }
    return EStatus::SuccessCode();
}

Then just before we render, we do this:

    if (_frameSpec.getNumMultiSamples() > 0) {
        GL( glBindFramebufferEXT( GL_FRAMEBUFFER_EXT, multiSampleTarget.fboId ) );
    }
    else {
        GL( glBindFramebufferEXT( GL_FRAMEBUFFER_EXT, singleSampleTarget.fboId ) );
    }
@zbjornson zbjornson referenced this issue in Automattic/node-canvas Mar 7, 2016
Open

Possible to render three.js on serverside? #730

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment