New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2D Text Sprites #1321

Closed
mscongdon opened this Issue Feb 13, 2012 · 23 comments

Comments

Projects
None yet
@mscongdon

mscongdon commented Feb 13, 2012

I'd like to render simple text strings that "always face the camera". To save overhead, 2D text would be ideal. Unfortunately, I have not found any features in Three.js that support this. I don't want to put the text into an image [sprite] because I need to update/change it dynamically.

Please prove my searches insufficient and point me to an example that illustrates this. It would greatly enhance our user experience!

Cheers,
Michael

@WestLangley

This comment has been minimized.

Show comment
Hide comment
@mscongdon

This comment has been minimized.

Show comment
Hide comment
@mscongdon

mscongdon Feb 13, 2012

I've been using that for 3D geometry and it looks nice. But it's overkill when I just need some simple, 2D text. Is there any 2D text (or text sprite equivalent) in Three.js?

I plan to have 20-30 elements of text that change each time I update my animation (continuously). Think of a stop watch that continuously shows you the number of milliseconds,seconds, minutes, etc. throughout the animation). Creating 20+ 3D text elements would be exponentially more CPU-intensive than a 2D approach. It already turned my animation from smooth to jittery when I tried to do this with just 1 element of text in 3D.

mscongdon commented Feb 13, 2012

I've been using that for 3D geometry and it looks nice. But it's overkill when I just need some simple, 2D text. Is there any 2D text (or text sprite equivalent) in Three.js?

I plan to have 20-30 elements of text that change each time I update my animation (continuously). Think of a stop watch that continuously shows you the number of milliseconds,seconds, minutes, etc. throughout the animation). Creating 20+ 3D text elements would be exponentially more CPU-intensive than a 2D approach. It already turned my animation from smooth to jittery when I tried to do this with just 1 element of text in 3D.

@chandlerprall

This comment has been minimized.

Show comment
Hide comment
@chandlerprall

chandlerprall Feb 13, 2012

Contributor

You can overlay normal HTML elements on top of the 3D canvas.

Contributor

chandlerprall commented Feb 13, 2012

You can overlay normal HTML elements on top of the 3D canvas.

@mscongdon

This comment has been minimized.

Show comment
Hide comment
@mscongdon

mscongdon Feb 14, 2012

If I were to take a guess at that, it would be more like having game scores at the top of the canvas regardless of what changes within the canvas...right? I'd need this text to move within the canvas. Imagine multiple people walking around a scene and, instead of a light bulb over their head, they have some text that changes frequently (like the timer example I mentioned above). Text in the distance would be smaller than text in the foreground. Someone walking left-to-right would need their text to move left-to-right with them. Can Three.js support that?

mscongdon commented Feb 14, 2012

If I were to take a guess at that, it would be more like having game scores at the top of the canvas regardless of what changes within the canvas...right? I'd need this text to move within the canvas. Imagine multiple people walking around a scene and, instead of a light bulb over their head, they have some text that changes frequently (like the timer example I mentioned above). Text in the distance would be smaller than text in the foreground. Someone walking left-to-right would need their text to move left-to-right with them. Can Three.js support that?

@mrdoob

This comment has been minimized.

Show comment
Hide comment
@mrdoob

mrdoob Feb 14, 2012

Owner

"A picture is worth a thousand words" :)

Owner

mrdoob commented Feb 14, 2012

"A picture is worth a thousand words" :)

@mscongdon

This comment has been minimized.

Show comment
Hide comment
@mscongdon

mscongdon Feb 14, 2012

Do you think I can reduce the 3D overhead dramatically and implement this feature if I still used TextGeometry, but simply do not extrude the text more than 1 pixel...and continuously translate and rotate each text element to face the camera? Or does the TextGeometry use the same amount of CPU whether it creates a 1-pixel deep-object vs. a 20-pixel-deep object?

mscongdon commented Feb 14, 2012

Do you think I can reduce the 3D overhead dramatically and implement this feature if I still used TextGeometry, but simply do not extrude the text more than 1 pixel...and continuously translate and rotate each text element to face the camera? Or does the TextGeometry use the same amount of CPU whether it creates a 1-pixel deep-object vs. a 20-pixel-deep object?

@chandlerprall

This comment has been minimized.

Show comment
Hide comment
@chandlerprall

chandlerprall Feb 14, 2012

Contributor

That's still a lot of processing. You can calculate where an object is in the 2D space of the canvas and position the text, in an HTML element, at that point, changing the size based on the object's distance to the camera. To my knowledge this is how games such as Runescape display player names in the world.

Contributor

chandlerprall commented Feb 14, 2012

That's still a lot of processing. You can calculate where an object is in the 2D space of the canvas and position the text, in an HTML element, at that point, changing the size based on the object's distance to the camera. To my knowledge this is how games such as Runescape display player names in the world.

@mrdoob

This comment has been minimized.

Show comment
Hide comment
@mrdoob

mrdoob Feb 14, 2012

Owner

#1312 has a better snippet for projecting 3D vectors.

Owner

mrdoob commented Feb 14, 2012

#1312 has a better snippet for projecting 3D vectors.

@BKcore

This comment has been minimized.

Show comment
Hide comment
@BKcore

BKcore Feb 14, 2012

Or you can draw your text on a 2D canvas with fillText(), and use that canvas as a texture for your billboard sprites.

Or with the same technique, render all your texts on a canvas 2D, then blend that texture in a postprocess pass.

BKcore commented Feb 14, 2012

Or you can draw your text on a 2D canvas with fillText(), and use that canvas as a texture for your billboard sprites.

Or with the same technique, render all your texts on a canvas 2D, then blend that texture in a postprocess pass.

@talktough

This comment has been minimized.

Show comment
Hide comment
@talktough

talktough Feb 17, 2012

@mscongdon - this is exactly what I am trying to do at the moment as well.

I need to put label annotations on a canvas next to objects that are being rotated by the user. I also need to be able to detect events when either the objects or text are clicked by the user.

I managed to get a piece of text next to an object using canvas text and then using 3D to 2D conversion of the coordinates as outlined above. However I can't work out how to rotate the text or detect click events on them.

When I was using Flash I used to remove all the texts and then add them back at the new positions for every animation frame which worked ok but I can't work out an equivalent here for canvas text.

Just wondering if you have you made any progress ?

Thanks

Chris

talktough commented Feb 17, 2012

@mscongdon - this is exactly what I am trying to do at the moment as well.

I need to put label annotations on a canvas next to objects that are being rotated by the user. I also need to be able to detect events when either the objects or text are clicked by the user.

I managed to get a piece of text next to an object using canvas text and then using 3D to 2D conversion of the coordinates as outlined above. However I can't work out how to rotate the text or detect click events on them.

When I was using Flash I used to remove all the texts and then add them back at the new positions for every animation frame which worked ok but I can't work out an equivalent here for canvas text.

Just wondering if you have you made any progress ?

Thanks

Chris

@mscongdon

This comment has been minimized.

Show comment
Hide comment
@mscongdon

mscongdon Feb 17, 2012

I'll be working this over the next week and will update this thread with my findings. Thanks, everyone, for your suggestions so far!

mscongdon commented Feb 17, 2012

I'll be working this over the next week and will update this thread with my findings. Thanks, everyone, for your suggestions so far!

@mrdoob mrdoob closed this Feb 17, 2012

@mscongdon

This comment has been minimized.

Show comment
Hide comment
@mscongdon

mscongdon Feb 29, 2012

The code samples (including issue #1312) are helpful to get the 2D coordinates. However, I still am unclear how to put 2D text onto a canvas that's already using the WebGLRenderer.

I tried putting 2 renderers the same canvas like this:

var canvas = document.getElementById('myCanvas');
var renderer = new THREE.WebGLRenderer({canvas: canvas});

// do all of my 3D stuff

var renderer2D = new THREE.CanvasRenderer({canvas: canvas});
renderer2D.domElement.getContext('2d').fillText("Hello", 50, 50);

The renderer2D.domElement.getContext('2d') returns null

If I reverse my code to initialize my renderer2D before my renderer, then I can get the '2d' context just fine, but the WebGLRenderer initialization fails with the error Error creating WebGL context.

Hence, I cannot appear to get both a webgl (or experimental-webgl) and a '2d' context on the same HTMLCanvasElement to apply both my 3D and my 2D renderings.

I'm not convinced that the API spec at this URL http://www.w3.org/TR/html5/the-canvas-element.html#the-canvas-element says this is NOT possible.

"Returns null if the given context ID is not supported or if the canvas has already been initialised with some other (incompatible) context type (e.g. trying to get a "2d" context after getting a "webgl" context)."

Does the use of experimental-webgl (rather than simply webgl) as the primary context cause '2d' to not work?

Am I close to getting my 2D text sprites working? Or should I be thinking of this entirely differently? Please advise.

mscongdon commented Feb 29, 2012

The code samples (including issue #1312) are helpful to get the 2D coordinates. However, I still am unclear how to put 2D text onto a canvas that's already using the WebGLRenderer.

I tried putting 2 renderers the same canvas like this:

var canvas = document.getElementById('myCanvas');
var renderer = new THREE.WebGLRenderer({canvas: canvas});

// do all of my 3D stuff

var renderer2D = new THREE.CanvasRenderer({canvas: canvas});
renderer2D.domElement.getContext('2d').fillText("Hello", 50, 50);

The renderer2D.domElement.getContext('2d') returns null

If I reverse my code to initialize my renderer2D before my renderer, then I can get the '2d' context just fine, but the WebGLRenderer initialization fails with the error Error creating WebGL context.

Hence, I cannot appear to get both a webgl (or experimental-webgl) and a '2d' context on the same HTMLCanvasElement to apply both my 3D and my 2D renderings.

I'm not convinced that the API spec at this URL http://www.w3.org/TR/html5/the-canvas-element.html#the-canvas-element says this is NOT possible.

"Returns null if the given context ID is not supported or if the canvas has already been initialised with some other (incompatible) context type (e.g. trying to get a "2d" context after getting a "webgl" context)."

Does the use of experimental-webgl (rather than simply webgl) as the primary context cause '2d' to not work?

Am I close to getting my 2D text sprites working? Or should I be thinking of this entirely differently? Please advise.

@mscongdon

This comment has been minimized.

Show comment
Hide comment
@mscongdon

mscongdon Feb 29, 2012

OK. I figured out a workable solution.

Step 1. Create 2 canvases - One for 3D and One for 2D.

<canvas id="canvas2D" width="500" height="500"></canvas>
<canvas id="canvas3D" width="500" height="500"></canvas>

Step 2. Give both canvases the same dimensions and location, and set the z-index of them so that the 2D canvas is on top of the 3D canvas. And be sure to set the background-color to 'transparent' for the 2D canvas.

#canvas3D{
  position: absolute;
  left: 0;
  top: 0;
  z-index: 1;
}
#canvas2D{
  background-color: transparent;
  position: absolute;
  left: 0;
  top: 0;
  z-index: 2;
}

This essentially treats "canvas2D" like a pane of glass onto which you paint your 2D images, yet still able to see through to the 3D canvas behind it.

Step 3: Lookup both canvases and create both a CanvasRenderer and a WebGLRenderer:

canvas2D = document.getElementById('canvas2D');
renderer2D = new THREE.CanvasRenderer({canvas: canvas2D});
canvas3D = document.getElementById('canvas3D');
renderer3D = new THREE.WebGLRenderer({canvas: canvas3D});

Step 4: Aside from the regular 3D renderings, here is how we start to draw the 2D text sprites. First, calculate the X,Y location where you want to fill the text from some position within your 3D world. I used a variation of the function in Issue #78:

var coord =  toScreenXY(myPosition, camera, canvas3D);

function toScreenXY(position, camera, canvas) {
  var pos = position.clone();
  var projScreenMat = new THREE.Matrix4();
  projScreenMat.multiply(camera.projectionMatrix, camera.matrixWorldInverse);
  projScreenMat.multiplyVector3( pos );

  return { x: ( pos.x + 1 ) * canvas.width / 2 + canvas.offsetLeft,
      y: ( - pos.y + 1) * canvas.height / 2 + canvas.offsetTop };
}

Step 5: Clear the 2D canvas and re-write the text:

ctx2d = renderer2D.domElement.getContext('2d');
ctx2d.clearRect (0, 0, window.innerWidth, window.innerHeight);
ctx2d.fillText("Hello", coord.x, coord.y);

I'm happy to at least get this working. I'll worry about efficiencies or other drawbacks later! :-)

Cheers,
Michael

mscongdon commented Feb 29, 2012

OK. I figured out a workable solution.

Step 1. Create 2 canvases - One for 3D and One for 2D.

<canvas id="canvas2D" width="500" height="500"></canvas>
<canvas id="canvas3D" width="500" height="500"></canvas>

Step 2. Give both canvases the same dimensions and location, and set the z-index of them so that the 2D canvas is on top of the 3D canvas. And be sure to set the background-color to 'transparent' for the 2D canvas.

#canvas3D{
  position: absolute;
  left: 0;
  top: 0;
  z-index: 1;
}
#canvas2D{
  background-color: transparent;
  position: absolute;
  left: 0;
  top: 0;
  z-index: 2;
}

This essentially treats "canvas2D" like a pane of glass onto which you paint your 2D images, yet still able to see through to the 3D canvas behind it.

Step 3: Lookup both canvases and create both a CanvasRenderer and a WebGLRenderer:

canvas2D = document.getElementById('canvas2D');
renderer2D = new THREE.CanvasRenderer({canvas: canvas2D});
canvas3D = document.getElementById('canvas3D');
renderer3D = new THREE.WebGLRenderer({canvas: canvas3D});

Step 4: Aside from the regular 3D renderings, here is how we start to draw the 2D text sprites. First, calculate the X,Y location where you want to fill the text from some position within your 3D world. I used a variation of the function in Issue #78:

var coord =  toScreenXY(myPosition, camera, canvas3D);

function toScreenXY(position, camera, canvas) {
  var pos = position.clone();
  var projScreenMat = new THREE.Matrix4();
  projScreenMat.multiply(camera.projectionMatrix, camera.matrixWorldInverse);
  projScreenMat.multiplyVector3( pos );

  return { x: ( pos.x + 1 ) * canvas.width / 2 + canvas.offsetLeft,
      y: ( - pos.y + 1) * canvas.height / 2 + canvas.offsetTop };
}

Step 5: Clear the 2D canvas and re-write the text:

ctx2d = renderer2D.domElement.getContext('2d');
ctx2d.clearRect (0, 0, window.innerWidth, window.innerHeight);
ctx2d.fillText("Hello", coord.x, coord.y);

I'm happy to at least get this working. I'll worry about efficiencies or other drawbacks later! :-)

Cheers,
Michael

@mrdoob

This comment has been minimized.

Show comment
Hide comment
@mrdoob

mrdoob Mar 2, 2012

Owner

What about using <div>?

var text = document.createElement( 'div' );
text.style.position = 'absolute';
text.innerHTML = 'Oh hai!';
//
text.style.left = coord.x + 'px';
text.style.top = coord.y + 'px';
Owner

mrdoob commented Mar 2, 2012

What about using <div>?

var text = document.createElement( 'div' );
text.style.position = 'absolute';
text.innerHTML = 'Oh hai!';
//
text.style.left = coord.x + 'px';
text.style.top = coord.y + 'px';
@firendust

This comment has been minimized.

Show comment
Hide comment
@firendust

firendust May 20, 2012

i had the same text relate 3d objects problem here, and i think mrdoob's "div" solution is much more convenient... :D

firendust commented May 20, 2012

i had the same text relate 3d objects problem here, and i think mrdoob's "div" solution is much more convenient... :D

@Yona-Appletree

This comment has been minimized.

Show comment
Hide comment
@Yona-Appletree

Yona-Appletree Jan 16, 2013

I'm having a similar problem, but I'm not sure the general "paint your text outside of three.js" will work for me. I want the text to exist inside the 3d world such that occlusion by other objects and scaling with distance work correctly.

Although I haven't yet tried it, it seems that you could create a plane geometry with a texture that was created from another canvas that had text drawn onto it, then every such plane at the camera during each render phase. This seems like a significant amount of work, however, and while I'm rather new to 3d programming, I understand that many engines support having a textured-plane that always faces the camera.

Thoughts?

Yona-Appletree commented Jan 16, 2013

I'm having a similar problem, but I'm not sure the general "paint your text outside of three.js" will work for me. I want the text to exist inside the 3d world such that occlusion by other objects and scaling with distance work correctly.

Although I haven't yet tried it, it seems that you could create a plane geometry with a texture that was created from another canvas that had text drawn onto it, then every such plane at the camera during each render phase. This seems like a significant amount of work, however, and while I'm rather new to 3d programming, I understand that many engines support having a textured-plane that always faces the camera.

Thoughts?

@mrdoob

This comment has been minimized.

Show comment
Hide comment
@mrdoob

mrdoob Jan 16, 2013

Owner

As stated in the guidelines, help requests should be done in stackoverflow. This board is for bugs or feature requests.

Owner

mrdoob commented Jan 16, 2013

As stated in the guidelines, help requests should be done in stackoverflow. This board is for bugs or feature requests.

@Yona-Appletree

This comment has been minimized.

Show comment
Hide comment
@Yona-Appletree

Yona-Appletree Jan 16, 2013

My apologies. I searched for the problem on google and ended up here, where it appeared there was already a discussion on the topic, so it seemed appropriate. I'll post something on stackoverflow.

Yona-Appletree commented Jan 16, 2013

My apologies. I searched for the problem on google and ended up here, where it appeared there was already a discussion on the topic, so it seemed appropriate. I'll post something on stackoverflow.

@jareiko

This comment has been minimized.

Show comment
Hide comment
@jareiko

jareiko Mar 13, 2013

Contributor

FYI, you can create a 2D text mesh using ShapeGeometry instead of TextGeometry.

This reduces the number of faces generated because it doesn't extrude.

var shapes, geom, mat, mesh;

shapes = THREE.FontUtils.generateShapes( "Hello world", {
  font: "helvetiker",
  weight: "bold",
  size: 10
} );
geom = new THREE.ShapeGeometry( shapes );
mat = new THREE.MeshBasicMaterial();
mesh = new THREE.Mesh( geom, mat );
Contributor

jareiko commented Mar 13, 2013

FYI, you can create a 2D text mesh using ShapeGeometry instead of TextGeometry.

This reduces the number of faces generated because it doesn't extrude.

var shapes, geom, mat, mesh;

shapes = THREE.FontUtils.generateShapes( "Hello world", {
  font: "helvetiker",
  weight: "bold",
  size: 10
} );
geom = new THREE.ShapeGeometry( shapes );
mat = new THREE.MeshBasicMaterial();
mesh = new THREE.Mesh( geom, mat );
@natewilliford

This comment has been minimized.

Show comment
Hide comment
@natewilliford

natewilliford Jul 22, 2013

Contributor

My solution was to use a Particle with the ParticleCanvasMaterial. This exposes the canvas context through the program parameter which you can use to draw your text directly on the canvas.

var material = new THREE.ParticleCanvasMaterial({
  color: 0x000000,
  program: function(context) {
    context.font = "5pt Helvetica";
    context.fillText("Hello World", 0, 0);
  }
});
var particle = new THREE.Particle(material);

I had to tweak the position and scale of the particle to get it to display properly, but it seems to work well. Plus you can treat your text as an object in your scene and position it (etc) like you would any object.

EDIT: This is using the CanvasRenderer. I'm not sure how/if it will work with webGL.

Contributor

natewilliford commented Jul 22, 2013

My solution was to use a Particle with the ParticleCanvasMaterial. This exposes the canvas context through the program parameter which you can use to draw your text directly on the canvas.

var material = new THREE.ParticleCanvasMaterial({
  color: 0x000000,
  program: function(context) {
    context.font = "5pt Helvetica";
    context.fillText("Hello World", 0, 0);
  }
});
var particle = new THREE.Particle(material);

I had to tweak the position and scale of the particle to get it to display properly, but it seems to work well. Plus you can treat your text as an object in your scene and position it (etc) like you would any object.

EDIT: This is using the CanvasRenderer. I'm not sure how/if it will work with webGL.

@oleduc

This comment has been minimized.

Show comment
Hide comment
@oleduc

oleduc Aug 27, 2013

Canvas as texture is fast enough

var canvas1 = document.createElement('canvas');
    var context1 = canvas1.getContext('2d');
    context1.font = "Bold 40px Arial";
    context1.fillStyle = "rgba(255,0,0,0.95)";
    context1.fillText(text, 0, 50);

    var texture1 = new THREE.Texture(canvas1);
    texture1.needsUpdate = true;

    var material1 = new THREE.MeshBasicMaterial( { map: texture1, side:THREE.DoubleSide } );
    material1.transparent = true;

    var mesh1 = new THREE.Mesh(
        new THREE.PlaneGeometry(canvas1.width, canvas1.height),
        material1
    );

    mesh1.position.set(position.x + 10,position.y,position.z);

    scene.add( mesh1 );

oleduc commented Aug 27, 2013

Canvas as texture is fast enough

var canvas1 = document.createElement('canvas');
    var context1 = canvas1.getContext('2d');
    context1.font = "Bold 40px Arial";
    context1.fillStyle = "rgba(255,0,0,0.95)";
    context1.fillText(text, 0, 50);

    var texture1 = new THREE.Texture(canvas1);
    texture1.needsUpdate = true;

    var material1 = new THREE.MeshBasicMaterial( { map: texture1, side:THREE.DoubleSide } );
    material1.transparent = true;

    var mesh1 = new THREE.Mesh(
        new THREE.PlaneGeometry(canvas1.width, canvas1.height),
        material1
    );

    mesh1.position.set(position.x + 10,position.y,position.z);

    scene.add( mesh1 );
@makc

This comment has been minimized.

Show comment
Hide comment
@makc

makc Aug 21, 2014

Contributor

What about using <div>?

That actually implies lot of stuff. I.e. parent div must be position:relative and overflow:hidden

Contributor

makc commented Aug 21, 2014

What about using <div>?

That actually implies lot of stuff. I.e. parent div must be position:relative and overflow:hidden

@endel

This comment has been minimized.

Show comment
Hide comment
@endel

endel Jan 10, 2016

Contributor

I know this is an old thread, but I'd like to share a module I've created to draw text from canvas into THREE.Mesh: http://gamestdio.github.io/three-text2d/

Contributor

endel commented Jan 10, 2016

I know this is an old thread, but I'd like to share a module I've created to draw text from canvas into THREE.Mesh: http://gamestdio.github.io/three-text2d/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment