Add sprite.raycast support for scale,rotation,center #14292

06wj · 2018-06-15T09:01:29Z

eg: https://rawgit.com/06wj/three.js/patch1_build/examples/webgl_raycast_sprite.html

mrdoob · 2018-06-16T16:27:28Z

@WestLangley Any thoughts about this one?

WestLangley · 2018-06-16T16:55:15Z

If you set the scale to be ( 1, 10, 1 ), it does not work well because the sprite is approximated as a sphere.

If we are going to the trouble to support sprite transforms, we should probably make click events on sprites accurate.

A sprite is just two billboarded triangles. I expect we could use ray.intersectTriangle() to get accurate click results.

06wj · 2018-06-17T08:52:30Z

@WestLangley Hi, I updated the code to use ray.intersectTriangle() for raycast.

WestLangley · 2018-06-17T12:03:11Z

src/objects/Sprite.js

+		function transformVertex( vertexPosition, mvPosition, center, scale, sin, cos ) {
+
+			// compute position in camera space
+			alignedPosition.set( ( vertexPosition.x - center.x ) * scale.x, ( vertexPosition.y - center.y ) * scale.y );


You omitted the 0.5 offset (on purpose, I assume) because you compensated for it elsewhere. I am not a fan of doing that because this formula now looks incorrect.

In theory it should be::

alignedPosition.subVectors( vertexPosition, center ).addScalar( 0.5 ).multiply( scale );

WestLangley · 2018-06-17T12:05:55Z

src/objects/Sprite.js

+			// compute position in camera space
+			alignedPosition.set( ( vertexPosition.x - center.x ) * scale.x, ( vertexPosition.y - center.y ) * scale.y );
+
+			if ( sin !== undefined ) {


I am not a fan of this either, to tell you the truth. You are really checking if rotation is zero.

I want to cache the sin and cos value.I will add a comment here

WestLangley · 2018-06-17T12:10:44Z

src/objects/Sprite.js

+
+			var center = this.center;
+
+			transformVertex( vA.set( 0, 0, 1 ), mvPosition, center, worldScale, sin, cos );


This is where you compensate for the missing offset. If you are not going to change this, at least add add a comment here... Why is the z-coordinate not zero?

because the z is not used, i can change it to zero.

I will refact it to show the calculated offset

WestLangley · 2018-06-17T12:31:21Z

Nice work! And thank you for the excellent example!

Make sure you test the case where the sprite parent is transformed.

Unrelated tip: You don't need to instantiate a new Raycaster and Vector3 every time the mouse moves. Use raycaster.setFromCamera().

06wj · 2018-06-17T13:28:58Z

@WestLangley I updated the code and example, added the case where the sprite parent is transformed.

WestLangley · 2018-06-17T17:49:10Z

Your triangles have the incorrect winding order, and consequently, the front face is incorrect. This is why you have to prevent back-face culling to get the correct result.

The face vertices should be counterclockwise. Back-face culling should be true.

Specify the faces in the same order as they are specified in WebGLSpriteRenderer.js.

var faces = new Uint16Array( [
	0, 1, 2,
	0, 2, 3
] );

Like so:

transformVertex( vA.set( - 0.5, - 0.5, 0 ), mvPosition, center, worldScale, sin, cos );
transformVertex( vB.set( 0.5, - 0.5, 0 ), mvPosition, center, worldScale, sin, cos );
transformVertex( vC.set( 0.5,  0.5, 0 ), mvPosition, center, worldScale, sin, cos );

// check first triangle
var intersect = raycaster.ray.intersectTriangle( vA, vB, vC, true, intersectPoint );

06wj · 2018-06-17T21:41:16Z

In the WebGLSpriteRenderer.js, it disabled the cullface, Sprite should not be culled( eg: when scale is -1, 1, the order will be changed, it will be displayed ), so I think the order can be any value.

WestLangley · 2018-06-17T23:22:05Z

You are correct. Since SpriteMaterial.side is not supported, we disable culling to accommodate negatively-scaled sprites.

Even so, one of your triangles faces away from the camera; the other one towards. As I suggested, I would correct that as a courtesy to future maintainers. Specify the faces in the same order as they are specified in WebGLSpriteRenderer.js.

06wj · 2018-06-17T23:35:28Z

I updated the code to specify the faces in the same order as they are specified in WebGLSpriteRenderer.js.

WestLangley · 2018-06-17T23:55:57Z

@mrdoob This is good to go. Sprite raycasting should now be pixel-perfect.

Thanks @06wj !

06wj · 2018-06-26T13:29:12Z

@mrdoob Can this be merged to the dev?

mrdoob · 2018-06-27T10:42:28Z

Thanks!

linkenneth · 2018-07-17T06:39:24Z

Thanks! Is there a demo of this on the threejs.org docs? If not, may I suggest we add something like the rawgit.com demo to the docs?

06wj · 2018-07-17T08:28:21Z

@linkenneth #14488

linkenneth · 2018-07-17T09:53:25Z

@06wj thanks!

makc · 2018-08-25T12:04:01Z

could anyone of you also copy this bit? since it does not return triangle info, one has to re-implement the entire raycast to get uvs now

makc · 2018-08-25T12:19:20Z

here is altered version:

THREE.Sprite.prototype.raycast = ( function () {

		var intersectPoint = new THREE.Vector3();
		var worldScale = new THREE.Vector3();
		var mvPosition = new THREE.Vector3();

		var alignedPosition = new THREE.Vector2();
		var rotatedPosition = new THREE.Vector2();
		var viewWorldMatrix = new THREE.Matrix4();

		var vA = new THREE.Vector3();
		var vB = new THREE.Vector3();
		var vC = new THREE.Vector3();

		function transformVertex( vertexPosition, mvPosition, center, scale, sin, cos ) {

			// compute position in camera space
			alignedPosition.subVectors( vertexPosition, center ).addScalar( 0.5 ).multiply( scale );

			// to check if rotation is not zero
			if ( sin !== undefined ) {

				rotatedPosition.x = ( cos * alignedPosition.x ) - ( sin * alignedPosition.y );
				rotatedPosition.y = ( sin * alignedPosition.x ) + ( cos * alignedPosition.y );

			} else {

				rotatedPosition.copy( alignedPosition );

			}


			vertexPosition.copy( mvPosition );
			vertexPosition.x += rotatedPosition.x;
			vertexPosition.y += rotatedPosition.y;

			// transform to world space
			vertexPosition.applyMatrix4( viewWorldMatrix );

		}

		var barycoord = new THREE.Vector3();

		function uvIntersection( point, p1, p2, p3, uv1, uv2, uv3 ) {

			THREE.Triangle.getBarycoord( point, p1, p2, p3, barycoord );

			uv1.multiplyScalar( barycoord.x );
			uv2.multiplyScalar( barycoord.y );
			uv3.multiplyScalar( barycoord.z );

			uv1.add( uv2 ).add( uv3 );

			return uv1.clone();

		}

		return function raycast( raycaster, intersects ) {

			worldScale.setFromMatrixScale( this.matrixWorld );
			viewWorldMatrix.getInverse( this.modelViewMatrix ).premultiply( this.matrixWorld );
			mvPosition.setFromMatrixPosition( this.modelViewMatrix );

			var rotation = this.material.rotation;
			var sin, cos;
			if ( rotation !== 0 ) {

				cos = Math.cos( rotation );
				sin = Math.sin( rotation );

			}

			var center = this.center;

			transformVertex( vA.set( - 0.5, - 0.5, 0 ), mvPosition, center, worldScale, sin, cos );
			transformVertex( vB.set( 0.5, - 0.5, 0 ), mvPosition, center, worldScale, sin, cos );
			transformVertex( vC.set( 0.5, 0.5, 0 ), mvPosition, center, worldScale, sin, cos );

			var uvA = new THREE.Vector2( 0, 0 );
			var uvB = new THREE.Vector2( 1, 0 );
			var uvC = new THREE.Vector2( 1, 1 );

			// check first triangle
			var intersect = raycaster.ray.intersectTriangle( vA, vB, vC, false, intersectPoint );

			if ( intersect === null ) {

				// check second triangle
				uvB.set( 0, 1 );
				transformVertex( vB.set( - 0.5, 0.5, 0 ), mvPosition, center, worldScale, sin, cos );
				intersect = raycaster.ray.intersectTriangle( vA, vC, vB, false, intersectPoint );
				if ( intersect === null ) {

					return;

				}

			}

			var distance = raycaster.ray.origin.distanceTo( intersectPoint );

			if ( distance < raycaster.near || distance > raycaster.far ) return;

			intersects.push( {

				distance: distance,
				point: intersectPoint.clone(),
				uv: uvIntersection( intersectPoint, vA, vB, vC, uvA, uvB, uvC ),
				face: null,
				object: this

			} );

		};

}() );

06wj · 2018-08-25T12:32:44Z

@makc I'll make a PR for it.

acfinity · 2018-11-22T10:56:22Z

When I set SpriteMaterial.sizeAttenuation to false, intersectObjects() does not work anymore.

Can anyone tell me what to do with this problem? thanks~

06wj · 2018-11-22T11:07:18Z

When I set SpriteMaterial.sizeAttenuation to false, intersectObjects() does not work anymore.

Can anyone tell me what to do with this problem? thanks~

#14936

acfinity · 2018-11-22T12:32:05Z

When I set SpriteMaterial.sizeAttenuation to false, intersectObjects() does not work anymore.
Can anyone tell me what to do with this problem? thanks~

#14936

thanks, my library version is 99dev, it's too new.
but after comparing 99dev and 97dev, i found that some codes had been removed, i don't understand why.

if ( ! this.material.sizeAttenuation ) {
    worldScale.multiplyScalar( - mvPosition.z );
}

mrdoob added this to the r94 milestone Jun 16, 2018

06wj force-pushed the patch1 branch from 982369c to 2ba9276 Compare June 17, 2018 09:08

06wj added 6 commits June 17, 2018 17:27

Add sprite.raycast support for scale,rotation,center

1831377

fix lint

048d260

Optimalize sprite.raycast

2b21dd6

fix lint

290f80e

update sprite.raycast

8cc8425

sprite.raycast use ray.intersectTriangle to check

42fa968

06wj force-pushed the patch1 branch from 2ba9276 to 42fa968 Compare June 17, 2018 09:28

WestLangley reviewed Jun 17, 2018

View reviewed changes

sprite.raycast refact&add comment

ec65c8f

sprite.raycast:change the triangle order

febf946

mrdoob merged commit 30bfaa8 into mrdoob:dev Jun 27, 2018

06wj deleted the patch1 branch June 28, 2018 03:24

06wj mentioned this pull request Jul 17, 2018

Examples: Add sprite raycast demo #14488

Merged

06wj mentioned this pull request Aug 25, 2018

Add Triangle.getUV & The result of sprite.raycast add uv #14782

Merged

Mugen87 mentioned this pull request Sep 7, 2018

Raycaster no longer works with Sprites using CanvasRenderer #14868

Closed


		var center = this.center;

		transformVertex( vA.set( 0, 0, 1 ), mvPosition, center, worldScale, sin, cos );

Uh oh!

Add sprite.raycast support for scale,rotation,center #14292

Add sprite.raycast support for scale,rotation,center #14292

Uh oh!

Conversation

06wj commented Jun 15, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mrdoob commented Jun 16, 2018

Uh oh!

WestLangley commented Jun 16, 2018

Uh oh!

06wj commented Jun 17, 2018

Uh oh!

WestLangley Jun 17, 2018

Choose a reason for hiding this comment

Uh oh!

WestLangley Jun 17, 2018

Choose a reason for hiding this comment

Uh oh!

WestLangley Jun 17, 2018

Choose a reason for hiding this comment

Uh oh!

06wj Jun 17, 2018

Choose a reason for hiding this comment

Uh oh!

WestLangley Jun 17, 2018

Choose a reason for hiding this comment

Uh oh!

06wj Jun 17, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

06wj Jun 17, 2018

Choose a reason for hiding this comment

Uh oh!

WestLangley commented Jun 17, 2018

Uh oh!

06wj commented Jun 17, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

WestLangley commented Jun 17, 2018

Uh oh!

06wj commented Jun 17, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

WestLangley commented Jun 17, 2018

Uh oh!

06wj commented Jun 17, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

WestLangley commented Jun 17, 2018

Uh oh!

06wj commented Jun 26, 2018

Uh oh!

mrdoob commented Jun 27, 2018

Uh oh!

linkenneth commented Jul 17, 2018

Uh oh!

06wj commented Jul 17, 2018

Uh oh!

linkenneth commented Jul 17, 2018

Uh oh!

makc commented Aug 25, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

makc commented Aug 25, 2018

Uh oh!

06wj commented Aug 25, 2018

Uh oh!

acfinity commented Nov 22, 2018

Uh oh!

06wj commented Nov 22, 2018

Uh oh!

acfinity commented Nov 22, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

06wj commented Jun 15, 2018 •

edited

Loading

06wj Jun 17, 2018 •

edited

Loading

06wj commented Jun 17, 2018 •

edited

Loading

06wj commented Jun 17, 2018 •

edited

Loading

06wj commented Jun 17, 2018 •

edited

Loading

makc commented Aug 25, 2018 •

edited

Loading