-
-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add screenX, screenY and screenZ #1553
Comments
that sounds great. i think we need to use projection of vectors on planes. it would be particularly nice if we could do this. //example code
var screenVector = threeDVector.getCanvasPoint(); |
Hi @brendandawes , Fortunately for us it is exactly what OpenGL and the graphics pipeline do when we push vertices to the GPU to be rendered. All this steps are done using transformation 4x4 matrices in different stages changing the space coordinates of our vertices :
Perspective vs NDC space (credits to songho). So depending on our needs we have to simulate this steps (world to canvas coordinates) or undo them (canvas to world coordinates), to get our vertices in the right space 😉 Here's what you're looking for : /* unlicense.org */
/* Multiply a 4x4 homogeneous matrix by a Vector4 considered as point
* (ie, subject to translation). */
function multMatrixVector(m, v) {
if (!(m instanceof p5.Matrix) || !(v instanceof p5.Vector)) {
print('multMatrixVector : Invalid arguments');
return;
}
var _dest = createVector();
var mat = m.mat4;
// Multiply in column major order.
_dest.x = mat[0] * v.x + mat[4] * v.y + mat[8] * v.z + mat[12];
_dest.y = mat[1] * v.x + mat[5] * v.y + mat[9] * v.z + mat[13];
_dest.z = mat[2] * v.x + mat[6] * v.y + mat[10] * v.z + mat[14];
var w = mat[3] * v.x + mat[7] * v.y + mat[11] * v.z + mat[15];
if (Math.abs(w) > Number.EPSILON) {
_dest.mult(1.0 / w);
}
return _dest;
}
/* Project a vector from Canvas to World coordinates. */
function projectCanvasToWorld(canvas, vCanvas) {
// Retrieve the ModelView and Projection matrices.
var mv = canvas.uMVMatrix.copy();
var p = canvas.uPMatrix.copy();
// Compute the ModelViewProjection matrix.
var mvp = mv.mult(p);
// Inverts the MVP.
var invMVP = mvp.invert(mvp);
// Transform the canvas vector to Normalized Device Coordinates (in [-1, 1]³),
// Here viewport is (0, 0, drawingBufferWidth, drawingBufferHeight).
var vNDC = createVector();
vNDC.x = (-1.0 + 2.0 * (vCanvas.x / canvas.GL.drawingBufferWidth));
vNDC.y = (-1.0 + 2.0 * (vCanvas.y / canvas.GL.drawingBufferHeight));
vNDC.z = (-1.0 + 2.0 * (vCanvas.z));
// Transform vector from NDC to world coordinates.
var vWorld = multMatrixVector(invMVP, vNDC);
return vWorld;
}
/* Project a vector from World to Canvas coordinates. */
function projectWorldToCanvas(canvas, vWorld) {
// Calculate the ModelViewProjection Matrix.
var mvp = (canvas.uMVMatrix.copy()).mult(canvas.uPMatrix);
// Transform the vector to Normalized Device Coordinate.
var vNDC = multMatrixVector(mvp, vWorld);
// Transform vector from NDC to Canvas coordinates.
var vCanvas = createVector();
vCanvas.x = 0.5 * (vNDC.x + 1.0) * canvas.GL.drawingBufferWidth;
vCanvas.y = 0.5 * (vNDC.y + 1.0) * canvas.GL.drawingBufferHeight;
vCanvas.z = 0.5 * (vNDC.z + 1.0);
return vCanvas;
} Note : Matrices operations being costly (especially inversion) you might want to optimize this for bulk transforms. Due to a lack of 2d / 3d interop I'm not one hundred percent sure of the need to implement this to p5.js right now, but this could still be useful for mouse picking in 3d (especially for interactive applications). Cheers 👍 |
Wow! @tcoppex Thanks for such a great explanation @AkashGutha |
@AkashGutha - this is cool. I don't fully understand why some kind of conversion function wouldn't be useful to implement in p5js right now, though - it sure seems like it would be handy! |
thanks for the suggestion @brendandawes! I think this is beyond the scope of p5 for now, we're trying to focus on getting core webgl working before expanding the api. but this could make a nice addon library using some of @tcoppex's solution above. see the docs here for how to create an addon: https://github.com/processing/p5.js/wiki/Libraries#creating-a-new-library |
@tcoppex Thanks to your wonderful piece of code I've started to build the interface I've wanted to do for a while as a way to navigate my work archive. A few work in progress vids here: https://www.instagram.com/p/BeV2tckn0dX/?taken-by=brendandawes The text overlays are all html divs and I'm using the EasyCam library for navigation, together with the sound lib. Amazing what Javascript can do these days. Love p5js! @lmccart |
hi there... i saw this code as i was wandering around to find a way to do this transformation... i just got confused... in your function projectCanvasToWorld(canvas, vCanvas), what is canvas and what is vCanvas? thank you so much again for your help... |
Hi @nevahid, It's old code but to my remembrance Hope that helps. |
@tcoppex thank you very much. i'm trying to figure it. but it gives me confusing positions, maybe i'm wrong somewhere... what i try to do is to get mouse position in p5.easycam camera... as i had to set the camera canvas to null, i used the p5.renderer as the canvas... maybe that is the point... |
Can anyone provide some insight for how the above code can be used? I don't quite understand what params should be fed as |
@ffd8: Here is my solution. It works with 2D and WEBGL mode. Both have its own difficulties: You'll find the function here: And how it's used here: |
@bohnacker Really nice solution and easy to implement, many thanks! |
I don't understand if your library is suitable for my needs: I want to draw with the mouse over the sides of a cube which has a corner in the origin; so how do I convert MouseX and MouseY to 3d coordinates using your library? Or should I use @nevahid solution instead? Or p5.projection? Or this snippet? |
Hi @jumpjack, I found a jsfiddle that is more or less doing what you need. Unfortunately it is using three.js and not p5.js. |
@nevahid solution is working quite good ... as long as I don't rotate the scene: orbitContol() disrupts everything... :-( |
How can I know the rotation angles used by orbitControl(), i.e. the angles of X, Y and Z w.r.t. screen normal? |
It looks like this topic is actually named "3d picking". |
I found the answer: createCamera()._getLocalAxes() returns {x[], y[], z[]}, which represents orientation of axes after moving them with orbitControl(); asin(cam._getLocalAxes().x[0]) will give you rotation of X axis of the scene w.r.t X axis of the camera, and so on. Also cameraNear, cameraFar and cameraFOV are interesting, although I dind't yet understand how to use them. |
I found in p5js libraries the P5XR library and its getRayFromScreen(x, y) ad intersectsPlane([ray]) ... but I don't get how to port the algorithm to "standard" p5JS (not VR). |
Would be great if you could add screenX etc for getting the 2D screen coordinates of a 3D vector. Currently trying to map HTML text to a 3D object and maths is making my head hurt!
Thanks for all the great work.
The text was updated successfully, but these errors were encountered: