Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PerspectiveCamera - New helper method .getMarginAt(distance) #27556

Closed
Bug-Reaper opened this issue Jan 12, 2024 · 17 comments
Closed

PerspectiveCamera - New helper method .getMarginAt(distance) #27556

Bug-Reaper opened this issue Jan 12, 2024 · 17 comments

Comments

@Bug-Reaper
Copy link
Contributor

Bug-Reaper commented Jan 12, 2024

Description

Looking to PR a new PerspectiveCamera helper method .getMarginAt(distance) which returns the height, width and Vector3s for the corners of the camera's viewable area at the input arg distance.

A PerspectiveCamera's viewable margins at an arbitrary distance is a foundational piece of info you can build a lot of clean experiences from. It enables consistent position and scale for 3D designs and is practical for devices or UX with a variable aspect ratio that must be accounted for.

Some common use-cases include:
  • Guarantee your rendered content fits on the screen.
  • Scale 3D content to take up an exact percentage of available screen height/width.
  • Situations where you want to have objects cleanly enter and exit the visible area.
  • Reset a user controlled camera to an ideal-default.
  • Place a backdrop plane that exactly covers the visible area.
  • Solution

    Took a function I've been copying around for years and modified it to be a helper method like so.

    Alternatives

    Wanted some takes on how to best return the result, currently returns the info in a JSON obj like this:

    {
      topLeft:<Vector3>,
      topRight:<Vector3>,
      bottomRight:<Vector3>,
      bottomLeft:<Vector3>,
      height:<float>,
      width:<float>
    }

    I can't think of any methods that return JSON like this but perhaps it's okay? I also couldn't find any native THREE class that contains all of the above in an easily accessible way. If we don't want to just return JSON obj like this perhaps there's an opportunity to extend an existing class or create a new one for this use-case.

    @WestLangley
    Copy link
    Collaborator

    Related: https://stackoverflow.com/questions/13350875/three-js-width-of-view/13351534#13351534

    @Mugen87
    Copy link
    Collaborator

    Mugen87 commented Jan 13, 2024

    @WestLangley How about we turn your answer into a separate helper method first. Considering the upvotes for your answer, it seems a lot of devs are searching for this topic.

    Maybe something like PerspectiveCamera.getDimensionAtDistance( target: Vector2 )?

    Sidenote: Since the dimension at a distance between [near,far] is equal for orthographic cameras, we don't need an abstract method on Camera level.

    @gkjohnson
    Copy link
    Collaborator

    gkjohnson commented Jan 13, 2024

    WestLangley How about we turn your answer into a separate helper method first. Considering the upvotes for your answer, it seems a lot of devs are searching for this topic.

    Just a note that the answer provided in the link is only relevant for the symmetrical projection matrices set via the PerspectiveCamera fields. It will not provide a correct answer for manually assigned skewed or other custom projection matrices like those WebXRManager assigns.

    @WestLangley
    Copy link
    Collaborator

    1. Since the camera can be tilted, I'd suggest "frustum height" instead of "visible height".
    2. But we have to take into consideration camera.zoom, so maybe "visible height" is better after all. 🤔
    3. In addition to what @gkjohnson said, "view offset" is not accommodated.

    Anyway, I would prefer an API that is something like the following:

    frustumHeight( distance ) { // or visibleHeight()
    
    	return 2 * Math.tan( MathUtils.DEG2RAD * 0.5 * this.fov ) * distance / this.zoom;
    
    }
    
    frustumWidth( distance ) { // or visibleWidth()
    
    	return this.frustumHeight( distance ) * this.aspect;
    
    }

    @Bug-Reaper
    Copy link
    Contributor Author

    Bug-Reaper commented Jan 14, 2024

    I like the API suggestion proposed 👍

    Took that and added a couple of things:

    • Made frustumWidth() & frustumHeight() account for the .scale of the camera & its ancestors.
    • Added a frustumCorners() function to return Vector3s for the four frustum corners at a given distance.

    Plan to commit in the docs bump soon and PR.

    @Mugen87
    Copy link
    Collaborator

    Mugen87 commented Jan 14, 2024

    Do devs typically want both the frustum height and width at the same time? Or are there use cases where you only want one of them?

    If you need both most of the time, I find it a waste that frustumHeight() is effectively called twice. On the other hand if frustumWidth() and frustumHeight() are usually not called in the animation loop, I guess this tiny issue can be ignored.

    Made frustumWidth() & frustumHeight() account for the .scale of the camera & its ancestors.

    As long as we didn't solve #26659, maybe it's better to ignore the scale topic at the moment.

    @gkjohnson
    Copy link
    Collaborator

    gkjohnson commented Jan 14, 2024

    Imo this should operate on the projection matrix itself rather than using the settings on the camera. And I think it's okay to make an assumption that the projection matrix does not include other rotation or translation transforms in it.

    Again because these matrices can be off-axis it's not enough to return a width and height. If a user is using this with a WebXR frustum to place something the result will not be what's expected. Providing a 2d min bounds and 2d max bounds point should be enough to represent the view bounds at a certain distance in the frustum. These would be bounds provided in the camera frame.

    const distance = 1;
    const min = new Vector2();
    const max = new Vector2();
    camera.getViewBounds( distance, min, max )

    Do devs typically want both the frustum height and width at the same time?

    I think I would expect both of them to be provided simultaneously as all other size getter functions do.

    @Bug-Reaper
    Copy link
    Contributor Author

    Bug-Reaper commented Jan 14, 2024

    Do devs typically want both the frustum height and width at the same time? Or are there use cases where you only want one of them?

    Of my own common use-cases, most of them need both. Most of the dev inquiries about this in discord/forum/SO are for cases where both are needed.

    If you need both most of the time, I find it a waste that frustumHeight() is effectively called twice.

    I noticed this too, but did like the API layout of separate height/width methods 🤔. For performance sensitive implementation you could always run frustumHeightand calc width on your own but you'd have to be aware of the caveat w/frustumWidth.

    As long as we didn't solve #26659, maybe it's better to ignore the scale topic at the moment.

    Agree with this.


    Here's an alternate version where we have:

    • frustumDimensions( distance ) => returns: { height:<float>, width:<float> }

    • frustumCorners( distance ) => returns: { topLeft:<Vec3>, topRight:<Vec3>, bottomRight:<Vec3>, bottomLeft:<Vec3> }

    I'd like to improve and account for WebXR frustum use-cases also. Believe either of the past two implementations adds a lot of value for default PerspectiveCamera based development already.

    @Bug-Reaper
    Copy link
    Contributor Author

    Imo this should operate on the projection matrix itself rather than using the settings on the camera. And I think it's okay to make an assumption that the projection matrix does not include other rotation or translation transforms in it.

    So I see that we can extract some props from the PerspectiveMatrix:

      const aspectRatio = projectionMatrix.elements[5] / projectionMatrix.elements[0];
      const fovRadians = 2 * Math.atan(1 / projectionMatrix.elements[5]);
      // const zoomFactor = ???? Couldn't figure this one? Possibly irrelevant b/c of update to matrix[0]/matrix[5]?

    I think fov calc is marginally faster and aspect ratio is marginally slower. I guess the benefits are more to do with custom perspectiveMatrix situations.

    Also I'm a little unclear how two Vector2s provide a more accurate height/width? Think I need to better understand what type of manipulation the custom WebXR perspective matrix apply to the camera frustum?

    @gkjohnson
    Copy link
    Collaborator

    I think fov calc is marginally faster and aspect ratio is marginally slower. I guess the benefits are more to do with custom perspectiveMatrix situations.

    I'd start with a correct answer before worrying about marginal performance differences.

    Think I need to better understand what type of manipulation the custom WebXR perspective matrix apply to the camera frustum?

    As an example here's a visualization of a more extreme off axis frustum from this presentation. As you can see providing just the width and height of the frustum at a certain distance is not enough to capture the box that describes the view.

    image

    Also I'm a little unclear how two Vector2s provide a more accurate height/width?

    It's the same principle as how Box3 can describe an axis aligned bounding box with two 3d Vector3s. Each point describes the minimum and maximum extent along the x and y axis

    You can find the bottom left and top right points of this 2d bounds by projecting a ray out onto the distance you want the bounds at, similar to raycaster.setFromCamera. Note that this code is notional and should be validated:

    function getBounds( dist, minTarget, maxTarget ) {
    
      temp.set( - 1, - 1, 0.5 ).unproject( camera );
      temp.multiplyScalar( dist / Math.abs( temp.z ) );
      minTarget.x = temp.x;
      minTarget.y = temp.y;
    
      temp.set( 1, 1, 0.5 ).unproject( camera );
      temp.multiplyScalar( dist / Math.abs( temp.z ) );
      maxTarget.x = temp.x;
      maxTarget.y = temp.y;
    
    }

    @Mugen87
    Copy link
    Collaborator

    Mugen87 commented Jan 15, 2024

    So if developers want the width and height of the visible rectangular region at a certain distance, they would do this:

    camera.getBounds( dist, minTarget, maxTarget );
    const width = maxTarget.x - minTarget.x;
    const height = maxTarget.y - minTarget.y;

    How about we simplify the width/height computation and provide an additional method like Box3.getSize(). This method could internally use getBounds() and does the above computation for the user. Based on @Bug-Reaper's suggestion in #27556 (comment):

    getFrustumDimensions( dist, target ) {
    
        camera.getBounds( dist, minTarget, maxTarget );
        target.x =  maxTarget.x - minTarget.x;
        target.y = maxTarget.y - minTarget.y;
    
        return target;
    
    }

    frustumCorners() would need no further updates.

    Everyone happy with this? 😇

    @gkjohnson
    Copy link
    Collaborator

    How about we simplify the width/height computation and provide an additional method like Box3.getSize()

    Are there use cases that only need the dimensions of the view port but not the bounds? The ones all listed in OP require placing or animating something relative to the view rectangle. I suppose if a user could just be assuming that the frustum is symmetrical when placing something?

    Either way I'm fine with the proposal. I'm just curious if the width and height are actually what's needed for the provided use cases.

    @Mugen87
    Copy link
    Collaborator

    Mugen87 commented Jan 15, 2024

    I guess it does not hurt if PerspectiveCamera provides both getters for bounds and dimensions (as a convenient method). Devs can then pick the best method for their code.

    @Bug-Reaper Are you in for a PR^^?

    @Bug-Reaper
    Copy link
    Contributor Author

    Bug-Reaper commented Jan 16, 2024

    Either way I'm fine with the proposal. I'm just curious if the width and height are actually what's needed for the provided use cases.

    Some of the common scenarios lend themselves to a centerpiece object that needs to be scaled to fit perfectly within height/width. In these cases, the specific bounds are not needed since object is placed in the already-known center of view.

    Example codepen (not mine) where some 3D stuff is sized based on frustum height/width and not bounds.


    Proposal sounds good to me 🤝

    Tested getBounds() code from @gkjohnson's comment #27556 (comment), just added a slight bump so the method remains accurate when the camera.position/camera.rotation weren't (0,0,0)/(0,0,0,1).

    // ...
      temp.set( - 1, - 1, 0.5 ).unproject( camera ).applyMatrix4(cam.matrixWorldInverse);
    // ...
      temp.set( 1, 1, 0.5 ).unproject( camera ).applyMatrix4(cam.matrixWorldInverse);
    // ...
    // EDIT: Looks like we can temp.set( x, y, z ).applyMatrix4( camera.projectionMatrixInverse ) to accomplish same as above 3x faster. Plan to amend in final proposed code

    Love this approach!


    All together here's 3 methods: getBounds(), frustumDimensions(), frustumCorners()

     getBounds( distance : float, minTarget : Vector2, maxTarget : Vector2 ) : undefined
     frustumDimensions( distance : float ) : Vector2
     frustumCorners( distance : float ) : Object

    All methods should now account for asymmetric perspective matrix via the improved getBounds() foundation. Plan to add a bit more polish, doc updates, and PR.

    @Mugen87
    Copy link
    Collaborator

    Mugen87 commented Jan 16, 2024

    @Bug-Reaper Some feedback for your commit:

    • Instead of using this.updateMatrixWorld(); in getBounds(), do this:
    this.updateWorldMatrix( true, false );
    • Please use no THREE namespace in modules.
    • Ideally you put temporary variables into the module scope.

    @Bug-Reaper
    Copy link
    Contributor Author

    @Bug-Reaper Some feedback for your commit:

    • Instead of using this.updateMatrixWorld(); in getBounds(), do this:
    this.updateWorldMatrix( true, false );
    • Please use no THREE namespace in modules.

    • Ideally you put temporary variables into the module scope.

    Thanks @Mugen87 ! Got these changes in and added docs 👌

    As an aside, realized that getBounds/frustumDimensions are agnostic to the matrix world lol. We now run this.updateWorldMatrix( true, false ); in frustumCorners instead as it's still relevant there.

    Also did some simple benchmarks and these all should be renderLoop safe ~0.01ms or less

    @Bug-Reaper
    Copy link
    Contributor Author

    Bug-Reaper commented Jan 20, 2024

    Thanks everyone 👍

    New PerspectiveCamera methods added #27574

    Method names updated here #27605

    getViewSize- height/width of camera's viewable rectangle at a distance.
    getViewBounds - Min/Max Vectors of camera's viewable rectangle.

    Both should be available in version >161

    Plan to start a new discussion/PR for a CameraUtils method that gets the Vector3 positions of the camera corners (in world-space) at a given distance at some later time.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants