How do I get the angle between the camera and a pixel point in an aligned frame?

[ydixon]

Hello, I'm newbie with camera maths and configuration. My goal is find a object's height (distance from ground), knowing the camera height. I was thinking I could probably calculate it if I can obtain the angle between the camera and the object with triangle rules.

I have Hc, the height of the camera. Do, distance of the object given by aligned depth frame.
I'm trying to find Ho in the diagram. Is there a method to achieve this?

[MartyG-RealSense]

Hi @ydixon The links below discuss methods of using mathematics to obtain the angle of an object observed by a RealSense camera.

#5553

https://answers.ros.org/question/300639/how-to-get-the-distance-and-the-angle-distance-and-bearing-from-a-depth-camera/

[ydixon]

Thanks for the links @MartyG-RealSense.

Why would you need the distance to find the angle?

Example:
Say you are using RGB camera and have set it to 720p resolution. W = 1280, H = 720
Assume the HFOV = 62 and VFOV=46 (Check values for your camera, assumed at random)
Center of camera in image = (W/2,H/2) = (640,360)

Horizontal Angle of say random pixel (x,y) = ((x - W/2)/(W/2))(HFOV/2)
Vertical Angle of say random pixel (x,y) = ((y - H/2)/(H/2))(VFOV/2)

For a random pixel say (320,180) it will be:
H Angle = ((320 - 640)/640)(62/2) = - 15.5 degree
H Angle = ((180 - 360)/360)(46/2) = - 11.5 degree

You can get the euclidean angle by using simple (h-angle^2+v-angle^2)^0.5

By this formula, if my object is in the middle of the screen. The angle will be zero.
I currently have a D435 attached to upper corner of the room looking down.
If I use 0 angle to calculate the object height, it will be the same as the camera height? Am I missing something?

[MartyG-RealSense]

In another case where the camera was mounted in an elevated position and tilted downwards towards objects on the floor instead of facing straight ahead, a RealSense user provided the following advice:

"If the floor and camera plane is parallel, then the depth should be same. The depth calculated through stereo disparity always gives perpendicular distance to camera plane. So it won't matter if the pixel is at the center or at the edge of the image".

#6800 (comment)

[ydixon]

Sorry, I'm confused after reading the comment and the thread. I don't understand how it's related to my question.

My goal is to find for any pixel point in the depth frame, it's distance from the ground (height), given that I know the exact height of the camera.

I have the depth value (distance from camera) for a point. If I have the camera's angle to the object, I can calculate the object's height by trig rules.

After reading the formula above from the link you shared, I wanted to point out a counter example. So I used center pixel as example. If my camera is 2 meter height above the ground, tilted downward.
By the formula, I will get 0 degrees. Thus, the object height will be same as the camera?

For #6800 (comment)
Is it talking about how the depth values of a flat surface not constant in the depth frame? I'm don't know how it's related to my problem.

[MartyG-RealSense]

The intention of the quote was to highlight some of the complexities of the maths that can be involved in performing calculations from the elevated and tilted camera perspective that you are using.

The online calculator in the link below for finding the height of an object by providing distance and angle may be helpful in developing a calculation.

https://www.calctool.org/CALC/math/trigonometry/height

This page states: "The height of an object is calculated by measuring the distance from the object and the angle of elevation of the top of the object. The tangent of the angle is the object height divided by the distance from the object. Thus, the height is found".

As an example, providing the calculator with an angle of 36 degrees and a distance from the object of 2 meters resulted in a calculated height for the object of 1.45309 meters.

I believe that if you wanted to measure in the opposite direction (diagonally downwards from a point elevated above the object instead of diagonally upwards from the ground), you could subtract '180' from the angle - for example so that 36 degrees becomes -144 degrees. Or alternatively, 216 degrees (36+180).

Checking this method in the online calculator, 216 degrees angle and 2 meters distance to object also resulted in an object height of 1.45309 meters.

More information on the maths of angle to height conversion can be found by googling for convert angle to height

[ydixon]

Thanks for the detailed response. I've reread the threads and rethink about it again.

In the #5553, it's calculating only the angle of pixel from the camera, from the camera perspective.

If I use the angles from above directly, it won't be right, since I didn't include the angle of camera tilted downward.

So do I have to measure the angle tilted when I setup the camera and combine the angle(Theta_v) from FOV to get the real theta?

[MartyG-RealSense]

If the original calculation is based on the assumption that the camera is looking straight ahead, I'm not sure that tilting the camera downward would radically change the object height measurement, as the line of vision would likely swing inward towards the center of the object.

[ydixon]

I'm not quite sure what it means when you say the camera is looking straight ahead. We have a d435 attached to ceiling, slightly tilted downward roughly around 45 degrees. When you mention "straight ahead", do you mean the camera direction is parallel to the floor or it's always aiming toward the object?

A sample use case, would be trying to detect the height(distance from ground) of the person's head

[MartyG-RealSense]

I meant that the calctool online calculator likely makes the assumption that the viewpoint is pointing straight ahead and that if you were to use a similar height calculation formula with your RealSense camera then tilting the camera probably would not have much impact on the final calculated object height.

[MartyG-RealSense]

Hi @ydixon Do you require further assistance with this case, please? Thanks!

[ydixon]

Hi @MartyG-RealSense. I ended up using a different approach. Thank you for your assistance!

Btw, the point cloud on the floor(flat ground) looks curvey / wavey. Is there any built-in functionality to deal with it?

[MartyG-RealSense]

Hi @ydixon That's great to hear that you found a solution that worked for you!

Regarding the waviness of the point cloud, moving the camera further back from the floor may reduce waviness. If that is not possible, you could try using the Medium Density camera configuration preset to sharpen curved / blobby edges. You can test this quicky by selecting Medium Density from the drop-down selector in the options side-panel of the RealSense Viewer.

If the preset does not resolve your waviness problem then a discussion in the link below (and the list of cases at the bottom of that discussion) look at the phenomenon of 'ghost noise' that can cause waviness.

#4553

[ydixon]

My viewer is crashing when I try to change to medium density. Any idea why?

[MartyG-RealSense]

Which RealSense SDK version are you experiencing the preset-changing error in, please?

[MartyG-RealSense]

Hi @ydixon Do you require further assistance with this case, please? Thanks!

[ydixon]

@MartyG-RealSense Sorry, I wasn't able to get remote access to the machine till now.

I can't find the installer. It should be 2.16.2?

[MartyG-RealSense]

SDK version 2.16.2 is very old, dating from October 2018. The current version at the time of writing this is 2.48.0.

https://github.com/IntelRealSense/librealsense/releases/tag/v2.48.0

If you wish to use 2.48.0 and you have a Windows computer then you can use an automated installer file in the Assets file list at the bottom of the version information listing.

Otherwise, if you are using Linux then you can either build the latest SDK from Debian packages:

https://github.com/IntelRealSense/librealsense/blob/master/doc/distribution_linux.md

Or you can build the SDK from source code, also available in the Assets list.

https://github.com/IntelRealSense/librealsense/blob/master/doc/installation.md

Use of special instructions will be required if you are using a Raspberry Pi or Nvidia Jetson board.

[ydixon]

@MartyG-RealSense Thank you. Closing the issue.