How to generate a depth image with adjusted range

[tadashi-hiraoka]

Hi there,

Thank you for publishing a very valuable tool.

My objective is to generate a ring depth image. We can generate a distance image with the following config.

    {
      "module": "renderer.RgbRenderer",
      "config": {
        "render_distance": True,
        "distance_start": 140.5,
        "distance_range": 0.2,
        "render_depth": True,
        "depth_start": 140.0,
        "depth_range": 0.2,
      }
    },

However, we cannot generate the expected depth image with that config. The maximum value of numpy.array in the generated depth image was 1e10 and the minimum value was 140.0. I expected the maximum value to be 140.2 and the minimum value to be 140.0. What kind of method is there?

The images below are the generated distance and depth images. In the distance image, the pixel values inside and outside the ring are different. In the depth image, the pixel values inside and outside the ring will be about the same, and the pixel values of the ring defects will be different. The defect in the ring can be seen below in the distance image.

[cornerfarmer]

Hey @tadashi-hiraoka

there is actually no such config parameter as depth_start or depth_range. The depth image contains always the full range of values. If you want to clip the values here, you would need to do that yourself in postprocessing.
For distance images, the start / range parameter exist and should work as expected. If I use your renderer config in the basic example, it does so at least.

Maybe you can give a bit more information how the rest of your config looks like and what exactly you want to achieve.

[tadashi-hiraoka]

Hi @cornerfarmer

Thank you for your reply.

I will explain with the image of "distance in 0.hdf5". The center of the distance image is (x, y, z) = (0,0,0) with the + x axis to the right, the + y axis to the top, and the + z axis to the front. The z-values for that ring are almost the same, but for that distance image, the inside of the ring is closer and the outside is farther.

My requirement is to have the same distance inside and outside the ring. By doing so, the defects in the ring will be noticeable.

Is it possible to solve this problem by treating orthographic distance images instead of depth images?

[cornerfarmer]

Just to make sure, you are talking about the red area inside the ring vs the red area outside, right? These values should all be 140.7 in your case.

Or are you talking about the area on the ring itself?

[tadashi-hiraoka]

I'm sorry. It caused confusion.

The red area is the background and has nothing to do with my interests. I am talking about the area on the ring itself. The inside of the ring is blue and the outside is gradually light blue. The defect is yellow.

[cornerfarmer]

Ok got it! This behaviour is totally normal, as there are points on ring which are closer to the camera and points that are further away. Thats why they are having different values in the depth/distance images
Also using an orthographic camera will not change that, unless the area on the ring is completely parallel to the camera plane, which is never the case as a ring is smooth and round.
Make sure you understand how distance and depth images work:

I suspect that in the end you want a mask of the ring's defect. One thing you could try is to first render the depth image of a completely intact ring and then subtract it from the image of the corrupted ring. In this way you should get higher differences in the area of the defect.
Let me know if you need any further help.

[tadashi-hiraoka]

The shape of the ring is a pipe like the URL below, and its top surface is flat.

https://ncalculators.com/geometry/pipe-volume-calculator.htm

Its top surface of the ring can be approximately parallel to the orthographic camera. Is it possible to use orthographic projection with BlenderProc?

[cornerfarmer]

Ah ok, indeed this pipe has a parallel surface. In the distance image, different points have still different values, as they have different distances to the camera. Therefore, you should use the depth image, as there, all points on a plane parallel to the camera have the same values (as you also noticed in your first post). So, why can you not just use the depth image? If its only about the clipping, you can do that yourself in a postprocessing step, e.q. via np.clip(image, 140, 140.2).

If you really want to use only distance images, you can also transform them to depth images by using the pose processing step:

BlenderProc/examples/random_room_constructor/config.yaml

Line 112 in fe5ea25

"module": "postprocessing.Dist2Depth",

Orthographic projection is not yet possible with BlenderProc, however you could add the functionality by yourself, as described here

[tadashi-hiraoka]

Thank you for your advice. I will try the method of generating a depth image using the clipping you showed. If that doesn't work, I'll try the orthographic projection method of generating a distance image. Either method is fine.

[tadashi-hiraoka]

The method using clipping for the depth image you showed worked fine. The code below worked. Thank you very much.

import h5py
import numpy as np
from PIL import Image

file_path = "0.hdf5"
with h5py.File(file_path, "r") as data:
    value = np.array(data["depth"])
    value = ((value - 140.0) * 255 * 5)
    value = np.clip(value, 0, 255)
    value = value.astype(np.uint8)
    pil_image = Image.fromarray(value)
    pil_image.save("depth.png")