Post-processing depth images

[lucaBartolomei]

Hi,

thanks a lot for this nice tutorial!
I have a (basic) question about the depth images. In the following, I refer to this part of the code. If I got it right, the process is as follows:

1. The shader gets the depth value;
2. The value is mapped between 0 and 1 considering the near and far planes;
3. The output value then gets "compressed";
4. The final output is then the depth image, which is saved to a PNG file.

I now need to do some offline post-processing (i.e. outside Unity). How can I get back the original depth value for every pixel from the stored image?
Thanks a lot for the help!

[kartik-innovasea]

I have the same doubt, were you able to find out this?

[cloudlakecho]

Thanks for Immersive-Limit for sharing the nice code. It generated good outputs especially depth map and segmentation. Segmentation working great.

At the same time, I have the same question on the depth image. This is my nice depth image (top is depth and bottom is optical as reference):

and the pixel values (RGB in PNG) in the depth image are:
     [(255, 255, 255), (221, 221, 221), (222, 222, 222), (194, 194, 194), (195, 195, 195), (223, 223, 223), (196, 196, 196), (224, 224, 224), (225, 225, 225), (197, 197, 197), (226, 226, 226), (227, 227, 227), (198, 198, 198), (199, 199, 199), (228, 228, 228), (229, 229, 229), (217, 217, 217), (218, 218, 218), (219, 219, 219), (220, 220, 220), (234, 234, 234), (235, 235, 235), (236, 236, 236), (237, 237, 237), (249, 249, 249), (248, 248, 248), (250, 250, 250), (251, 251, 251), (233, 233, 233), (230, 230, 230), (232, 232, 232), (231, 231, 231), (107, 107, 107), (58, 58, 58)]

I see the depth to RGB are:
     (1) float near = _ProjectionParams.y;
     (2) float far = _ProjectionParams.z;
     (3) linearZFromNear = (depth01 - near/far) * (1 + near/far);
     (4) float k = 0.25; // compression factor
     (5) RGB = pow(linearZFromNear, k);

According to Unity Shader Variable in the manual, the "_ProjectionParams.y" should be Camera Clipping Near and "_ProjectionParams.z" is Far. I reversed the conversion to see the depth in real life (meter I guess) and the results are from 11,040,483.927439027 to 4,125,122,561.0006104 .

The result seems too large because the farest barrel is around 100 meter from the camera.

I have two questions:
     Do I missing something in the conversion back?
     If possible, is there way to use "DepthMultichannel" instead of "DepthCompressed" (ref: "UberReplacment.shade" code)?

---

Here is my conversion code in Python as reference (I assume "DepthCompressed" not "DepthMultichannel" because RGB are the same):

# Portion of conversion in UberReplacement.shader
# float near = _ProjectionParams.y;
# float far = _ProjectionParams.z;
# return (depth01 - near/far) * (1 + near/far);
# float linearZFromNear = Linear01FromEyeToLinear01FromNear(depth01); 
# float k = 0.25; // compression factor
# return pow(linearZFromNear, k);

k = 0.25  # Compression factor      
# These are from Main Camera Clipping Planes setting
near, far = 250, 10000
pixel_val, cur_depth = 0, 0.0
for each_depth in RGB:
    pixel_val = each_depth[0]
    cur_depth = math.pow(pixel_val, 1 / k)
    cur_depth /= (1 + near / far)
    cur_depth += near / far
    if (DEBUGGING):
        print ("Pixel value: {}".format(pixel_val))
        print ("Depth in meter: {}".format(cur_depth))