Question regarding the architecture

[oOXpycTOo]

Hi, I really appreciate your work. An impressive idea. I want to ask a question regarding the 3D CNN architecture:

1. It's written, that CBR3D0 takes T which is cost volume of shape (BATCH_SIZE, CHANNELS (32), N_DEPTH, HEIGHT / 4, WIDTH / 4) and I which are concatenated images, as I guess (BATCH_SIZE, 9, 1, HEIGHT, WIDTH). So, there appears to be a shape inconsistency. Could you, please, clarify this one, if possible.
2. It also states, that CBT3D2 takes outputs from CTB3D1 and CBR3D2, which is a total of 48 channels instead of 64. Is it a typo?
   Thanks in advance for your reply!

[apchenstu]

Hi there, thanks, sorry about we didn't explain the network structure clearly. we will revise it in the near future.

1. it is not concatenated directly, we also project the color to the volume, the shape is (BATCH_SIZE, 9, N_DEPTH, HEIGHT / 4, WIDTH / 4).
2. oh, the chns are incorrect in the CTB3D_1,2,3. you just bypass CTB3D_0, and 1-3 should be 64/32, 32/16, 16/8.

[oOXpycTOo]

Oh, great, thank you! That explains everything. The only questions left are:

1. Do you downsample color images and then project them onto a reference plane?
2. When doing raymarching, it's no longer needed to downsample images, as we're working in NDC, is it correct?

[apchenstu]

exactly, yeah we don't have to downscale the images when doing raymarching.

[oOXpycTOo]

I see. Great thanks for all of your replies. They helped a lot in understanding your paper! Have a nice day!