Tensor size matching error

[hdacnw]

Dear authors, thanks for the great work! I'm trying to train with a custom dataset containing images and Pseudo Dense Representations Generation of size H * W * 1 and have changed the Resnet encoder dimension from 2 to 1 accordingly. However, I'm getting RuntimeError: The size of tensor a (10) must match the size of tensor b (15) at non-singleton dimension 3 at x = input_features[-1] + beam_features[-1] in depth_decoder.py. I guess it's related to scaling as the Pseudo Dense Representations Generation has the original scale while for image it's scaled down. However in your original inputs["2channel"] = self.load_4beam_2channel(folder, frame_index, side, do_flip) it seems that there's no scaling down involved. Do you have any idea what might be the issue? Thanks!

[fengziyue]

Hi:
Thank you for your interest, could you provide the size of your image and PDR?

[hdacnw]

Hi, thanks for the quick reply! The original size of my image is (360x480x3) and (360x480x1) for PDR. While passing --height=224 --width=320 with a batch size of 12 the shape of input_features[-1] is torch.Size([12, 512, 7, 10]) and the shape of beam_features[-1] is torch.Size([12, 512, 12, 15]), hence causing the above error. When passing instead --height=384 --width=480, I'm getting another error RuntimeError: The size of tensor a (24) must match the size of tensor b (23) at non-singleton dimension 2 instead at line x += [input_features[i - 1] + beam_features[i - 1]] in depth_decoder.py, as the shape of input_features[i - 1] and beam_features[i - 1] is now torch.Size([12, 256, 24, 30]) and torch.Size([12, 256, 23, 30]) respectively.

[fengziyue]

Hi：
Could you check the shape of input_features[0] and beam_features[0]?

[hdacnw]

Hi,
For --height=384 --width=480: torch.Size([12, 64, 192, 240]), torch.Size([12, 64, 180, 240])
For --height=224--width=320: torch.Size([12, 64, 112, 160]), torch.Size([12, 64, 180, 240])

[fengziyue]

I think it probably comes from the gen2channel.py
The image will be resized to the "--height=224 --width=320" in data loader but the PDR will not. the size of PDR is determined in the gen2channel.py

[fengziyue]

Why does your PDR only have one channel? did you discard the confidence channel?
Another thing that needs to note is when you resize the PDR or depth map from sparse lidar, make sure to use max-pooling instead of interpolation which will blur the sparse depth points. you can project the resized depth map or PDR to a 3D point cloud to double-check it is not blurred.

[hdacnw]

I've discarded the confidence channel just for experimentation. Thanks for your advice!

I think it probably comes from the gen2channel.py The image will be resized to the "--height=224 --width=320" in data loader but the PDR will not. the size of PDR is determined in the gen2channel.py

Yes indeed. After resizing the PDR the problem goes away. I was using a custom generation script and missed out on that part. Thank you so much!