Hi,when I run viz.py

[elegentbamboo]

Thanks for pointing this out. I have added the attention map visualization code.

We visualize the attention maps from the last attention layer for each of the four heads on a subset of Town05_tiny data since it contains lots of intersections. The main change in the model (model_viz.py) is to return the attention maps along with the waypoint predictions.

Let me know if there are any issues with running the code.

Originally posted by @ap229997 in #18 (comment)

[elegentbamboo]

When I run viz.py, I meet an error:
Traceback (most recent call last):
File "/home/lyl/transfuser_main/leaderboard/team_code/transfuser/viz.py", line 83, in
pred_wp, attn_map = model(fronts, lidars, target_point, gt_velocity )
File "/software/Anaconda3/envs/transfuser/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/lyl/transfuser_main/leaderboard/team_code/transfuser/model_viz.py", line 447, in forward
z = self.join(fused_features)
File "/software/Anaconda3/envs/transfuser/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/software/Anaconda3/envs/transfuser/lib/python3.7/site-packages/torch/nn/modules/container.py", line 139, in forward
input = module(input)
File "/software/Anaconda3/envs/transfuser/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/software/Anaconda3/envs/transfuser/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 96, in forward
return F.linear(input, self.weight, self.bias)
File "/software/Anaconda3/envs/transfuser/lib/python3.7/site-packages/torch/nn/functional.py", line 1847, in linear
return torch._C._nn.linear(input, weight, bias)
TypeError: linear(): argument 'input' (position 1) must be Tensor, not tuple

I can't figure this out......

[ap229997]

Thanks for pointing this out. I updated the code, try again.

[elegentbamboo]

After I tried, another error occurs:
First, in the line 94 of viz.py, I guess it is data['lidars'] not data['lidar_bevs'] because data['lidar_bevs'] don't seem to exist.
But when I change it, in the code below, 'img' and 'lidar_bev ' don't have the same number of dimensions, so they can't be combined.
for idx in range(args.batch_size):
img = np.transpose(data['fronts'][0][idx].numpy(), (1,2,0))
lidar_bev = (data['lidar_bevs'][0][idx].squeeze(0).numpy()*255).astype(np.uint8)
lidar_bev = np.stack([lidar_bev]*3, 2)
combined_img = np.vstack([img, lidar_bev])
I don't quite understand how you transform the dimension of input data here.

[ap229997]

I added a separate data loader for visualization which should resolve the issues.

After I tried, another error occurs: First, in the line 94 of viz.py, I guess it is data['lidars'] not data['lidar_bevs'] because data['lidar_bevs'] don't seem to exist. But when I change it, in the code below, 'img' and 'lidar_bev ' don't have the same number of dimensions, so they can't be combined. for idx in range(args.batch_size): img = np.transpose(data['fronts'][0][idx].numpy(), (1,2,0)) lidar_bev = (data['lidar_bevs'][0][idx].squeeze(0).numpy()*255).astype(np.uint8) lidar_bev = np.stack([lidar_bev]*3, 2) combined_img = np.vstack([img, lidar_bev]) I don't quite understand how you transform the dimension of input data here.

The input lidar_bevs is in the range [0,1]. It is converted to [0,255] and broadcasted to 3 channels to make it more convenient to concatenate with image input to compute the attended regions.