Training the MOTR, Deformable DETR Weights

[tdchua]

Hello, I am trying to train the MOTR model, but I am not sure which weights did you use for the Deformable DETR.

You can download COCO pretrained weights from Deformable DETR. Then training MOTR on 8 GPUs as following:

I am not sure which weights did you choose, because there are several listed on the Deformable DETR Github page. I went to this section of the page.
https://github.com/fundamentalvision/Deformable-DETR#main-results

Please advise, thank you!

[dbofseuofhust]

Hello, I am trying to train the MOTR model, but I am not sure which weights did you use for the Deformable DETR.

You can download COCO pretrained weights from Deformable DETR. Then training MOTR on 8 GPUs as following:

I am not sure which weights did you choose, because there are several listed on the Deformable DETR Github page. I went to this section of the page. https://github.com/fundamentalvision/Deformable-DETR#main-results

Please advise, thank you!

Hi~, you should use r50_deformable_detr_plus_iterative_bbox_refinement.

[tdchua]

Thank you very much for replying! Unfortunately, I ran into this error when training the model.

Traceback (most recent call last):
File "main.py", line 396, in
main(args)
File "main.py", line 335, in main
model, criterion, data_loader_train, optimizer, device, epoch, args.clip_max_norm)
File "/home/canzone/Desktop/research/repositories/MOTR/engine.py", line 106, in train_one_epoch_mot
outputs = model(data_dict)
File "/home/canzone/anaconda3/envs/MOTR/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/canzone/anaconda3/envs/MOTR/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 799, in forward
output = self.module(*inputs[0], **kwargs[0])
File "/home/canzone/anaconda3/envs/MOTR/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/canzone/Desktop/research/repositories/MOTR/models/motr.py", line 605, in forward
frame_res = self._forward_single_image(frame, track_instances)
File "/home/canzone/Desktop/research/repositories/MOTR/models/motr.py", line 514, in _forward_single_image
hs, init_reference, inter_references, enc_outputs_class, enc_outputs_coord_unact = self.transformer(srcs, masks, pos, track_instances.query_pos, ref_pts=track_instances.ref_pts)
File "/home/canzone/anaconda3/envs/MOTR/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/canzone/Desktop/research/repositories/MOTR/models/deformable_transformer_plus.py", line 162, in forward
memory = self.encoder(src_flatten, spatial_shapes, level_start_index, valid_ratios, lvl_pos_embed_flatten, mask_flatten)
File "/home/canzone/anaconda3/envs/MOTR/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/canzone/Desktop/research/repositories/MOTR/models/deformable_transformer_plus.py", line 266, in forward
output = layer(output, pos, reference_points, spatial_shapes, level_start_index, padding_mask)
File "/home/canzone/anaconda3/envs/MOTR/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/canzone/Desktop/research/repositories/MOTR/models/deformable_transformer_plus.py", line 232, in forward
src2 = self.self_attn(self.with_pos_embed(src, pos), reference_points, src, spatial_shapes, level_start_index, padding_mask)
File "/home/canzone/anaconda3/envs/MOTR/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/canzone/Desktop/research/repositories/MOTR/models/ops/modules/ms_deform_attn.py", line 111, in forward
+ sampling_offsets / input_spatial_shapes[None, None, None, :, None, (1, 0)]
RuntimeError: CUDA out of memory. Tried to allocate 28.00 MiB (GPU 0; 10.92 GiB total capacity; 9.39 GiB already allocated; 14.69 MiB free; 10.08 GiB reserved in total by PyTorch)

Is there any way to reduce the size of the input or to change a parameter so that it can run smoothly? This happened after epoch 49 by the way.