Error while train when get dataset

[pjy125175]

Hi,When I train the model with python src/train.py with mot17 deformable multi_frame tracking output_dir=models/mot17_deformable_multi_frame, the following errors occur：
Traceback (most recent call last):
File "src/train.py", line 357, in
train(args)
File "src/train.py", line 284, in train
visualizers['train'], args)
File "/home/ubuntu/track/trackformer/src/trackformer/engine.py", line 119, in train_one_epoch
for i, (samples, targets) in enumerate(metric_logger.log_every(data_loader, epoch)):
File "/home/ubuntu/track/trackformer/src/trackformer/util/misc.py", line 230, in log_every
for obj in iterable:
File "/home/ubuntu/anaconda3/envs/trackformer/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 345, in next
data = self._next_data()
File "/home/ubuntu/anaconda3/envs/trackformer/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 856, in _next_data
return self._process_data(data)
File "/home/ubuntu/anaconda3/envs/trackformer/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 881, in _process_data
data.reraise()
File "/home/ubuntu/anaconda3/envs/trackformer/lib/python3.7/site-packages/torch/_utils.py", line 395, in reraise
raise self.exc_type(msg)
IndexError: Caught IndexError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/ubuntu/anaconda3/envs/trackformer/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "/home/ubuntu/anaconda3/envs/trackformer/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/ubuntu/anaconda3/envs/trackformer/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/ubuntu/track/trackformer/src/trackformer/datasets/mot.py", line 49, in getitem
img, target = self._getitem_from_id(idx, random_state, random_jitter=False)
File "/home/ubuntu/track/trackformer/src/trackformer/datasets/coco.py", line 63, in _getitem_from_id
img, target = self.prepare(img, target)
File "/home/ubuntu/track/trackformer/src/trackformer/datasets/coco.py", line 220, in call
masks = convert_coco_poly_to_mask(segmentations, h, w)
File "/home/ubuntu/track/trackformer/src/trackformer/datasets/coco.py", line 177, in convert_coco_poly_to_mask
rles = coco_mask.frPyObjects(polygons, height, width)
File "pycocotools/_mask.pyx", line 292, in pycocotools._mask.frPyObjects
IndexError: list index out of range

I found that the data set is not a pair. But the dataset I generated with this command：python src/generate_coco_from_mot.py.I don't know what's wrong.

[timmeinhardt]

What do you mean by "data set is not a pair"? Did the generate_coco_from_mot.py scripts finish successfully? This looks like a data path issue.

[pjy125175]

Thanks for your answer,I found that the problem was to set the masks of train.yaml to true.
I want to generate_ attention_ maps and masks is set to true in track.yaml to evaluate，but I got：
RuntimeError: Error(s) in loading state_dict for DeformableDETRSegmTracking:
Missing key(s) in state_dict: "bbox_attention.q_linear.weight", "bbox_attention.q_linear.bias", "bbox_attention.k_linear.weight", "bbox_attention.k_linear.bias", "mask_head.lay1.weight", "mask_head.lay1.bias", "mask_head.gn1.weight", "mask_head.gn1.bias", "mask_head.lay2.weight", "mask_head.lay2.bias", "mask_head.gn2.weight", "mask_head.gn2.bias", "mask_head.lay3.weight", "mask_head.lay3.bias", "mask_head.gn3.weight", "mask_head.gn3.bias", "mask_head.lay4.weight", "mask_head.lay4.bias", "mask_head.gn4.weight", "mask_head.gn4.bias", "mask_head.lay5.weight", "mask_head.lay5.bias", "mask_head.gn5.weight", "mask_head.gn5.bias", "mask_head.out_lay.weight", "mask_head.out_lay.bias", "mask_head.adapter1.weight", "mask_head.adapter1.bias", "mask_head.adapter2.weight", "mask_head.adapter2.bias", "mask_head.adapter3.weight", "mask_head.adapter3.bias".

I think this is because the pre-train model does not set masks to true, so I want to set masks to true to retrain my model, and then the problems mentioned at the beginning appear

[timmeinhardt]

Attention maps and mask prediction are two separate things. However, only for our model for mask prediction the attention map plotting is implemented. If you want to generate attention maps for the bbox model, you need to implement attention mask plotting. Not train a model for masks.