DETR checkpoint mismatch for JHMDB

[wenzhengzeng]

Thanks for your great work. I notice that the DETR model used in JHMDB is different from that in AVA. For example, as mentioned in models/tuber_ava.py line 43:

if self.dataset_mode != 'ava':
self.avg_s = nn.AdaptiveAvgPool3d((1, 1, 1))
self.query_embed = nn.Embedding(num_queries * temporal_length, hidden_dim)
else:
self.query_embed = nn.Embedding(num_queries, hidden_dim)

Currently, it seems only detr.pth for AVA dataset is provided. As a result, when running the code for JHMDB, there will be an error:
size mismatch for module.query_embed.weight: copying a param with shape torch.Size([10, 256]) from checkpoint, the shape in current model is torch.Size([320, 256]).

Can you provide the right pre-trained DETR checkpoint (i.e., detr.pth) for JHMDB? Thanks.

[coocoo90]

Thanks for being interested with TubeR. We have fixed the issue, please see PR #6.

[wenzhengzeng]

Thanks for your help. I wonder where the pre-trained DETR comes from, is it directly pre-trained on COCO for object detection?

[coocoo90]

We load the weights from COCO and pre-trained it on AVA for detecting person-only. But I think it won't be much difference with COCO only weights. 

…

------------------ 原始邮件 ------------------ 发件人: "Alex ***@***.***>; 发送时间: 2022年9月3日(星期六) 晚上9:53 收件人: ***@***.***>; 抄送: "YANYI ***@***.***>; ***@***.***>; 主题: Re: [amazon-research/tubelet-transformer] DETR checkpoint mismatch for JHMDB (Issue #4) Thanks for your help. I wonder where the pre-trained DETR comes from, is it directly pre-trained on COCO for object detection? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[wenzhengzeng]

Thanks for your reply.