swintransformerv2预训练权重似乎加载不进去

[whitewings-hub]

+Model info---------+----------------------+---------------+-----------------+
| Backbone | Neck | Head | Loss |
+-------------------+----------------------+---------------+-----------------+
| SwinTransformerV2 | GlobalAveragePooling | LinearClsHead | LabelSmoothLoss |
+-------------------+----------------------+---------------+-----------------+
Initialize the weights.
Loading swinv2_base_patch4_window8_256.pth
The model and loaded state dict do not match exactly

unexpected key in source state_dict: model

missing keys in source state_dict: backbone.patch_embed.projection.weight, backbone.patch_embed.projection.bias, backbone.patch_embed.norm.weight, backbone.patch_embed.norm.bias, backbone.stages.0.blocks.0.attn.w_msa.logit_scale, backbone.stages.0.blocks.0.attn.w_msa.q_bias, backbone.stages.0.blocks.0.attn.w_msa.v_bias, backbone.stages.0.blocks.0.attn.w_msa.relative_coords_table, backbone.stages.0.blocks.0.attn.w_msa.relative_position_index, backbone.stages.0.blocks.0.attn.w_msa.cpb_mlp.0.weight, backbone.stages.0.blocks.0.attn.w_msa.cpb_mlp.0.bias, backbone.stages.0.blocks.0.attn.w_msa.cpb_mlp.2.weight, backbone.stages.0.blocks.0.attn.w_msa.qkv.weight, backbone.stages.0.blocks.0.attn.w_msa.proj.weight, backbone.stages.0.blocks.0.attn.w_msa.proj.bias, backbone.stages.0.blocks.0.norm1.weight, backbone.stages.0.blocks.0.norm1.bias, backbone.stages.0.blocks.0.ffn.layers.0.0.weight, backbone.stages.0.blocks.0.ffn.layers.0.0.bias, backbone.stages.0.blocks.0.ffn.layers.1.weight, backbone.stages.0.blocks.0.ffn.layers.1.bias, backbone.stages.0.blocks.0.norm2.weight, backbone.stages.0.blocks.0.norm2.bias, backbone.stages.0.blocks.1.attn.w_msa.logit_scale, backbone.stages.0.blocks.1.attn.w_msa.q_bias, backbone.stages.0.blocks.1.attn.w_msa.v_bias, backbone.stages.0.blocks.1.attn.w_msa.relative_coords_table, backbone.stages.0.blocks.1.attn.w_msa.relative_position_index, backbone.stages.0.blocks.1.attn.w_msa.cpb_mlp.0.weight, backbone.stages.0.blocks.1.attn.w_msa.cpb_mlp.0.bias, backbone.stages.0.blocks.1.attn.w_msa.cpb_mlp.2.weight, backbone.stages.0.blocks.1.attn.w_msa.qkv.weight, backbone.stages.0.blocks.1.attn.w_msa.proj.weight, backbone.stages.0.blocks.1.attn.w_msa.proj.bias, backbone.stages.0.blocks.1.norm1.weight, backbone.stages.0.blocks.1.norm1.bias, backbone.stages.0.blocks.1.ffn.layers.0.0.weight, backbone.stages.0.blocks.1.ffn.layers.0.0.bias, backbone.stages.0.blocks.1.ffn.layers.1.weight, backbone.stages.0.blocks.1.ffn.layers.1.bias, backbone.stages.0.blocks.1.norm2.weight, backbone.stages.0.blocks.1.norm2.bias, backbone.stages.1.downsample.reduction.weight, backbone.stages.1.downsample.norm.weight, backbone.stages.1.downsample.norm.bias, backbone.stages.1.blocks.0.attn.w_msa.logit_scale, backbone.stages.1.blocks.0.attn.w_msa.q_bias, backbone.stages.1.blocks.0.attn.w_msa.v_bias, backbone.stages.1.blocks.0.attn.w_msa.relative_coords_table, backbone.stages.1.blocks.0.attn.w_msa.relative_position_index, backbone.stages.1.blocks.0.attn.w_msa.cpb_mlp.0.weight, backbone.stages.1.blocks.0.attn.w_msa.cpb_mlp.0.bias, backbone.stages.1.blocks.0.attn.w_msa.cpb_mlp.2.weight, backbone.stages.1.blocks.0.attn.w_msa.qkv.weight, backbone.stages.1.blocks.0.attn.w_msa.proj.weight, backbone.stages.1.blocks.0.attn.w_msa.proj.bias, backbone.stages.1.blocks.0.norm1.weight, backbone.stages.1.blocks.0.norm1.bias, backbone.stages.1.blocks.0.ffn.layers.0.0.weight, backbone.stages.1.blocks.0.ffn.layers.0.0.bias, backbone.stages.1.blocks.0.ffn.layers.1.weight, backbone.stages.1.blocks.0.ffn.layers.1.bias, backbone.stages.1.blocks.0.norm2.weight, backbone.stages.1.blocks.0.norm2.bias, backbone.stages.1.blocks.1.attn.w_msa.logit_scale, backbone.stages.1.blocks.1.attn.w_msa.q_bias, backbone.stages.1.blocks.1.attn.w_msa.v_bias, backbone.stages.1.blocks.1.attn.w_msa.relative_coords_table, backbone.stages.1.blocks.1.attn.w_msa.relative_position_index, backbone.stages.1.blocks.1.attn.w_msa.cpb_mlp.0.weight, backbone.stages.1.blocks.1.attn.w_msa.cpb_mlp.0.bias, backbone.stages.1.blocks.1.attn.w_msa.cpb_mlp.2.weight, backbone.stages.1.blocks.1.attn.w_msa.qkv.weight, backbone.stages.1.blocks.1.attn.w_msa.proj.weight, backbone.stages.1.blocks.1.attn.w_msa.proj.bias, backbone.stages.1.blocks.1.norm1.weight, backbone.stages.1.blocks.1.norm1.bias, backbone.stages.1.blocks.1.ffn.layers.0.0.weight, backbone.stages.1.blocks.1.ffn.layers.0.0.bias, backbone.stages.1.blocks.1.ffn.layers.1.weight, backbone.stages.1.blocks.1.ffn.layers.1.bias, backbone.stages.1.blocks.1.norm2.weight, backbone.stages.1.blocks.1.norm2.bias, backbone.stages.2.downsample.reduction.weight, backbone.stages.2.downsample.norm.weight, backbone.stages.2.downsample.norm.bias, backbone.stages.2.blocks.0.attn.w_msa.logit_scale, backbone.stages.2.blocks.0.attn.w_msa.q_bias, backbone.stages.2.blocks.0.attn.w_msa.v_bias, backbone.stages.2.blocks.0.attn.w_msa.relative_coords_table, backbone.stages.2.blocks.0.attn.w_msa.relative_position_index, backbone.stages.2.blocks.0.attn.w_msa.cpb_mlp.0.weight, backbone.stages.2.blocks.0.attn.w_msa.cpb_mlp.0.bias, backbone.stages.2.blocks.0.attn.w_msa.cpb_mlp.2.weight, backbone.stages.2.blocks.0.attn.w_msa.qkv.weight, backbone.stages.2.blocks.0.attn.w_msa.proj.weight, backbone.stages.2.blocks.0.attn.w_msa.proj.bias, backbone.stages.2.blocks.0.norm1.weight, backbone.stages.2.blocks.0.norm1.bias, backbone.stages.2.blocks.0.ffn.layers.0.0.weight, backbone.stages.2.blocks.0.ffn.layers.0.0.bias, backbone.stages.2.blocks.0.ffn.layers.1.weight, backbone.stages.2.blocks.0.ffn.layers.1.bias, backbone.stages.2.blocks.0.norm2.weight, backbone.stages.2.blocks.0.norm2.bias, backbone.stages.2.blocks.1.attn.w_msa.logit_scale, backbone.stages.2.blocks.1.attn.w_msa.q_bias, backbone.stages.2.blocks.1.attn.w_msa.v_bias, backbone.stages.2.blocks.1.attn.w_msa.relative_coords_table, backbone.stages.2.blocks.1.attn.w_msa.relative_position_index, backbone.stages.2.blocks.1.attn.w_msa.cpb_mlp.0.weight, backbone.stages.2.blocks.1.attn.w_msa.cpb_mlp.0.bias, backbone.stages.2.blocks.1.attn.w_msa.cpb_mlp.2.weight, backbone.stages.2.blocks.1.attn.w_msa.qkv.weight, backbone.stages.2.blocks.1.attn.w_msa.proj.weight, backbone.stages.2.blocks.1.attn.w_msa.proj.bias, backbone.stages.2.blocks.1.norm1.weight, backbone.stages.2.blocks.1.norm1.bias, backbone.stages.2.blocks.1.ffn.layers.0.0.weight, backbone.stages.2.blocks.1.ffn.layers.0.0.bias, backbone.stages.2.blocks.1.ffn.layers.1.weight, backbone.stages.2.blocks.1.ffn.layers.1.bias, backbone.stages.2.blocks.1.norm2.weight, backbone.stages.2.blocks.1.norm2.bias, backbone.stages.2.blocks.2.attn.w_msa.logit_scale, backbone.stages.2.blocks.2.attn.w_msa.q_bias, backbone.stages.2.blocks.2.attn.w_msa.v_bias, backbone.stages.2.blocks.2.attn.w_msa.relative_coords_table, backbone.stages.2.blocks.2.attn.w_msa.relative_position_index, backbone.stages.2.blocks.2.attn.w_msa.cpb_mlp.0.weight, backbone.stages.2.blocks.2.attn.w_msa.cpb_mlp.0.bias, backbone.stages.2.blocks.2.attn.w_msa.cpb_mlp.2.weight, backbone.stages.2.blocks.2.attn.w_msa.qkv.weight, backbone.stages.2.blocks.2.attn.w_msa.proj.weight, backbone.stages.2.blocks.2.attn.w_msa.proj.bias, backbone.stages.2.blocks.2.norm1.weight, backbone.stages.2.blocks.2.norm1.bias, backbone.stages.2.blocks.2.ffn.layers.0.0.weight, backbone.stages.2.blocks.2.ffn.layers.0.0.bias, backbone.stages.2.blocks.2.ffn.layers.1.weight, backbone.stages.2.blocks.2.ffn.layers.1.bias, backbone.stages.2.blocks.2.norm2.weight, backbone.stages.2.blocks.2.norm2.bias, backbone.stages.2.blocks.3.attn.w_msa.logit_scale, backbone.stages.2.blocks.3.attn.w_msa.q_bias, backbone.stages.2.blocks.3.attn.w_msa.v_bias, backbone.stages.2.blocks.3.attn.w_msa.relative_coords_table, backbone.stages.2.blocks.3.attn.w_msa.relative_position_index, backbone.stages.2.blocks.3.attn.w_msa.cpb_mlp.0.weight, backbone.stages.2.blocks.3.attn.w_msa.cpb_mlp.0.bias, backbone.stages.2.blocks.3.attn.w_msa.cpb_mlp.2.weight, backbone.stages.2.blocks.3.attn.w_msa.qkv.weight, backbone.stages.2.blocks.3.attn.w_msa.proj.weight, backbone.stages.2.blocks.3.attn.w_msa.proj.bias, backbone.stages.2.blocks.3.norm1.weight, backbone.stages.2.blocks.3.norm1.bias, backbone.stages.2.blocks.3.ffn.layers.0.0.weight, backbone.stages.2.blocks.3.ffn.layers.0.0.bias, backbone.stages.2.blocks.3.ffn.layers.1.weight, backbone.stages.2.blocks.3.ffn.layers.1.bias, backbone.stages.2.blocks.3.norm2.weight, backbone.stages.2.blocks.3.norm2.bias, backbone.stages.2.blocks.4.attn.w_msa.logit_scale, backbone.stages.2.blocks.4.attn.w_msa.q_bias, backbone.stages.2.blocks.4.attn.w_msa.v_bias, backbone.stages.2.blocks.4.attn.w_msa.relative_coords_table, backbone.stages.2.blocks.4.attn.w_msa.relative_position_index, backbone.stages.2.blocks.4.attn.w_msa.cpb_mlp.0.weight, backbone.stages.2.blocks.4.attn.w_msa.cpb_mlp.0.bias, backbone.stages.2.blocks.4.attn.w_msa.cpb_mlp.2.weight, backbone.stages.2.blocks.4.attn.w_msa.qkv.weight, backbone.stages.2.blocks.4.attn.w_msa.proj.weight, backbone.stages.2.blocks.4.attn.w_msa.proj.bias, backbone.stages.2.blocks.4.norm1.weight, backbone.stages.2.blocks.4.norm1.bias, backbone.stages.2.blocks.4.ffn.layers.0.0.weight, backbone.stages.2.blocks.4.ffn.layers.0.0.bias, backbone.stages.2.blocks.4.ffn.layers.1.weight, backbone.stages.2.blocks.4.ffn.layers.1.bias, backbone.stages.2.blocks.4.norm2.weight, backbone.stages.2.blocks.4.norm2.bias, backbone.stages.2.blocks.5.attn.w_msa.logit_scale, backbone.stages.2.blocks.5.attn.w_msa.q_bias, backbone.stages.2.blocks.5.attn.w_msa.v_bias, backbone.stages.2.blocks.5.attn.w_msa.relative_coords_table, backbone.stages.2.blocks.5.attn.w_msa.relative_position_index, backbone.stages.2.blocks.5.attn.w_msa.cpb_mlp.0.weight, backbone.stages.2.blocks.5.attn.w_msa.cpb_mlp.0.bias, backbone.stages.2.blocks.5.attn.w_msa.cpb_mlp.2.weight, backbone.stages.2.blocks.5.attn.w_msa.qkv.weight, backbone.stages.2.blocks.5.attn.w_msa.proj.weight, backbone.stages.2.blocks.5.attn.w_msa.proj.bias, backbone.stages.2.blocks.5.norm1.weight, backbone.stages.2.blocks.5.norm1.bias, backbone.stages.2.blocks.5.ffn.layers.0.0.weight, backbone.stages.2.blocks.5.ffn.layers.0.0.bias, backbone.stages.2.blocks.5.ffn.layers.1.weight, backbone.stages.2.blocks.5.ffn.layers.1.bias, backbone.stages.2.blocks.5.norm2.weight, backbone.stages.2.blocks.5.norm2.bias, backbone.stages.2.blocks.6.attn.w_msa.logit_scale, backbone.stages.2.blocks.6.attn.w_msa.q_bias, backbone.stages.2.blocks.6.attn.w_msa.v_bias, backbone.stages.2.blocks.6.attn.w_msa.relative_coords_table, backbone.stages.2.blocks.6.attn.w_msa.relative_position_index, backbone.stages.2.blocks.6.attn.w_msa.cpb_mlp.0.weight, backbone.stages.2.blocks.6.attn.w_msa.cpb_mlp.0.bias, backbone.stages.2.blocks.6.attn.w_msa.cpb_mlp.2.weight, backbone.stages.2.blocks.6.attn.w_msa.qkv.weight, backbone.stages.2.blocks.6.attn.w_msa.proj.weight, backbone.stages.2.blocks.6.attn.w_msa.proj.bias, backbone.stages.2.blocks.6.norm1.weight, backbone.stages.2.blocks.6.norm1.bias, backbone.stages.2.blocks.6.ffn.layers.0.0.weight, backbone.stages.2.blocks.6.ffn.layers.0.0.bias, backbone.stages.2.blocks.6.ffn.layers.1.weight, backbone.stages.2.blocks.6.ffn.layers.1.bias, backbone.stages.2.blocks.6.norm2.weight, backbone.stages.2.blocks.6.norm2.bias, backbone.stages.2.blocks.7.attn.w_msa.logit_scale, backbone.stages.2.blocks.7.attn.w_msa.q_bias, backbone.stages.2.blocks.7.attn.w_msa.v_bias, backbone.stages.2.blocks.7.attn.w_msa.relative_coords_table, backbone.stages.2.blocks.7.attn.w_msa.relative_position_index, backbone.stages.2.blocks.7.attn.w_msa.cpb_mlp.0.weight, backbone.stages.2.blocks.7.attn.w_msa.cpb_mlp.0.bias, backbone.stages.2.blocks.7.attn.w_msa.cpb_mlp.2.weight, backbone.stages.2.blocks.7.attn.w_msa.qkv.weight, backbone.stages.2.blocks.7.attn.w_msa.proj.weight, backbone.stages.2.blocks.7.attn.w_msa.proj.bias, backbone.stages.2.blocks.7.norm1.weight, backbone.stages.2.blocks.7.norm1.bias, backbone.stages.2.blocks.7.ffn.layers.0.0.weight, backbone.stages.2.blocks.7.ffn.layers.0.0.bias, backbone.stages.2.blocks.7.ffn.layers.1.weight, backbone.stages.2.blocks.7.ffn.layers.1.bias, backbone.stages.2.blocks.7.norm2.weight, backbone.stages.2.blocks.7.norm2.bias, backbone.stages.2.blocks.8.attn.w_msa.logit_scale, backbone.stages.2.blocks.8.attn.w_msa.q_bias, backbone.stages.2.blocks.8.attn.w_msa.v_bias, backbone.stages.2.blocks.8.attn.w_msa.relative_coords_table, backbone.stages.2.blocks.8.attn.w_msa.relative_position_index, backbone.stages.2.blocks.8.attn.w_msa.cpb_mlp.0.weight, backbone.stages.2.blocks.8.attn.w_msa.cpb_mlp.0.bias, backbone.stages.2.blocks.8.attn.w_msa.cpb_mlp.2.weight, backbone.stages.2.blocks.8.attn.w_msa.qkv.weight, backbone.stages.2.blocks.8.attn.w_msa.proj.weight, backbone.stages.2.blocks.8.attn.w_msa.proj.bias, backbone.stages.2.blocks.8.norm1.weight, backbone.stages.2.blocks.8.norm1.bias, backbone.stages.2.blocks.8.ffn.layers.0.0.weight, backbone.stages.2.blocks.8.ffn.layers.0.0.bias, backbone.stages.2.blocks.8.ffn.layers.1.weight, backbone.stages.2.blocks.8.ffn.layers.1.bias, backbone.stages.2.blocks.8.norm2.weight, backbone.stages.2.blocks.8.norm2.bias, backbone.stages.2.blocks.9.attn.w_msa.logit_scale, backbone.stages.2.blocks.9.attn.w_msa.q_bias, backbone.stages.2.blocks.9.attn.w_msa.v_bias, backbone.stages.2.blocks.9.attn.w_msa.relative_coords_table, backbone.stages.2.blocks.9.attn.w_msa.relative_position_index, backbone.stages.2.blocks.9.attn.w_msa.cpb_mlp.0.weight, backbone.stages.2.blocks.9.attn.w_msa.cpb_mlp.0.bias, backbone.stages.2.blocks.9.attn.w_msa.cpb_mlp.2.weight, backbone.stages.2.blocks.9.attn.w_msa.qkv.weight, backbone.stages.2.blocks.9.attn.w_msa.proj.weight, backbone.stages.2.blocks.9.attn.w_msa.proj.bias, backbone.stages.2.blocks.9.norm1.weight, backbone.stages.2.blocks.9.norm1.bias, backbone.stages.2.blocks.9.ffn.layers.0.0.weight, backbone.stages.2.blocks.9.ffn.layers.0.0.bias, backbone.stages.2.blocks.9.ffn.layers.1.weight, backbone.stages.2.blocks.9.ffn.layers.1.bias, backbone.stages.2.blocks.9.norm2.weight, backbone.stages.2.blocks.9.norm2.bias, backbone.stages.2.blocks.10.attn.w_msa.logit_scale, backbone.stages.2.blocks.10.attn.w_msa.q_bias, backbone.stages.2.blocks.10.attn.w_msa.v_bias, backbone.stages.2.blocks.10.attn.w_msa.relative_coords_table, backbone.stages.2.blocks.10.attn.w_msa.relative_position_index, backbone.stages.2.blocks.10.attn.w_msa.cpb_mlp.0.weight, backbone.stages.2.blocks.10.attn.w_msa.cpb_mlp.0.bias, backbone.stages.2.blocks.10.attn.w_msa.cpb_mlp.2.weight, backbone.stages.2.blocks.10.attn.w_msa.qkv.weight, backbone.stages.2.blocks.10.attn.w_msa.proj.weight, backbone.stages.2.blocks.10.attn.w_msa.proj.bias, backbone.stages.2.blocks.10.norm1.weight, backbone.stages.2.blocks.10.norm1.bias, backbone.stages.2.blocks.10.ffn.layers.0.0.weight, backbone.stages.2.blocks.10.ffn.layers.0.0.bias, backbone.stages.2.blocks.10.ffn.layers.1.weight, backbone.stages.2.blocks.10.ffn.layers.1.bias, backbone.stages.2.blocks.10.norm2.weight, backbone.stages.2.blocks.10.norm2.bias, backbone.stages.2.blocks.11.attn.w_msa.logit_scale, backbone.stages.2.blocks.11.attn.w_msa.q_bias, backbone.stages.2.blocks.11.attn.w_msa.v_bias, backbone.stages.2.blocks.11.attn.w_msa.relative_coords_table, backbone.stages.2.blocks.11.attn.w_msa.relative_position_index, backbone.stages.2.blocks.11.attn.w_msa.cpb_mlp.0.weight, backbone.stages.2.blocks.11.attn.w_msa.cpb_mlp.0.bias, backbone.stages.2.blocks.11.attn.w_msa.cpb_mlp.2.weight, backbone.stages.2.blocks.11.attn.w_msa.qkv.weight, backbone.stages.2.blocks.11.attn.w_msa.proj.weight, backbone.stages.2.blocks.11.attn.w_msa.proj.bias, backbone.stages.2.blocks.11.norm1.weight, backbone.stages.2.blocks.11.norm1.bias, backbone.stages.2.blocks.11.ffn.layers.0.0.weight, backbone.stages.2.blocks.11.ffn.layers.0.0.bias, backbone.stages.2.blocks.11.ffn.layers.1.weight, backbone.stages.2.blocks.11.ffn.layers.1.bias, backbone.stages.2.blocks.11.norm2.weight, backbone.stages.2.blocks.11.norm2.bias, backbone.stages.2.blocks.12.attn.w_msa.logit_scale, backbone.stages.2.blocks.12.attn.w_msa.q_bias, backbone.stages.2.blocks.12.attn.w_msa.v_bias, backbone.stages.2.blocks.12.attn.w_msa.relative_coords_table, backbone.stages.2.blocks.12.attn.w_msa.relative_position_index, backbone.stages.2.blocks.12.attn.w_msa.cpb_mlp.0.weight, backbone.stages.2.blocks.12.attn.w_msa.cpb_mlp.0.bias, backbone.stages.2.blocks.12.attn.w_msa.cpb_mlp.2.weight, backbone.stages.2.blocks.12.attn.w_msa.qkv.weight, backbone.stages.2.blocks.12.attn.w_msa.proj.weight, backbone.stages.2.blocks.12.attn.w_msa.proj.bias, backbone.stages.2.blocks.12.norm1.weight, backbone.stages.2.blocks.12.norm1.bias, backbone.stages.2.blocks.12.ffn.layers.0.0.weight, backbone.stages.2.blocks.12.ffn.layers.0.0.bias, backbone.stages.2.blocks.12.ffn.layers.1.weight, backbone.stages.2.blocks.12.ffn.layers.1.bias, backbone.stages.2.blocks.12.norm2.weight, backbone.stages.2.blocks.12.norm2.bias, backbone.stages.2.blocks.13.attn.w_msa.logit_scale, backbone.stages.2.blocks.13.attn.w_msa.q_bias, backbone.stages.2.blocks.13.attn.w_msa.v_bias, backbone.stages.2.blocks.13.attn.w_msa.relative_coords_table, backbone.stages.2.blocks.13.attn.w_msa.relative_position_index, backbone.stages.2.blocks.13.attn.w_msa.cpb_mlp.0.weight, backbone.stages.2.blocks.13.attn.w_msa.cpb_mlp.0.bias, backbone.stages.2.blocks.13.attn.w_msa.cpb_mlp.2.weight, backbone.stages.2.blocks.13.attn.w_msa.qkv.weight, backbone.stages.2.blocks.13.attn.w_msa.proj.weight, backbone.stages.2.blocks.13.attn.w_msa.proj.bias, backbone.stages.2.blocks.13.norm1.weight, backbone.stages.2.blocks.13.norm1.bias, backbone.stages.2.blocks.13.ffn.layers.0.0.weight, backbone.stages.2.blocks.13.ffn.layers.0.0.bias, backbone.stages.2.blocks.13.ffn.layers.1.weight, backbone.stages.2.blocks.13.ffn.layers.1.bias, backbone.stages.2.blocks.13.norm2.weight, backbone.stages.2.blocks.13.norm2.bias, backbone.stages.2.blocks.14.attn.w_msa.logit_scale, backbone.stages.2.blocks.14.attn.w_msa.q_bias, backbone.stages.2.blocks.14.attn.w_msa.v_bias, backbone.stages.2.blocks.14.attn.w_msa.relative_coords_table, backbone.stages.2.blocks.14.attn.w_msa.relative_position_index, backbone.stages.2.blocks.14.attn.w_msa.cpb_mlp.0.weight, backbone.stages.2.blocks.14.attn.w_msa.cpb_mlp.0.bias, backbone.stages.2.blocks.14.attn.w_msa.cpb_mlp.2.weight, backbone.stages.2.blocks.14.attn.w_msa.qkv.weight, backbone.stages.2.blocks.14.attn.w_msa.proj.weight, backbone.stages.2.blocks.14.attn.w_msa.proj.bias, backbone.stages.2.blocks.14.norm1.weight, backbone.stages.2.blocks.14.norm1.bias, backbone.stages.2.blocks.14.ffn.layers.0.0.weight, backbone.stages.2.blocks.14.ffn.layers.0.0.bias, backbone.stages.2.blocks.14.ffn.layers.1.weight, backbone.stages.2.blocks.14.ffn.layers.1.bias, backbone.stages.2.blocks.14.norm2.weight, backbone.stages.2.blocks.14.norm2.bias, backbone.stages.2.blocks.15.attn.w_msa.logit_scale, backbone.stages.2.blocks.15.attn.w_msa.q_bias, backbone.stages.2.blocks.15.attn.w_msa.v_bias, backbone.stages.2.blocks.15.attn.w_msa.relative_coords_table, backbone.stages.2.blocks.15.attn.w_msa.relative_position_index, backbone.stages.2.blocks.15.attn.w_msa.cpb_mlp.0.weight, backbone.stages.2.blocks.15.attn.w_msa.cpb_mlp.0.bias, backbone.stages.2.blocks.15.attn.w_msa.cpb_mlp.2.weight, backbone.stages.2.blocks.15.attn.w_msa.qkv.weight, backbone.stages.2.blocks.15.attn.w_msa.proj.weight, backbone.stages.2.blocks.15.attn.w_msa.proj.bias, backbone.stages.2.blocks.15.norm1.weight, backbone.stages.2.blocks.15.norm1.bias, backbone.stages.2.blocks.15.ffn.layers.0.0.weight, backbone.stages.2.blocks.15.ffn.layers.0.0.bias, backbone.stages.2.blocks.15.ffn.layers.1.weight, backbone.stages.2.blocks.15.ffn.layers.1.bias, backbone.stages.2.blocks.15.norm2.weight, backbone.stages.2.blocks.15.norm2.bias, backbone.stages.2.blocks.16.attn.w_msa.logit_scale, backbone.stages.2.blocks.16.attn.w_msa.q_bias, backbone.stages.2.blocks.16.attn.w_msa.v_bias, backbone.stages.2.blocks.16.attn.w_msa.relative_coords_table, backbone.stages.2.blocks.16.attn.w_msa.relative_position_index, backbone.stages.2.blocks.16.attn.w_msa.cpb_mlp.0.weight, backbone.stages.2.blocks.16.attn.w_msa.cpb_mlp.0.bias, backbone.stages.2.blocks.16.attn.w_msa.cpb_mlp.2.weight, backbone.stages.2.blocks.16.attn.w_msa.qkv.weight, backbone.stages.2.blocks.16.attn.w_msa.proj.weight, backbone.stages.2.blocks.16.attn.w_msa.proj.bias, backbone.stages.2.blocks.16.norm1.weight, backbone.stages.2.blocks.16.norm1.bias, backbone.stages.2.blocks.16.ffn.layers.0.0.weight, backbone.stages.2.blocks.16.ffn.layers.0.0.bias, backbone.stages.2.blocks.16.ffn.layers.1.weight, backbone.stages.2.blocks.16.ffn.layers.1.bias, backbone.stages.2.blocks.16.norm2.weight, backbone.stages.2.blocks.16.norm2.bias, backbone.stages.2.blocks.17.attn.w_msa.logit_scale, backbone.stages.2.blocks.17.attn.w_msa.q_bias, backbone.stages.2.blocks.17.attn.w_msa.v_bias, backbone.stages.2.blocks.17.attn.w_msa.relative_coords_table, backbone.stages.2.blocks.17.attn.w_msa.relative_position_index, backbone.stages.2.blocks.17.attn.w_msa.cpb_mlp.0.weight, backbone.stages.2.blocks.17.attn.w_msa.cpb_mlp.0.bias, backbone.stages.2.blocks.17.attn.w_msa.cpb_mlp.2.weight, backbone.stages.2.blocks.17.attn.w_msa.qkv.weight, backbone.stages.2.blocks.17.attn.w_msa.proj.weight, backbone.stages.2.blocks.17.attn.w_msa.proj.bias, backbone.stages.2.blocks.17.norm1.weight, backbone.stages.2.blocks.17.norm1.bias, backbone.stages.2.blocks.17.ffn.layers.0.0.weight, backbone.stages.2.blocks.17.ffn.layers.0.0.bias, backbone.stages.2.blocks.17.ffn.layers.1.weight, backbone.stages.2.blocks.17.ffn.layers.1.bias, backbone.stages.2.blocks.17.norm2.weight, backbone.stages.2.blocks.17.norm2.bias, backbone.stages.3.downsample.reduction.weight, backbone.stages.3.downsample.norm.weight, backbone.stages.3.downsample.norm.bias, backbone.stages.3.blocks.0.attn.w_msa.logit_scale, backbone.stages.3.blocks.0.attn.w_msa.q_bias, backbone.stages.3.blocks.0.attn.w_msa.v_bias, backbone.stages.3.blocks.0.attn.w_msa.relative_coords_table, backbone.stages.3.blocks.0.attn.w_msa.relative_position_index, backbone.stages.3.blocks.0.attn.w_msa.cpb_mlp.0.weight, backbone.stages.3.blocks.0.attn.w_msa.cpb_mlp.0.bias, backbone.stages.3.blocks.0.attn.w_msa.cpb_mlp.2.weight, backbone.stages.3.blocks.0.attn.w_msa.qkv.weight, backbone.stages.3.blocks.0.attn.w_msa.proj.weight, backbone.stages.3.blocks.0.attn.w_msa.proj.bias, backbone.stages.3.blocks.0.norm1.weight, backbone.stages.3.blocks.0.norm1.bias, backbone.stages.3.blocks.0.ffn.layers.0.0.weight, backbone.stages.3.blocks.0.ffn.layers.0.0.bias, backbone.stages.3.blocks.0.ffn.layers.1.weight, backbone.stages.3.blocks.0.ffn.layers.1.bias, backbone.stages.3.blocks.0.norm2.weight, backbone.stages.3.blocks.0.norm2.bias, backbone.stages.3.blocks.1.attn.w_msa.logit_scale, backbone.stages.3.blocks.1.attn.w_msa.q_bias, backbone.stages.3.blocks.1.attn.w_msa.v_bias, backbone.stages.3.blocks.1.attn.w_msa.relative_coords_table, backbone.stages.3.blocks.1.attn.w_msa.relative_position_index, backbone.stages.3.blocks.1.attn.w_msa.cpb_mlp.0.weight, backbone.stages.3.blocks.1.attn.w_msa.cpb_mlp.0.bias, backbone.stages.3.blocks.1.attn.w_msa.cpb_mlp.2.weight, backbone.stages.3.blocks.1.attn.w_msa.qkv.weight, backbone.stages.3.blocks.1.attn.w_msa.proj.weight, backbone.stages.3.blocks.1.attn.w_msa.proj.bias, backbone.stages.3.blocks.1.norm1.weight, backbone.stages.3.blocks.1.norm1.bias, backbone.stages.3.blocks.1.ffn.layers.0.0.weight, backbone.stages.3.blocks.1.ffn.layers.0.0.bias, backbone.stages.3.blocks.1.ffn.layers.1.weight, backbone.stages.3.blocks.1.ffn.layers.1.bias, backbone.stages.3.blocks.1.norm2.weight, backbone.stages.3.blocks.1.norm2.bias, backbone.norm3.weight, backbone.norm3.bias, head.fc.weight, head.fc.bias
/home/user/miniconda3/envs/wmh/lib/python3.8/site-packages/torch/functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:2157.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]

[whitewings-hub]

我是笨蛋，用up给的预训练权重就好了，一开始没看到。o(╥﹏╥)o