关于“fusion_arbitary_size_TransMEF_gray.py”

[keviner1]

你好，非常感谢关于上一个issue的解答！再次打扰一下，我在测试中发现以“fusion_arbitary_size_TransMEF_gray.py”融合后的图片和以“fusion_gray_TransMEF.py”融合后的图片不同。其中滑窗融合方法得到图片质量较差。两个方法采用的数据，模型是一致的。

[keviner1]

对拼接后的全局图像进行一次MaxMinNormalization可以解决该问题，如下：

def fuse(img1, img2):
'''
block fusion
'''
block_num = img1.shape[0]
final_fusion = np.zeros_like(img1)
for i in range(block_num):
img1_inblock = _tensor(img1[i, :, :]).unsqueeze(0).to(device)
img2_inblock = _tensor(img2[i, :, :]).unsqueeze(0).to(device)
img_fusion = fusion(x1=img1_inblock, x2=img2_inblock, model=model)
img_fusion = _pil_gray(img_fusion)
img_fusion = np.asarray(img_fusion)
final_fusion[i,:,:] = img_fusion

**# Perform Global MaxMinNormalization**
final_fusion = (final_fusion - np.min(final_fusion)) / (np.max(final_fusion) - np.min(final_fusion))
final_fusion = np.clip(final_fusion*255, 0, 255)
return final_fusion

[miccaiif]

Thanks for solving this issue!

[yusibo2751358]

你好 我下载了谷歌网盘里的best_model.pth，然后运行fusion_gray_TransMEF.py时，说模型有问题。raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for TransNet:
Missing key(s) in state_dict: "encoder.inc.double_conv.0.weight", "encoder.inc.double_conv.0.bias", "encoder.inc.double_conv.2.weight", "encoder.inc.double_conv.2.bias", "encoder.layer1.double_conv.0.weight", "encoder.layer1.double_conv.0.bias", "encoder.layer1.double_conv.2.weight", "encoder.layer1.double_conv.2.bias", "encoder.layer2.double_conv.0.weight", "encoder.layer2.double_conv.0.bias", "encoder.layer2.double_conv.2.weight", "encoder.layer2.double_conv.2.bias", "encoder.transformer.cls_token", "encoder.transformer.to_patch_embedding.1.weight", "encoder.transformer.to_patch_embedding.1.bias", "encoder.transformer.transformer.layers.0.0.norm.weight", "encoder.transformer.transformer.layers.0.0.norm.bias", "encoder.transformer.transformer.layers.0.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.0.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.0.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.0.1.norm.weight", "encoder.transformer.transformer.layers.0.1.norm.bias", "encoder.transformer.transformer.layers.0.1.fn.net.0.weight", "encoder.transformer.transformer.layers.0.1.fn.net.0.bias", "encoder.transformer.transformer.layers.0.1.fn.net.3.weight", "encoder.transformer.transformer.layers.0.1.fn.net.3.bias", "encoder.transformer.transformer.layers.1.0.norm.weight", "encoder.transformer.transformer.layers.1.0.norm.bias", "encoder.transformer.transformer.layers.1.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.1.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.1.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.1.1.norm.weight", "encoder.transformer.transformer.layers.1.1.norm.bias", "encoder.transformer.transformer.layers.1.1.fn.net.0.weight", "encoder.transformer.transformer.layers.1.1.fn.net.0.bias", "encoder.transformer.transformer.layers.1.1.fn.net.3.weight", "encoder.transformer.transformer.layers.1.1.fn.net.3.bias", "encoder.transformer.transformer.layers.2.0.norm.weight", "encoder.transformer.transformer.layers.2.0.norm.bias", "encoder.transformer.transformer.layers.2.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.2.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.2.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.2.1.norm.weight", "encoder.transformer.transformer.layers.2.1.norm.bias", "encoder.transformer.transformer.layers.2.1.fn.net.0.weight", "encoder.transformer.transformer.layers.2.1.fn.net.0.bias", "encoder.transformer.transformer.layers.2.1.fn.net.3.weight", "encoder.transformer.transformer.layers.2.1.fn.net.3.bias", "encoder.transformer.transformer.layers.3.0.norm.weight", "encoder.transformer.transformer.layers.3.0.norm.bias", "encoder.transformer.transformer.layers.3.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.3.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.3.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.3.1.norm.weight", "encoder.transformer.transformer.layers.3.1.norm.bias", "encoder.transformer.transformer.layers.3.1.fn.net.0.weight", "encoder.transformer.transformer.layers.3.1.fn.net.0.bias", "encoder.transformer.transformer.layers.3.1.fn.net.3.weight", "encoder.transformer.transformer.layers.3.1.fn.net.3.bias", "encoder.transformer.transformer.layers.4.0.norm.weight", "encoder.transformer.transformer.layers.4.0.norm.bias", "encoder.transformer.transformer.layers.4.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.4.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.4.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.4.1.norm.weight", "encoder.transformer.transformer.layers.4.1.norm.bias", "encoder.transformer.transformer.layers.4.1.fn.net.0.weight", "encoder.transformer.transformer.layers.4.1.fn.net.0.bias", "encoder.transformer.transformer.layers.4.1.fn.net.3.weight", "encoder.transformer.transformer.layers.4.1.fn.net.3.bias", "encoder.transformer.transformer.layers.5.0.norm.weight", "encoder.transformer.transformer.layers.5.0.norm.bias", "encoder.transformer.transformer.layers.5.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.5.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.5.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.5.1.norm.weight", "encoder.transformer.transformer.layers.5.1.norm.bias", "encoder.transformer.transformer.layers.5.1.fn.net.0.weight", "encoder.transformer.transformer.layers.5.1.fn.net.0.bias", "encoder.transformer.transformer.layers.5.1.fn.net.3.weight", "encoder.transformer.transformer.layers.5.1.fn.net.3.bias", "encoder.transformer.transformer.layers.6.0.norm.weight", "encoder.transformer.transformer.layers.6.0.norm.bias", "encoder.transformer.transformer.layers.6.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.6.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.6.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.6.1.norm.weight", "encoder.transformer.transformer.layers.6.1.norm.bias", "encoder.transformer.transformer.layers.6.1.fn.net.0.weight", "encoder.transformer.transformer.layers.6.1.fn.net.0.bias", "encoder.transformer.transformer.layers.6.1.fn.net.3.weight", "encoder.transformer.transformer.layers.6.1.fn.net.3.bias", "encoder.transformer.transformer.layers.7.0.norm.weight", "encoder.transformer.transformer.layers.7.0.norm.bias", "encoder.transformer.transformer.layers.7.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.7.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.7.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.7.1.norm.weight", "encoder.transformer.transformer.layers.7.1.norm.bias", "encoder.transformer.transformer.layers.7.1.fn.net.0.weight", "encoder.transformer.transformer.layers.7.1.fn.net.0.bias", "encoder.transformer.transformer.layers.7.1.fn.net.3.weight", "encoder.transformer.transformer.layers.7.1.fn.net.3.bias", "encoder.transformer.transformer.layers.8.0.norm.weight", "encoder.transformer.transformer.layers.8.0.norm.bias", "encoder.transformer.transformer.layers.8.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.8.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.8.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.8.1.norm.weight", "encoder.transformer.transformer.layers.8.1.norm.bias", "encoder.transformer.transformer.layers.8.1.fn.net.0.weight", "encoder.transformer.transformer.layers.8.1.fn.net.0.bias", "encoder.transformer.transformer.layers.8.1.fn.net.3.weight", "encoder.transformer.transformer.layers.8.1.fn.net.3.bias", "encoder.transformer.transformer.layers.9.0.norm.weight", "encoder.transformer.transformer.layers.9.0.norm.bias", "encoder.transformer.transformer.layers.9.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.9.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.9.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.9.1.norm.weight", "encoder.transformer.transformer.layers.9.1.norm.bias", "encoder.transformer.transformer.layers.9.1.fn.net.0.weight", "encoder.transformer.transformer.layers.9.1.fn.net.0.bias", "encoder.transformer.transformer.layers.9.1.fn.net.3.weight", "encoder.transformer.transformer.layers.9.1.fn.net.3.bias", "encoder.transformer.transformer.layers.10.0.norm.weight", "encoder.transformer.transformer.layers.10.0.norm.bias", "encoder.transformer.transformer.layers.10.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.10.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.10.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.10.1.norm.weight", "encoder.transformer.transformer.layers.10.1.norm.bias", "encoder.transformer.transformer.layers.10.1.fn.net.0.weight", "encoder.transformer.transformer.layers.10.1.fn.net.0.bias", "encoder.transformer.transformer.layers.10.1.fn.net.3.weight", "encoder.transformer.transformer.layers.10.1.fn.net.3.bias", "encoder.transformer.transformer.layers.11.0.norm.weight", "encoder.transformer.transformer.layers.11.0.norm.bias", "encoder.transformer.transformer.layers.11.0.fn.to_qkv.weight", "encoder.transformer.transformer.layers.11.0.fn.to_out.0.weight", "encoder.transformer.transformer.layers.11.0.fn.to_out.0.bias", "encoder.transformer.transformer.layers.11.1.norm.weight", "encoder.transformer.transformer.layers.11.1.norm.bias", "encoder.transformer.transformer.layers.11.1.fn.net.0.weight", "encoder.transformer.transformer.layers.11.1.fn.net.0.bias", "encoder.transformer.transformer.layers.11.1.fn.net.3.weight", "encoder.transformer.transformer.layers.11.1.fn.net.3.bias", "encoder.transformer.convd1.0.weight", "encoder.transformer.convd1.0.bias", "decoder.layer1.double_conv.0.weight", "decoder.layer1.double_conv.0.bias", "decoder.layer1.double_conv.2.weight", "decoder.layer1.double_conv.2.bias", "decoder.layer2.double_conv.0.weight", "decoder.layer2.double_conv.0.bias", "decoder.layer2.double_conv.2.weight", "decoder.layer2.double_conv.2.bias", "decoder.outc.conv.weight", "decoder.outc.conv.bias".
Unexpected key(s) in state_dict: "module.encoder.inc.double_conv.0.weight", "module.encoder.inc.double_conv.0.bias", "module.encoder.inc.double_conv.2.weight", "module.encoder.inc.double_conv.2.bias", "module.encoder.layer1.double_conv.0.weight", "module.encoder.layer1.double_conv.0.bias", "module.encoder.layer1.double_conv.2.weight", "module.encoder.layer1.double_conv.2.bias", "module.encoder.layer2.double_conv.0.weight", "module.encoder.layer2.double_conv.0.bias", "module.encoder.layer2.double_conv.2.weight", "module.encoder.layer2.double_conv.2.bias", "module.encoder.transformer.cls_token", "module.encoder.transformer.to_patch_embedding.1.weight", "module.encoder.transformer.to_patch_embedding.1.bias", "module.encoder.transformer.transformer.layers.0.0.norm.weight", "module.encoder.transformer.transformer.layers.0.0.norm.bias", "module.encoder.transformer.transformer.layers.0.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.0.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.0.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.0.1.norm.weight", "module.encoder.transformer.transformer.layers.0.1.norm.bias", "module.encoder.transformer.transformer.layers.0.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.0.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.0.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.0.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.1.0.norm.weight", "module.encoder.transformer.transformer.layers.1.0.norm.bias", "module.encoder.transformer.transformer.layers.1.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.1.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.1.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.1.1.norm.weight", "module.encoder.transformer.transformer.layers.1.1.norm.bias", "module.encoder.transformer.transformer.layers.1.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.1.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.1.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.1.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.2.0.norm.weight", "module.encoder.transformer.transformer.layers.2.0.norm.bias", "module.encoder.transformer.transformer.layers.2.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.2.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.2.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.2.1.norm.weight", "module.encoder.transformer.transformer.layers.2.1.norm.bias", "module.encoder.transformer.transformer.layers.2.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.2.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.2.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.2.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.3.0.norm.weight", "module.encoder.transformer.transformer.layers.3.0.norm.bias", "module.encoder.transformer.transformer.layers.3.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.3.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.3.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.3.1.norm.weight", "module.encoder.transformer.transformer.layers.3.1.norm.bias", "module.encoder.transformer.transformer.layers.3.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.3.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.3.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.3.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.4.0.norm.weight", "module.encoder.transformer.transformer.layers.4.0.norm.bias", "module.encoder.transformer.transformer.layers.4.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.4.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.4.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.4.1.norm.weight", "module.encoder.transformer.transformer.layers.4.1.norm.bias", "module.encoder.transformer.transformer.layers.4.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.4.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.4.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.4.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.5.0.norm.weight", "module.encoder.transformer.transformer.layers.5.0.norm.bias", "module.encoder.transformer.transformer.layers.5.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.5.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.5.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.5.1.norm.weight", "module.encoder.transformer.transformer.layers.5.1.norm.bias", "module.encoder.transformer.transformer.layers.5.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.5.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.5.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.5.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.6.0.norm.weight", "module.encoder.transformer.transformer.layers.6.0.norm.bias", "module.encoder.transformer.transformer.layers.6.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.6.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.6.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.6.1.norm.weight", "module.encoder.transformer.transformer.layers.6.1.norm.bias", "module.encoder.transformer.transformer.layers.6.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.6.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.6.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.6.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.7.0.norm.weight", "module.encoder.transformer.transformer.layers.7.0.norm.bias", "module.encoder.transformer.transformer.layers.7.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.7.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.7.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.7.1.norm.weight", "module.encoder.transformer.transformer.layers.7.1.norm.bias", "module.encoder.transformer.transformer.layers.7.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.7.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.7.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.7.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.8.0.norm.weight", "module.encoder.transformer.transformer.layers.8.0.norm.bias", "module.encoder.transformer.transformer.layers.8.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.8.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.8.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.8.1.norm.weight", "module.encoder.transformer.transformer.layers.8.1.norm.bias", "module.encoder.transformer.transformer.layers.8.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.8.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.8.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.8.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.9.0.norm.weight", "module.encoder.transformer.transformer.layers.9.0.norm.bias", "module.encoder.transformer.transformer.layers.9.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.9.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.9.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.9.1.norm.weight", "module.encoder.transformer.transformer.layers.9.1.norm.bias", "module.encoder.transformer.transformer.layers.9.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.9.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.9.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.9.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.10.0.norm.weight", "module.encoder.transformer.transformer.layers.10.0.norm.bias", "module.encoder.transformer.transformer.layers.10.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.10.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.10.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.10.1.norm.weight", "module.encoder.transformer.transformer.layers.10.1.norm.bias", "module.encoder.transformer.transformer.layers.10.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.10.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.10.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.10.1.fn.net.3.bias", "module.encoder.transformer.transformer.layers.11.0.norm.weight", "module.encoder.transformer.transformer.layers.11.0.norm.bias", "module.encoder.transformer.transformer.layers.11.0.fn.to_qkv.weight", "module.encoder.transformer.transformer.layers.11.0.fn.to_out.0.weight", "module.encoder.transformer.transformer.layers.11.0.fn.to_out.0.bias", "module.encoder.transformer.transformer.layers.11.1.norm.weight", "module.encoder.transformer.transformer.layers.11.1.norm.bias", "module.encoder.transformer.transformer.layers.11.1.fn.net.0.weight", "module.encoder.transformer.transformer.layers.11.1.fn.net.0.bias", "module.encoder.transformer.transformer.layers.11.1.fn.net.3.weight", "module.encoder.transformer.transformer.layers.11.1.fn.net.3.bias", "module.encoder.transformer.convd1.0.weight", "module.encoder.transformer.convd1.0.bias", "module.decoder.layer1.double_conv.0.weight", "module.decoder.layer1.double_conv.0.bias", "module.decoder.layer1.double_conv.2.weight", "module.decoder.layer1.double_conv.2.bias", "module.decoder.layer2.double_conv.0.weight", "module.decoder.layer2.double_conv.0.bias", "module.decoder.layer2.double_conv.2.weight", "module.decoder.layer2.double_conv.2.bias", "module.decoder.outc.conv.weight", "module.decoder.outc.conv.bias".
请问这是训练的模型和测试网络不一样了吗

[keviner1]

22年那时候下的模型是可以跑的，不知道作者的谷歌网盘后面有没有更新模型。