Not able to convert the 2x model from PyTorch to ONNX format

[qunliu]

Hi there,

First of all, thanks for your amazing work!

I am trying to use the script here to convert the pytorch model to onnx format: https://github.com/xinntao/Real-ESRGAN/blob/master/scripts/pytorch2onnx.py It works great and converts okay for the RealESRGAN_x4plus.pth model. However it gives error message below for the x2 model RealESRGAN_x2plus.pth. I was wondering if I missed anything very simple there in the script?

Also how would I make the conversion script to also factor in the "--outscale" switch so I may use it in ONNX format? I'd appreciate it if you could give me a heads-up.

Thanks!

RuntimeError: Error(s) in loading state_dict for RRDBNet:
        size mismatch for conv_first.weight: copying a param with shape torch.Size([64, 12, 3, 3]) from checkpoint, the shape in current model is torch.Size([64, 3, 3, 3]).

[xinntao]

For your problem, you should also modify the arguments of the model. As they have different config.

But converting the X2 model requires more work:

1. ncnn does not support well on the pixel-shuffle operation, which is used for X2 model.

[qunliu]

Thanks for the update. Yeah, it sounds like it won't be a simple change for the conversion script. Would you please make this a feature request when you get a chance to work on it and then update the pytorch2onnx.py script to support the 2x model and/or "--outscale" switch?

By the way, I don't use ncnn library, I use the onnx model directly if it is available, so if you could help on getting the onnx model ready, that's good enough for me.

Thanks!

[spacewalk01]

Same here, I want to convert 2x model to onnx. Is there any solution?

[ajaskier]

Same here

[amadeuzou]

just change "model = RRDBNet(num_in_ch=3, num_out_ch=3, num_feat=64, num_block=23, num_grow_ch=32, scale=4)" to "model = RRDBNet(num_in_ch=3, num_out_ch=3, num_feat=64, num_block=23, num_grow_ch=32, scale=2)" for RealESRGAN_x2plus.pth

[shakibakh]

@amadeuzou @xinntao I also try to fine-tune x2 network on my own data, but get the same error:
RuntimeError: Error(s) in loading state_dict for RRDBNet:
size mismatch for conv_first.weight: copying a param with shape torch.Size([64, 12, 3, 3]) from checkpoint, the shape in current model is torch.Size([64, 3, 3, 3]).

The config change as @amadeuzou would not solve it as it seems the network expects the num_in_ch to be 12! Can you explain why? and how can I prepare my data for it?

Edit: Figured out what I was doing wrong. I was using the file finetune_realesrgan_x4plus_pairdata.yml for finetuning and changed the scale to 2. But there is another scale in network_g which they didn't write it here. Once added and changed to 2, then it works