CUnet Model Conversion

[tcyrus]

When converting to CoreML, I get the following error

Traceback (most recent call last):
  File "scripts/caffe2coreml2onnx.py", line 41, in run
    convert_model(model_name, path)
  File "scripts/caffe2coreml2onnx.py", line 23, in convert_model
    coreml_model = coremltools.converters.caffe.convert((input_caffe_path, proto_file))
  File "/lib/python3.6/site-packages/coremltools/converters/caffe/_caffe_converter.py", line 192, in convert
    predicted_feature_name)
  File "/lib/python3.6/site-packages/coremltools/converters/caffe/_caffe_converter.py", line 260, in _export
    predicted_feature_name)
RuntimeError: Unsupported option '0' for the parameter 'axis' in layer 'Scale1' of type 'Scale' during caffe conversion.

This is caused by the check performed in CoreML

Based on my limited understanding of the documentation, I need to convert this to this, which doesn't seem as simple as I originally thought.

One solution is to write a script to build the ONNX model directly from the weights (see here).

[tcyrus]

Tried again with mmdnn. Encountered issue with the Crop Layer (microsoft/MMdnn#576)

[tcyrus]

Found a PyTorch implementation of CUnet (antonpaquin/waifu2x-cunet-pytorch), which exports ONNX models. This might be the easiest way to generate Cunet models.

[tcyrus]

The PyTorch Implementation was a pain to get working broke on all models which didn't involve scaling:

• noise0_model
• noise1_model
• noise2_model

Because of it's unique input shape (batch x 3 x width x height), I decided to use a pre-defined input shape (1x3x220x220) for onnx-simplifier (original caffe model was 1x3x200x200).

[tcyrus]

There are a couple of Differences between the Caffe Model and the ONNX Model when viewed in Netron (again).
I'm only going to list the ones which worry me, since the rest are probably fine:

1. Pooling is replaced with ReduceMean
   This is the same issue as with the UpResNet10 Model Conversion

2. Flatten + Scale is replaced with Mul
   This might have been a solution to the axis: 0 issue with Scale when converting models

3. Split + Crop + Eltwise is replaced with Slice + Slice + Add
   Unclear on how this works

4. New Clip at the end of the model
   Unclear on how this works either

[tcyrus]

After manual modification to the PyTorch Implementation (something which should've broken everything), I was able to export the missing models.

[tcyrus]

I decided to take a peek at the models used in the ncnn implementation, which is making things a bit more confusing.

• Slice + Slice + Add might need to be something along the lines of Slice + Sum
• ReduceMean should probably be GlobalAveragePool
• Clip shouldn't cause any problems, but the fact that it's there is concerning