ScatterElements not supported

[leavittx]

Hi. This is similar to #124
The solution suggested isn't any good: it's much worser quality-wise.
ScatterElements should be properly supported in order to allow sophisticated models usage inside Unity

[AlexRibard]

@leavittx thanks for the suggestion, we will take note.
If you have a specific model which uses that op, feel free to share it as it so we can test that it works

[leavittx]

Here is the model you can test with:
https://github.com/PeterL1n/BackgroundMattingV2
ONNX files:
https://drive.google.com/drive/folders/1cbetlrKREitIgjnIikG1HdM4x72FtgBh

[brgishy]

If Barracuda could run the BackgroundMattingV2 onnx model, that would be amazing!

[AlexRibard]

agreed, bumping up the priority

[FlorentGuinier]

ScatterElement support have been added to the backlog, closing the github issue. Will let you guys know when it happens (no ETA for now).

[AlexRibard]

Hey guys, I wanted to say that we added support for BackgroundMattingv2.
It will be present in 2.3.0
It does require you exporting the model with fixed input size. But otherwise than that it works on the base model.

[tramper2]

Hey guys, I wanted to say that we added support for BackgroundMattingv2. It will be present in 2.3.0 It does require you exporting the model with fixed input size. But otherwise than that it works on the base model.

I am using barracuda 2.3.1

Still Error "Asset import failed, "Assets/Artractive/Onnx/onnx_mobilenetv2_hd.onnx" > OnnxImportException: Unknown type ScatterElements encountered while parsing layer 878."

[AlexRibard]

Hello! In order to be able to use BackgroundMattingV2 models, you'll need to export the .pth to onnx with fixed input size
Here is the code to achieve that:

    // https://github.com/PeterL1n/BackgroundMattingV2
    // Download the github repo.
    // Download the pth models here https://drive.google.com/drive/folders/1cbetlrKREitIgjnIikG1HdM4x72FtgBh
    // Execute this script:
       import torch
    
       from model import MattingBase, MattingRefine
    
       for model_backbone in ['resnet101', 'resnet50', 'mobilenetv2']:
           model = MattingBase(model_backbone)
           model.load_state_dict(torch.load('pytorch_' + model_backbone + '.pth', map_location='cpu'), strict=False)
    
           src = torch.randn(2, 3, 224, 224)
           bgr = torch.randn(2, 3, 224, 224)
    
           # Export ONNX
           input_names=['src', 'bgr']
           output_names = ['pha', 'fgr', 'err', 'hid']
    
    
           torch.onnx.export(
               model=model,
               args=(src, bgr),
               f='MattingBase_' + model_backbone + '.onnx',
               verbose=True,
               opset_version=12,
               do_constant_folding=True,
               input_names=input_names,
               output_names=output_names)

Let me know how that works out for you

[leavittx]

Hi there!
I was able to convert the model and import in Unity 2021.2.15f1 / Barracuda 3.0.0. Great work!
Opening another issue for a newer model which was released just recently and doesn't require background clean image :)

[Thaina]

3.0.0 still got OnnxImportException: Unknown type ScatterElements encountered while parsing layer 2704.