OP set 10 limitation ?

[CristianPopov]

Hello,

I am attempting to quantize https://github.com/gengshan-y/high-res-stereo . So far I have a working PyTorch model, ONNX model, and IR model (after patching ModelOptimizer), all FP32, and all of them have the same accuracy (avg error ~4.7), as expected.

Since the tests with OpenVino POT gave me 2x gain in inference speed but my accuracy was severely affected ( from avg error 4.7 I got to 15 with AccuracyAwareQuantization and to 20-22 with DefaultQuantization, about 4-5 times worse), I wanted to give it a go with NNCF, since I tested this framework in the past, at version 1.6 and had good results on a different model.
But it seems that NNCF is attempting to do the ONNX export with OP set 10, while I need OP set 11 for that network. Is this a limitation?
If I modify NNCF to do the export with OP set 11, and then I have other NNCF bugs, are those bugs considered valid?
Or, if I do the export to ONNX myself, instead of calling the NNCF API, like:
torch.onnx.export(compressed_model, (imgL, imgR), 'hsm_nncf.onnx', opset_version=11, output_names=['prediction', 'entropy'], verbose=True, input_names=['left', 'right'], strip_doc_string=False, training=True, do_constant_folding=False),
and I encounter that other NNCF bug, is it valid?

The reason I need OP set 11 is due to "index_put", at https://github.com/gengshan-y/high-res-stereo/blob/master/models/hsm.py , line 41: def feature_vol(self, refimg_fea, targetimg_fea,maxdisp, leftview=True)

Thanks,
Cristi

[AlexKoff88]

Hi Cristi,

Thanks for your interest in the NNCF. If you have troubles with export to opset 11 please report an issue for that. NNCF should support PyTorch 1.8 which supports opset 11 as I can see.

[CristianPopov]

Hi Alexander,

The issue is not with the export to OP 11. This works fine without NNCF as a wrapper to the model. But when I apply NNCF (even without a compression algorithm), the model is broken and the output tensors have all the values 'inf'. I will document the steps and everything else in a issue and we can continue the discussion there.

[AlexKoff88]

Sounds good to me. Let's proceed with an issue.

[CristianPopov]

Issue created: #873