Mismatching Pytorch Model Output vs Converted Core ML Model

[HussainHaris]

I fine tuned and trained a resnet50 on Pytorch for 8 labels instead of 1000.
When testing the model/passing in images on Pytorch I get the correct tensor of probabilities as seen below.

On the Core ML model I get 0 for everything except for one label("living room") every single time, in which case I get 1. See below

Why could they not be matching? I've compared the resnet50 model provided by Apple converted in CoreML to mine converted from Pytorch -> ONNX -> CoreML in Netron and I don't understand why I'm getting mismatching confidence values from pytorch vs CoreML.

Thank you in advance!

System/Model Information + Conversion Snippets

• Mac OS 10.14

[aseemw]

Do you get the correct answer when running predict on the onnx model using onnxruntime? This will help narrow down the issue, which could be either in the pytorch to onnx stage or onnx to coreml stage.

[jbrown15]

Do you get the correct answer when running predict on the onnx model using onnxruntime? This will help narrow down the issue, which could be either in the pytorch to onnx stage or onnx to coreml stage.

yes, I have the same issue and I get the same prediction in both pytorch and onnx.

[TobyRoseman]

@jbrown15 and @HussainHaris - is this still an issue with the latest coremltools? If so, please try converting your model directly from PyTorch using the Unified Converter.

[jbrown15]

Thank you for your response. But I m trying to convert from coreML to pytorch

…

On Fri, Oct 8, 2021 at 4:44 PM Toby Roseman ***@***.***> wrote: @jbrown15 <https://github.com/jbrown15> and @HussainHaris <https://github.com/HussainHaris> - is this still an issue with the latest coremltools? If so, please try converting your model directly from PyTorch using the Unified Converter <https://coremltools.readme.io/docs/unified-conversion-api>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#489 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABEGISGIZPQL2GIMX2I4ATDUF5JS7ANCNFSM4JARMM3Q> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[TobyRoseman]

Thank you for your response. But I m trying to convert from coreML to pytorch
…

Do you mean you’re trying to convert a PyTorch model to Core ML?

[jbrown15]

No. The opposite. I have a coreML model and I I want to convert it to pytorch

…

On Fri, Oct 8, 2021 at 7:25 PM Toby Roseman ***@***.***> wrote: Thank you for your response. But I m trying to convert from coreML to pytorch … <#m_-3648117392743658673_> Do you mean you’re trying to convert a PyTorch model to Core ML? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#489 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABEGISFGERRZKMOE6USDOX3UF54TPANCNFSM4JARMM3Q> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[TobyRoseman]

I have a coreML model and I I want to convert it to pytorch

This is not supported by coremltools. I suggest you create an issue in the appropriate ONNX repository. It sounds like the issue is converting from ONNX to Core ML.