Simple custom model is not working on android

[nh9k]

Tutorial Select

Prepare Custom Model

Feedback

Hello, always thanks for contributions!
This is somewhat custom problem i think, so that i don't know that i can question.
My simple tensor manipulation model is not working on my react-native app, but it is working successfully at my python code.

My adb logcat of several code trials sometimes got output memory consumptions problem
or
my java script code sometimes got output [Error: Exception in HostFunction: vector] or std::bad_alloc
or
just app crashes.

Can i get some help?..

Model export at python code:

import torch, torchvision
import os
from typing import List, Dict
import cv2
import torchvision.transforms.functional as F

class PostCD(torch.nn.Module):
    def __init__(self):
        super().__init__()

    def forward(self, inputs: torch.Tensor, iouThreshold: float):
        #shape e.g., "shape": [1, 25200, 16]
        
        inputs = inputs[inputs[:,:,4] > iouThreshold]
        max_class_tensor = torch.transpose(torch.argmax(inputs[:,5:], dim=1).unsqueeze(0),0,1)
        outputs = torch.cat((inputs[:,:5], max_class_tensor), 1).unsqueeze(0)
        return outputs

    
pcd = PostCD()
scripted_model = torch.jit.script(pcd)

data = [[247.37535095214844, 525.0855712890625, 129.0428924560547, 31.98747444152832, 0.8668637275695801, 0.0027807741425931454, 0.002542165108025074, 0.00628146156668663, 0.0026534167118370533, 0.002410427201539278, 0.0035515064373612404, 0.00348650268279016, 0.006360366474837065, 0.002872081473469734, 0.0036974542308598757, 0.9905574321746826],
        [247.37535095214844, 525.0855712890625, 129.0428924560547, 31.98747444152832, 0.8668637275695801, 0.9027807741425931454, 0.002542165108025074, 0.00628146156668663, 0.0026534167118370533, 0.002410427201539278, 0.0035515064373612404, 0.00348650268279016, 0.006360366474837065, 0.002872081473469734, 0.0036974542308598757, 0.005574321746826],
        [247.37535095214844, 525.0855712890625, 129.0428924560547, 31.98747444152832, 0.1, 0.0027807741425931454, 0.002542165108025074, 0.00628146156668663, 0.0026534167118370533, 0.002410427201539278, 0.0035515064373612404, 0.00348650268279016, 0.006360366474837065, 0.002872081473469734, 0.0036974542308598757, 0.9905574321746826],
        [247.37535095214844, 525.0855712890625, 129.0428924560547, 31.98747444152832, 0.1, 0.0027807741425931454, 0.9, 0.00628146156668663, 0.0026534167118370533, 0.002410427201539278, 0.0035515064373612404, 0.00348650268279016, 0.006360366474837065, 0.002872081473469734, 0.0036974542308598757, 0.0004000],
        [247.37535095214844, 525.0855712890625, 129.0428924560547, 31.98747444152832, 0.1, 0.0027807741425931454, 0.9, 0.00628146156668663, 0.0026534167118370533, 0.002410427201539278, 0.0035515064373612404, 0.00348650268279016, 0.006360366474837065, 0.002872081473469734, 0.0036974542308598757, 0.0004000],
        [247.37535095214844, 525.0855712890625, 129.0428924560547, 31.98747444152832, 0.1, 0.0027807741425931454, 0.9, 0.00628146156668663, 0.0026534167118370533, 0.002410427201539278, 0.0035515064373612404, 0.00348650268279016, 0.006360366474837065, 0.002872081473469734, 0.0036974542308598757, 0.0004000],
        ]
x_data = torch.tensor(data).unsqueeze(0)

print(x_data)
print(x_data.shape, end='\n\n')
    
outputs = scripted_model(x_data, 0.3)

print(outputs)
print(outputs.shape)

scripted_model.save("post_cd.pt")

import torch
from torch.utils.mobile_optimizer import optimize_for_mobile

optimized_scripted_module=optimize_for_mobile(scripted_model)
optimized_scripted_module._save_for_lite_interpreter("post_cd.ptl")

Python output:

tensor([[[2.4738e+02, 5.2509e+02, 1.2904e+02, 3.1987e+01, 8.6686e-01,
          2.7808e-03, 2.5422e-03, 6.2815e-03, 2.6534e-03, 2.4104e-03,
          3.5515e-03, 3.4865e-03, 6.3604e-03, 2.8721e-03, 3.6975e-03,
          9.9056e-01],
         [2.4738e+02, 5.2509e+02, 1.2904e+02, 3.1987e+01, 8.6686e-01,
          9.0278e-01, 2.5422e-03, 6.2815e-03, 2.6534e-03, 2.4104e-03,
          3.5515e-03, 3.4865e-03, 6.3604e-03, 2.8721e-03, 3.6975e-03,
          5.5743e-03],
         [2.4738e+02, 5.2509e+02, 1.2904e+02, 3.1987e+01, 1.0000e-01,
          2.7808e-03, 2.5422e-03, 6.2815e-03, 2.6534e-03, 2.4104e-03,
          3.5515e-03, 3.4865e-03, 6.3604e-03, 2.8721e-03, 3.6975e-03,
          9.9056e-01],
         [2.4738e+02, 5.2509e+02, 1.2904e+02, 3.1987e+01, 1.0000e-01,
          2.7808e-03, 9.0000e-01, 6.2815e-03, 2.6534e-03, 2.4104e-03,
          3.5515e-03, 3.4865e-03, 6.3604e-03, 2.8721e-03, 3.6975e-03,
          4.0000e-04],
         [2.4738e+02, 5.2509e+02, 1.2904e+02, 3.1987e+01, 1.0000e-01,
          2.7808e-03, 9.0000e-01, 6.2815e-03, 2.6534e-03, 2.4104e-03,
          3.5515e-03, 3.4865e-03, 6.3604e-03, 2.8721e-03, 3.6975e-03,
          4.0000e-04],
         [2.4738e+02, 5.2509e+02, 1.2904e+02, 3.1987e+01, 1.0000e-01,
          2.7808e-03, 9.0000e-01, 6.2815e-03, 2.6534e-03, 2.4104e-03,
          3.5515e-03, 3.4865e-03, 6.3604e-03, 2.8721e-03, 3.6975e-03,
          4.0000e-04]]])
torch.Size([1, 6, 16])

tensor([[[247.3754, 525.0856, 129.0429,  31.9875,   0.8669,  10.0000],
         [247.3754, 525.0856, 129.0429,  31.9875,   0.8669,   0.0000]]])
torch.Size([1, 2, 6])

This is app code:

const MODEL_URL = localPath;
let pcd_model = null;

async function testPCD(){
    if (pcd_model == null) {
        const filePath = await MobileModel.download(require(MODEL_URL));
        pcd_model = await torch.jit._loadForMobile(filePath);
        console.log('Model successfully loaded');
    }


    var data = [[247.37535095214844, 525.0855712890625, 129.0428924560547, 31.98747444152832, 0.8668637275695801, 0.0027807741425931454, 0.002542165108025074, 0.00628146156668663, 0.0026534167118370533, 0.002410427201539278, 0.0035515064373612404, 0.00348650268279016, 0.006360366474837065, 0.002872081473469734, 0.0036974542308598757, 0.9905574321746826],
                [247.37535095214844, 525.0855712890625, 129.0428924560547, 31.98747444152832, 0.8668637275695801, 0.9027807741425931454, 0.002542165108025074, 0.00628146156668663, 0.0026534167118370533, 0.002410427201539278, 0.0035515064373612404, 0.00348650268279016, 0.006360366474837065, 0.002872081473469734, 0.0036974542308598757, 0.005574321746826],
                [247.37535095214844, 525.0855712890625, 129.0428924560547, 31.98747444152832, 0.1, 0.0027807741425931454, 0.002542165108025074, 0.00628146156668663, 0.0026534167118370533, 0.002410427201539278, 0.0035515064373612404, 0.00348650268279016, 0.006360366474837065, 0.002872081473469734, 0.0036974542308598757, 0.9905574321746826],
                [247.37535095214844, 525.0855712890625, 129.0428924560547, 31.98747444152832, 0.1, 0.0027807741425931454, 0.9, 0.00628146156668663, 0.0026534167118370533, 0.002410427201539278, 0.0035515064373612404, 0.00348650268279016, 0.006360366474837065, 0.002872081473469734, 0.0036974542308598757, 0.0004000],
                [247.37535095214844, 525.0855712890625, 129.0428924560547, 31.98747444152832, 0.1, 0.0027807741425931454, 0.9, 0.00628146156668663, 0.0026534167118370533, 0.002410427201539278, 0.0035515064373612404, 0.00348650268279016, 0.006360366474837065, 0.002872081473469734, 0.0036974542308598757, 0.0004000],
                [247.37535095214844, 525.0855712890625, 129.0428924560547, 31.98747444152832, 0.1, 0.0027807741425931454, 0.9, 0.00628146156668663, 0.0026534167118370533, 0.002410427201539278, 0.0035515064373612404, 0.00348650268279016, 0.006360366474837065, 0.002872081473469734, 0.0036974542308598757, 0.0004000],
                ]
    var x_data = torch.tensor(data).unsqueeze(0);
    console.log(x_data);
    console.log(x_data.shape);

    try{
        const startInferencTime = global.performance.now();

        const outputs = await pcd_model.forward(x_data, 0.3);

        console.log(outputs);
        console.log(outputs.shape);

        const inferenceTime = global.performance.now() - startInferencTime;
        console.log(`inference time ${inferenceTime.toFixed(3)} ms`);
    }
    catch(err){
        console.log(err);
    }
}

Node.js output:

 LOG  Model successfully loaded
 LOG  {"abs": [Function abs], "add": [Function add], "argmax": [Function argmax], "argmin": [Function argmin], "clamp": [Function clamp], "contiguous": [Function contiguous], "data": [Function data], "div": [Function div], "dtype": "float32", "expand": [Function expand], "flip": [Function flip], "item": [Function item], "mul": [Function mul], "permute": [Function permute], "reshape": [Function reshape], "shape": [1, 6, 16], "size": [Function size], "softmax": [Function softmax], "sqrt": [Function sqrt], "squeeze": [Function squeeze], "stride": [Function stride], "sub": [Function sub], "sum": [Function sum], "to": [Function to], "toString": [Function toString], "topk": [Function topk], "unsqueeze": [Function unsqueeze]}
 LOG  [1, 6, 16]

-> App crashes

[raedle]

@nh9k, what react-native-pytorch-core version are you using?

The model and the JavaScript code provided work on both Android (Samsung Galaxy S22) and iOS (iPhone Xs) and with the PlayTorch app (which uses react-native-pytorch-core 0.2.2).

PlayTorch Snack url: https://snack.playtorch.dev/@raedle/post-cd-test?supportedPlatforms=my-device

[nh9k]

@raedle, thank you very much for your testing 😃, but i am still getting same errors on my android Pixel 5, API 29, RAM 6GB/8GB, Pixel 5, API 30, 8GB, and Galaxy S9+, Android 10.0 at version react-native-core@0.2.2 😢....

[raedle]

@nh9k, what a bummer. Is this also happening with the PlayTorch Snack provided in my previous response?

[nh9k]

I am not familiar with the expo, but I am trying it with my galaxy S9+.
There is another error before the test properly.

[nh9k]

@raedle, this Playtorch Snack is working fine on my galaxy S9+!

[raedle]

@nh9k, the code examples that you provided work. I can probably look into it again, but it will require a minimal and constantly reproducible example (e.g., via GitHub repo)

[nh9k]

@raedle, thank you so much, i figure out what is the problem! 😢😢😢
The problem was .ptl model..
Your model is working fine at my native app.
Is it used same code with my question?

[raedle]

I copied and pasted your Python PostCD model code in Google Colab and exported the post_cd.ptl model

https://colab.research.google.com/drive/1MPM9v6TM_q7R4SqhMSzXwCCQYNqWk63F

[nh9k]

Current colab pytorch version is 1.12.1, but my pytorch version is 1.10.0.
I think version 1.10.0 may be the problem.
I'm always thankful and sorry to you.

[raedle]

@nh9k an older PyTorch version could indeed be the cause. Can you try PyTorch 1.12.1 and report back? Thank you!

[nh9k]

@raedle, I made a mistake. My PyTorch version was 1.7.1.
!conda list command in jupyter notebook did not work, so I used conda list instead.
PyTorch version 1.10.0, 1.11.0, and 1.12.1 were working fine on android! Thank you so much!