How to load original images for model inference

[EarthMu]

Hi, I am invsetigating in examples/portable/executor_runner/executor_runner.cpp.
And on the PrepareInputTensors, method of PrepareInputTensor generated just ones-initialized inputs.

So I would like to know how to load original dataset images and set them in Aten.
Is it using opencv or other way?

Thanks

[kimishpatel]

If you would like to load images then you will have to convert images to tensors. At the moment we dont have any utils that convert popular image formats to tensor.

cc: @tarun292

[haowhsu-quic]

Hi @EarthMu , you might take a look at the context of qnn_executor_runner, that's how the data get loaded.

[EarthMu]

@haowhsu-quic @kimishpatel
Thank you for your help.
Now I could do inference in following step

1. Loading test image by OpenCV
2. Converting image to Tensor
3. Inference by resnet18.pte

And I got inference output below.

(executorch) root@c2aef39cb16e:~/test/executorch# ./cmake-out/executor_runner --model_path ./resnet18.pte --img_path test.jpg 
Number of arguments: 5
Argument 0: ./cmake-out/executor_runner
Argument 1: --model_path
Argument 2: ./resnet18.pte
Argument 3: --img_path
Argument 4: test.jpg
I 00:00:00.347068 executorch:executor_runner.cpp:139] Model file ./resnet18.pte is loaded.
I 00:00:00.347126 executorch:executor_runner.cpp:148] Using method forward
I 00:00:00.347132 executorch:executor_runner.cpp:196] Setting up planned buffer 0, size 64346944.
I 00:00:00.413278 executorch:executor_runner.cpp:219] Method loaded.
I 00:00:00.413572 executorch:executor_runner.cpp:225] Inputs prepared.
I 00:02:00.670839 executorch:executor_runner.cpp:234] Model executed successfully.
I 00:02:00.670888 executorch:executor_runner.cpp:238] 1 outputs: 
Output type is: void printTypeName() [with T = torch::executor::EValue&]
Output 0: tensor(sizes=[1, 1000], [
  10.1631, 29.5934, -80.133, -70.9052, -59.4912, 30.5375, -108.018, -24.1582, -9.51439, -150.054, 
  -56.1975, -60.2627, -49.9485, -95.7751, -168.019, -114.918, -97.2954, -178.826, -141.454, -178.339, 
  -203.215, -42.9365, -156.571, -97.5982, -137.676, -65.9014, -115.241, -123.231, -48.8396, -105.065, 
  -133.597, -88.7256, -77.8725, -53.2645, -20.7157, -72.7306, -38.8375, -59.6909, -14.2671, -87.1652, 
  -117.911, -127.532, -103.583, -22.9555, -75.6276, -31.3488, -97.7795, -61.7843, -131.965, -218.106, 
  -73.1267, 13.7699, -78.0202, -112.569, -73.0757, -124.394, -48.369, -134.85, -152.321, -66.679, 
  -31.9279, -44.1484, -34.4682, -37.7557, -43.6102, -48.2627, -30.1534, -76.5923, -13.9441, -67.3019, 
  -146.33, -32.8276, -75.2721, 41.9233, -106.458, -14.8469, -63.1998, -20.4347, -19.5419, -22.3364, 
  -93.5414, -179.63, -36.7796, -32.3285, -224.914, -76.9614, -18.0298, -1.30809, -34.3693, -168.224, 
  -81.3687, -87.1171, 17.6901, -65.3884, -138.492, -116.954, -49.8375, -127.506, -82.367, -90.7725, 
  ...,
  -74.4797, 7.64197, 42.4845, 112.996, 44.8767, 24.4547, 27.0888, 13.6372, -74.9592, 112.531, 
  6.67652, 45.2616, 7.5626, -79.3771, -168.811, -79.7176, 177.784, 136.511, 57.1827, -53.6283, 
  -37.6729, 117.584, 99.0207, 40.6545, -37.3457, -41.9832, -5.2025, 27.3043, -73.377, -51.8356, 
  91.3868, 13.4542, 26.0998, -60.7956, -23.3684, -85.9262, 63.7492, 2.84164, 11.6112, 50.6081, 
  98.862, 52.0422, -17.6478, -22.7001, 64.781, 32.8963, -2.48067, -65.0722, 40.8774, 50.8649, 
  -6.48622, -46.266, 48.7241, 44.1465, 31.0656, -56.4973, 21.6369, -6.46106, 53.2719, 41.9332, 
  -10.633, -24.8715, 38.7212, 38.6888, 62.8701, -78.5487, 44.4625, -103.772, 43.0778, 76.7782, 
  -59.0828, -57.7173, -55.8327, -4.88989, -117.619, -62.0024, -63.264, -77.2408, -25.6094, -83.3731, 
  -70.2586, -20.0537, -4.33933, -84.4149, -37.0248, -12.0172, -67.3394, 108.699, -20.7449, -78.1383, 
  -4.86601, -154.709, -85.575, -84.8423, -141.719, -102.445, 9.6363, -82.6206, 93.7396, 96.0744, 
])

Then I need to get highest value from these output like using torch::nn::functional::softmax(tensor, /*dim=*/1);.
Should I install LibTorch and build from scratch?
Or is there other recommendation for this kind of post-process?

Thanks,

[kimishpatel]

Can you make softmax part of your model?

[EarthMu]

Thank you.
I could exported resnet.pte with softmax layer like following.

import torch
import torchvision.models as models
from torch._export import capture_pre_autograd_graph
from torch.export import export, ExportedProgram
import executorch.exir as exir


# ========== resnet18 + softmax layer ============
resnet18 = models.resnet18(weights=ResNet18_Weights.DEFAULT).eval()
resnet18.fc = torch.nn.Sequential(
        resnet18.fc,
        torch.nn.Softmax(dim=1)
        )
example_args = (torch.randn(1, 3, 224, 224), )
# ====================================
## export to exir
pre_autograd_aten_dialect = capture_pre_autograd_graph(resnet18, example_args)
## export to aten dialect
aten_dialect: ExportedProgram = export(pre_autograd_aten_dialect, example_args)
## export to edge
edge_program: exir.EdgeProgramManager = exir.to_edge(aten_dialect)
## export to executorch
from executorch.exir import ExecutorchBackendConfig, ExecutorchProgramManager
executorch_program: exir.ExecutorchProgramManager = edge_program.to_executorch(
    ExecutorchBackendConfig(
        passes=[],  # User-defined passes
    )
)
## save pte model
print("save pte file")
with open("exported-resnet18.pte", "wb") as file:
    file.write(executorch_program.buffer)

[SS-JIA]

closing this issue, since it seems that the original question was resolved.

[namba-ferix]

Hi @EarthMu , I'm currently struggling with the same process below.
Could you please share your .cpp and .bzl files?
Seeing how you've implemented these steps in your code would be incredibly helpful to me.

Thank you for your help. Now I could do inference in following step

1. Loading test image by OpenCV
2. Converting image to Tensor
3. Inference by resnet18.pte