Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[torchscript] RuntimeError: 'module' object is not subscriptable #101

Closed
czHappy opened this issue Apr 28, 2021 · 12 comments
Closed

[torchscript] RuntimeError: 'module' object is not subscriptable #101

czHappy opened this issue Apr 28, 2021 · 12 comments

Comments

@czHappy
Copy link

czHappy commented Apr 28, 2021

When I export the TorchScript version of MODNet, error occurs:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/sk49/new_workspace/cz/MODNet-master/torchscript/export_torchscript.py", line 46, in <module>
    scripted_model = torch.jit.script(modnet.module)
  File "/home/sk49/.local/lib/python3.6/site-packages/torch/jit/__init__.py", line 1203, in script
    return torch.jit.torch.jit._recursive.recursive_script(obj)
  File "/home/sk49/.local/lib/python3.6/site-packages/torch/jit/_recursive.py", line 173, in recursive_script
    return copy_to_script_module(mod, overload_stubs + stubs)
  File "/home/sk49/.local/lib/python3.6/site-packages/torch/jit/_recursive.py", line 95, in copy_to_script_module
    torch.jit._create_methods_from_stubs(script_module, stubs)
  File "/home/sk49/.local/lib/python3.6/site-packages/torch/jit/__init__.py", line 1423, in _create_methods_from_stubs
    self._c._create_methods(self, defs, rcbs, defaults)
  File "/home/sk49/.local/lib/python3.6/site-packages/torch/jit/_recursive.py", line 195, in make_strong_submodule
    new_strong_submodule = recursive_script(module)
  File "/home/sk49/.local/lib/python3.6/site-packages/torch/jit/_recursive.py", line 173, in recursive_script
    return copy_to_script_module(mod, overload_stubs + stubs)
  File "/home/sk49/.local/lib/python3.6/site-packages/torch/jit/_recursive.py", line 95, in copy_to_script_module
    torch.jit._create_methods_from_stubs(script_module, stubs)
  File "/home/sk49/.local/lib/python3.6/site-packages/torch/jit/__init__.py", line 1423, in _create_methods_from_stubs
    self._c._create_methods(self, defs, rcbs, defaults)
  File "/home/sk49/.local/lib/python3.6/site-packages/torch/jit/_recursive.py", line 195, in make_strong_submodule
    new_strong_submodule = recursive_script(module)
  File "/home/sk49/.local/lib/python3.6/site-packages/torch/jit/_recursive.py", line 173, in recursive_script
    return copy_to_script_module(mod, overload_stubs + stubs)
  File "/home/sk49/.local/lib/python3.6/site-packages/torch/jit/_recursive.py", line 95, in copy_to_script_module
    torch.jit._create_methods_from_stubs(script_module, stubs)
  File "/home/sk49/.local/lib/python3.6/site-packages/torch/jit/__init__.py", line 1423, in _create_methods_from_stubs
    self._c._create_methods(self, defs, rcbs, defaults)
  File "/home/sk49/.local/lib/python3.6/site-packages/torch/jit/_recursive.py", line 195, in make_strong_submodule
    new_strong_submodule = recursive_script(module)
  File "/home/sk49/.local/lib/python3.6/site-packages/torch/jit/_recursive.py", line 173, in recursive_script
    return copy_to_script_module(mod, overload_stubs + stubs)
  File "/home/sk49/.local/lib/python3.6/site-packages/torch/jit/_recursive.py", line 95, in copy_to_script_module
    torch.jit._create_methods_from_stubs(script_module, stubs)
  File "/home/sk49/.local/lib/python3.6/site-packages/torch/jit/__init__.py", line 1423, in _create_methods_from_stubs
    self._c._create_methods(self, defs, rcbs, defaults)
RuntimeError: 
'module' object is not subscriptable:
at /home/sk49/new_workspace/cz/MODNet-master/src/models/backbones/mobilenetv2.py:141:6
	def forward(self, x):
		# Stage1
		x = self.features[0](x)
      ~~~~~~~~~~~~~~~ <--- HERE
		x = self.features[1](x)
		# Stage2
		x = self.features[2](x)
		x = self.features[3](x)
		# Stage3
		x = self.features[4](x)
		x = self.features[5](x)
		x = self.features[6](x)
		# Stage4
'__torch__.src.models.backbones.mobilenetv2.MobileNetV2.forward' is being compiled since it was called from '__torch__.src.models.backbones.wrapper.MobileNetV2Backbone.forward'
at /home/sk49/new_workspace/cz/MODNet-master/src/models/backbones/wrapper.py:40:8
    def forward(self, x):
        # x = reduce(lambda x, n: self.model.features[n](x), list(range(0, 2)), x)
        x = self.model.features[0](x)
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
        x = self.model.features[1](x)
        enc2x = x

        # x = reduce(lambda x, n: self.model.features[n](x), list(range(2, 4)), x)
        x = self.model.features[2](x)
        x = self.model.features[3](x)
        enc4x = x

        # x = reduce(lambda x, n: self.model.features[n](x), list(range(4, 7)), x)
'__torch__.src.models.backbones.wrapper.MobileNetV2Backbone.forward' is being compiled since it was called from '__torch__.torchscript.modnet_torchscript.LRBranch.forward'
at /home/sk49/new_workspace/cz/MODNet-master/torchscript/modnet_torchscript.py:108:8
    def forward(self, img):
        enc_features = self.backbone.forward(img)
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
        enc2x, enc4x, enc32x = enc_features[0], enc_features[1], enc_features[4]

        enc32x = self.se_block(enc32x)
        lr16x = F.interpolate(enc32x, scale_factor=2.0, mode='bilinear', align_corners=False)
        lr16x = self.conv_lr16x(lr16x)
        lr8x = F.interpolate(lr16x, scale_factor=2.0, mode='bilinear', align_corners=False)
        lr8x = self.conv_lr8x(lr8x)

        return lr8x, enc2x, enc4x
'__torch__.torchscript.modnet_torchscript.LRBranch.forward' is being compiled since it was called from '__torch__.torchscript.modnet_torchscript.MODNet.forward'
at /home/sk49/new_workspace/cz/MODNet-master/torchscript/modnet_torchscript.py:230:8
    def forward(self, img):
        # NOTE
        lr_out = self.lr_branch(img)
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
        lr8x = lr_out[0]
        enc2x = lr_out[1]
        enc4x = lr_out[2]

        hr2x = self.hr_branch(img, enc2x, enc4x, lr8x)
        
        pred_matte = self.f_branch(img, lr8x, hr2x)

        return pred_matte

Environment: Centos 7 torch 1.3.1 gpu

@czHappy
Copy link
Author

czHappy commented Apr 28, 2021

@yarkable Could you please help me to solve the problem? Thank you very much!

@yarkable
Copy link
Contributor

It seems that you have some problems while importing modules, maybe you can check if you have a __init__.py in every package.

@czHappy
Copy link
Author

czHappy commented Apr 29, 2021

It seems that you have some problems while importing modules, maybe you can check if you have a __init__.py in every package.

This is the result of 'tree' command of the project root directory:

[sk49@g33 ~/new_workspace/cz/MODNet-master]$ tree
.
|-- demo
|   |-- image_matting
|   |   `-- colab
|   |       |-- inference.py
|   |       `-- README.md
|   `-- video_matting
|       |-- custom
|       |   |-- README.md
|       |   |-- requirements.txt
|       |   `-- run.py
|       `-- webcam
|           |-- README.md
|           |-- requirements.txt
|           `-- run.py
|-- doc
|   `-- gif
|       |-- homepage_demo.gif
|       |-- image_matting_demo.gif
|       `-- video_matting_demo.gif
|-- onnx
|   |-- export_onnx.py
|   |-- inference_onnx.py
|   |-- __init__.py
|   |-- modnet_onnx.py
|   |-- README.md
|   `-- requirements.txt
|-- pretrained
|   |-- mobilenetv2_human_seg.ckpt
|   |-- modnet_photographic_portrait_matting.ckpt
|   |-- modnet_webcam_portrait_matting.ckpt
|   `-- README.md
|-- README.md
|-- src
|   |-- __init__.py
|   |-- models
|   |   |-- backbones
|   |   |   |-- __init__.py
|   |   |   |-- mobilenetv2.py
|   |   |   |-- __pycache__
|   |   |   |   |-- __init__.cpython-36.pyc
|   |   |   |   |-- __init__.cpython-37.pyc
|   |   |   |   |-- mobilenetv2.cpython-36.pyc
|   |   |   |   |-- mobilenetv2.cpython-37.pyc
|   |   |   |   |-- wrapper.cpython-36.pyc
|   |   |   |   `-- wrapper.cpython-37.pyc
|   |   |   `-- wrapper.py
|   |   |-- __init__.py
|   |   |-- modnet.py
|   |   `-- __pycache__
|   |       |-- __init__.cpython-36.pyc
|   |       `-- __init__.cpython-37.pyc
|   |-- __pycache__
|   |   |-- __init__.cpython-36.pyc
|   |   `-- __init__.cpython-37.pyc
|   `-- trainer.py
`-- torchscript
    |-- export_torchscript.py
    |-- __init__.py
    |-- __init__.pyc
    |-- modnet_torchscript.py
    |-- __pycache__
    |   |-- export_torchscript.cpython-36.pyc
    |   |-- export_torchscript.cpython-37.pyc
    |   |-- __init__.cpython-36.pyc
    |   |-- __init__.cpython-37.pyc
    |   |-- modnet_torchscript.cpython-36.pyc
    |   `-- modnet_torchscript.cpython-37.pyc
    `-- README.md

I cloned the whole project and downloaded the models to pretrained folder.
Then I use the command in project root directory:

python3 -m torchscript.export_torchscript \
    --ckpt-path=pretrained/modnet_photographic_portrait_matting.ckpt \
    --output-path=pretrained/modnet_photographic_portrait_matting.torchscript

So I think the init.py is OK.
Could you tell me your environment details or other possible errors in my operations ?

@czHappy
Copy link
Author

czHappy commented Apr 29, 2021

By the way, I try to use the modnet.pt directly to build executable file, but fail to load the model .
The libtorch version is libtorch-cxx11-abi-shared-with-deps-1.3.1
It is cpu version of libtorch for linux.

CMakeLists.txt

cmake_minimum_required(VERSION 3.0 FATAL_ERROR)
project(example-app)

find_package(Torch REQUIRED)

add_executable(example-app example-app.cpp)
target_link_libraries(example-app "${TORCH_LIBRARIES}")
set_property(TARGET example-app PROPERTY CXX_STANDARD 14)

example-app.cpp:

#include <torch/script.h> // One-stop header.
#include <iostream>
#include <memory>

int main(int argc, const char* argv[]) {
  if (argc != 2) {
    std::cerr << "usage: example-app <path-to-exported-script-module>\n";
    return -1;
  }

  torch::jit::script::Module module;
  try {
    // Deserialize the ScriptModule from a file using torch::jit::load().
    module = torch::jit::load(argv[1]);
  }
  catch (const c10::Error& e) {
    std::cerr << "error loading the model\n";
    return -1;
  }

  std::cout << "ok\n";
}

build successfully but fail to execute ./example-app:

command: ./example-app ../modnet.pt
result: error loading the model.

Could you please tell us the environment required and how to use the modnet.pt in cpp?

@yarkable
Copy link
Contributor

Hi, I just clone the project and download official pretrained model, then I type the command

python3 -m torchscript.export_torchscript \
    --ckpt-path=pretrained/modnet_photographic_portrait_matting.ckpt \
    --output-path=pretrained/modnet_photographic_portrait_matting.torchscript

It works successfully, maybe you should clone it once again and see whether the error still occurs?
Btw, I just convert it to TorchScript version for IOS device, we have no problem using modnet.pt.

@czHappy
Copy link
Author

czHappy commented Apr 29, 2021

What is your torch version and libtorch version, please?

@yarkable
Copy link
Contributor

yarkable commented Apr 29, 2021

Btw, here is my code for classification using C++.

int main(int argc, char * argv[]){
    if (argc != 2) {
      std::cerr << "no module found !\n";
      return -1;
    }

    torch::jit::script::Module module;

    try {
        module = torch::jit::load(argv[1]);
    }
    catch (const c10::Error& e){
        std::cerr << "error loading the module\n";
        return -1;
    }

    std::cout << "ok!\n";

    /////////////////////////////////////////////////
    
    string path = "/pytorch-deployment/assets/3.jpg";
    Mat img = imread(path), img_float;
    cvtColor(img, img, CV_BGR2RGB); 
    bitwise_not(img, img);
    vector<Mat> mv;
    split(img, mv);
    img = mv[1]; 
    img.convertTo(img_float, CV_32F, 1.0 / 255);  
    resize(img_float, img_float , Size(28, 28));  
    auto img_tensor = torch::from_blob(img_float.data, {1, 28, 28, 1}, at::kFloat).permute({ 0,3,1,2 });
    auto img_var = torch::autograd::make_variable(img_tensor, false);

    // Create a vector of inputs.
    std::vector<torch::jit::IValue> inputs;
    inputs.push_back(img_var);

    auto output = module.forward(inputs).toTensor();  
    cout << output << endl;
    auto index = output.argmax(1);
    cout << "The predicted class is : " << index << endl;
}

You can re-clone the project and generate a TorchScript model to see if your code is correct.

@yarkable
Copy link
Contributor

torch 1.6.0 and I don't have a libtorch

@czHappy
Copy link
Author

czHappy commented Apr 29, 2021

I tried again with torch verion 1.3.1(gpu), the same errors occured. Maybe I should install torch 1.6.0 and try again.
And if you don't have a libtorch, how can you use 'torch::jit::script::Module' in your cpp file? Which headers do you include before main function?

@yarkable
Copy link
Contributor

@czHappy Lol, I used to use libtorch. But in this project, I just export it to TorchScript version and give it to the other engineer.🤣

@czHappy
Copy link
Author

czHappy commented May 1, 2021

@yarkable I use torch 1.6.0 (CPU) and modify your export script to produce a cpu torchscript model successfully. It proves that the required version (torch >= 1.2.0) is not accurate, I have tried torch 1.2.0, 1.3.1, 1.4.0 but all of them didn't work. Now I find torch 1.6.0 is OK. Then I use C++ to load the pt file and test forward function, it really works! Anyway, thanks for your excellent job and patient reply! The details are as follows:

CMakeLists.txt

cmake_minimum_required(VERSION 3.0 FATAL_ERROR)
project(example-app)

find_package(Torch REQUIRED)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${TORCH_CXX_FLAGS}")

add_executable(example-app example-app.cpp)
target_link_libraries(example-app "${TORCH_LIBRARIES}")
set_property(TARGET example-app PROPERTY CXX_STANDARD 14)

# The following code block is suggested to be used on Windows.
# According to https://github.com/pytorch/pytorch/issues/25457,
# the DLLs need to be copied to avoid memory errors.
if (MSVC)
  file(GLOB TORCH_DLLS "${TORCH_INSTALL_PREFIX}/lib/*.dll")
  add_custom_command(TARGET example-app
                     POST_BUILD
                     COMMAND ${CMAKE_COMMAND} -E copy_if_different
                     ${TORCH_DLLS}
                     $<TARGET_FILE_DIR:example-app>)
endif (MSVC)

example-app.cpp

#include <torch/script.h> // One-stop header.
#include <vector>
#include <iostream>
#include <memory>
int main(int argc, const char* argv[]) {
  if (argc != 2) {
    std::cerr << "usage: example-app <path-to-exported-script-module>\n";
    return -1;
  }

  torch::jit::script::Module module;
  try {
    // Deserialize the ScriptModule from a file using torch::jit::load().
    module = torch::jit::load(argv[1]);
  }
  catch (const c10::Error& e) {
    std::cerr << "error loading the model\n";
    return -1;
  }

  std::cout << "ok\n";
  
  // Create a vector of inputs.
std::vector<torch::jit::IValue> inputs;
inputs.push_back(torch::ones({1, 3, 640, 480}));

// Execute the model and turn its output into a tensor.
at::Tensor output = module.forward(inputs).toTensor();
//std::cout << output.slice(/*dim=*/1, /*start=*/0, /*end=*/5) << '\n';
std::cout<<output[0][0][0][0]<<std::endl;
}

Just like Minimal Example of official document, https://pytorch.org/cppdocs/installing.html
my environment:
ubuntu 18.04 LTS
GCC 7.5.0
cmake 3.10.2
torch 1.6.0 CPU
libtorch 1.6.0 CPU
I think GPU torch version is OK, too.

@yarkable
Copy link
Contributor

yarkable commented May 1, 2021

Good job

@ZHKKKe ZHKKKe closed this as completed Jun 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants