Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to load model on mobile for device type "c10::DeviceType::CUDA" #28379

Closed
ljk53 opened this issue Oct 21, 2019 · 4 comments
Assignees
Labels

Comments

@ljk53
Copy link
Contributor

@ljk53 ljk53 commented Oct 21, 2019

Failed to load model on mobile

Reported on forum: https://discuss.pytorch.org/t/anyone-tried-out-the-pytorch-mobile-demo-apps-yet/58336/12

Looks like it's trying to create device_="c10::DeviceType::CUDA" empty tensor while loading the model, while sample mobile model has device="c10::DeviceType::CPU".

To Reproduce

scripts/build_mobile.sh -DBUILD_BINARY=ON -DCMAKE_BUILD_TYPE=Debug
  • Run speed_benchmark_torch to load the model in gdb:
gdb build_mobile/bin/speed_benchmark_torch

(gdb) r build_mobile/bin/speed_benchmark_torch --model=mobilenet-model.pt --input_dims="1,3,224,224" --input_type=float --iter=1
...
Thread 1 "speed_benchmark" received signal SIGABRT, Aborted.
0x00007ffff6bfd207 in raise () from /lib64/libc.so.6
(gdb) bt
...
#5  0x00007ffff72f4993 in __cxa_throw () from /lib64/libstdc++.so.6
#6  0x00000000008d616c in at::empty (size=..., options=..., memory_format=...) at aten/src/ATen/Functions.h:3649
#7  0x00000000008d6546 in at::native::to_impl (self=..., options=..., non_blocking=false, copy=false, optional_memory_format=...) at ../aten/src/ATen/native/TensorConversions.cpp:42
#8  0x00000000008d6b4c in at::native::to (self=..., device=..., dtype=c10::ScalarType::Float, non_blocking=false, copy=false, optional_memory_format=...) at ../aten/src/ATen/native/TensorConversions.cpp:80
#9  0x0000000000b40491 in at::TypeDefault::to (self=..., device=..., dtype=c10::ScalarType::Float, non_blocking=false, copy=false, memory_format=...) at aten/src/ATen/TypeDefault.cpp:4886
#10 0x00000000005c5b7e in at::Tensor::to (this=0x7fffffffc418, device=..., dtype=c10::ScalarType::Float, non_blocking=false, copy=false, memory_format=...) at aten/src/ATen/core/TensorMethods.h:3412
#11 0x00000000016d31b3 in torch::jit::Unpickler::readInstruction (this=0x7fffffffccd0) at ../torch/csrc/jit/unpickler.cpp:585
...

(gdb) p options                                                                                                                                                                                                                                                                                       
$2 = (const c10::TensorOptions &) @0x7fffffffc2d0: {dtype_ = {data_ = 0x27f7260 <caffe2::detail::_typeMetaDataInstance_6>}, device_ = {type_ = c10::DeviceType::CUDA, index_ = 0}, layout_ = c10::Layout::Strided, requires_grad_ = false, is_variable_ = false, pinned_memory_ = false, 
  has_device_ = true, has_dtype_ = true, has_layout_ = true, has_requires_grad_ = false, has_is_variable_ = true, has_pinned_memory_ = false}

  • Compare with a sample model that can load on mobile:
(gdb) b aten/src/ATen/Functions.h:3641
Breakpoint 1 at 0x595b3c: aten/src/ATen/Functions.h:3641. (79 locations)
(gdb) r build_mobile/bin/speed_benchmark_torch --model=mobilenetv2.pk --input_dims="1,3,224,224" --input_type=float --iter=1
...
Breakpoint 1, at::empty (size=..., options=..., memory_format=...) at aten/src/ATen/Functions.h:3641
3641        switch(tensorTypeIdToBackend(impl::dispatchTypeId(at::detail::multi_dispatch_tensor_type_set(options)))) {
(gdb) p options                                                                                                                                                                                                                                                                                       
$3 = (const c10::TensorOptions &) @0x7fffffffd290: {dtype_ = {data_ = 0x27f7260 <caffe2::detail::_typeMetaDataInstance_6>}, device_ = {type_ = c10::DeviceType::CPU, index_ = -1}, layout_ = c10::Layout::Strided, requires_grad_ = false, is_variable_ = false, pinned_memory_ = false, 
  has_device_ = false, has_dtype_ = true, has_layout_ = false, has_requires_grad_ = false, has_is_variable_ = true, has_pinned_memory_ = false}
@lkhphuc

This comment has been minimized.

Copy link
Contributor

@lkhphuc lkhphuc commented Oct 21, 2019

Thanks for responding so quickly.

@ljk53 ljk53 changed the title Failed to load model on mobile Failed to load model on mobile for device type "c10::DeviceType::CUDA" Oct 22, 2019
@ljk53

This comment has been minimized.

Copy link
Contributor Author

@ljk53 ljk53 commented Oct 22, 2019

Looks similar to issue #22395? I was thinking of adding "map_location='cpu'" for mobile but seems it didn't really work and the recommendation there was to "switch it to CPU mode before tracing it and exporting it". Not sure if things have changed since then cc: @zdevito

@ljk53

This comment has been minimized.

Copy link
Contributor Author

@ljk53 ljk53 commented Oct 22, 2019

@lkhphuc do you want to try calling model.cpu() before tracing and exporting your gpu-trained model?

@lkhphuc

This comment has been minimized.

Copy link
Contributor

@lkhphuc lkhphuc commented Nov 3, 2019

@ljk53 sorry late response.
I confirmed the model converted to cpu() is able to load on the phone.

@ljk53 ljk53 closed this Nov 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.