Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

import error: from torch._C import * RuntimeError: stoi #7082

Closed
aespielberg opened this issue Apr 30, 2018 · 39 comments
Closed

import error: from torch._C import * RuntimeError: stoi #7082

aespielberg opened this issue Apr 30, 2018 · 39 comments

Comments

@aespielberg
Copy link

@aespielberg aespielberg commented Apr 30, 2018

Hi all,

I just recently installed pytorch. I'm using 3.5 but also tried it with 3.6 import torch gives me the following error:

>>> import torch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/aespielberg/anaconda3/lib/python3.5/site-packages/torch/__init__.py", line 78, in <module>
    from torch._C import *
RuntimeError: stoi

I've tried upgrading conda, numpy, and pytorch.

  • Pytorch
  • conda
  • PyTorch version:
  • Python version: Python 3.5.5 :: Anaconda custom (64-bit)
  • CUDA/cuDNN version: 8.0
@mitar

This comment has been minimized.

Copy link

@mitar mitar commented Apr 30, 2018

Same here. What is your OS? glibc version?

@stomachacheGE

This comment has been minimized.

Copy link

@stomachacheGE stomachacheGE commented Jun 1, 2018

I encountered the same error when my Matlab script runs python to import torch. However, I can import torch successfully when directly open a python console from terminal. Any suggestion is welcomed.

@mitar

This comment has been minimized.

Copy link

@mitar mitar commented Jun 1, 2018

In our case it was also connected to Matlab. It is the question of import order. If you load Matlab first in your Python script, then Torch 0.4.0 does not work. Other way around it seems to both work.

Torch 0.3.* continues to work though. So there is some difference between 0.4.0 and older versions.

@soumith

This comment has been minimized.

Copy link
Member

@soumith soumith commented Jun 1, 2018

matlab ships with it's own libstdc++ that is too old, i think. And that becomes an issue.

@mitar

This comment has been minimized.

Copy link

@mitar mitar commented Jun 1, 2018

Not really, there is a new release of Matlab having pretty recent version. Also, I removed their old libstdc++ so files and forced loading of system so files and it was still the same thing.

@mitar

This comment has been minimized.

Copy link

@mitar mitar commented Jun 1, 2018

My feeling (based on stack traces) is that it is something about threading. That or Matlab or Torch do not expect threading to be active when being imported, and one or the other does something with threads.

Also, because the issue does not happen 100% times, but very often. Sometimes it does load.

@stomachacheGE

This comment has been minimized.

Copy link

@stomachacheGE stomachacheGE commented Jun 4, 2018

@mitar Thanks for the updates. However, in my case, I need to run python scripts using Matlab "system()" command. Therefore I cannot change the import order of Matlab and torch. Secondly, since I need some features of torch 0.4.0, it is not feasible for me to downgrade to lower versions. The problem for me is still not solved...

@mitar

This comment has been minimized.

Copy link

@mitar mitar commented Jun 4, 2018

Hm, if you are using system() command, then those are two separate processes, no? Then it is maybe really something about libraries and not threading.

@stomachacheGE

This comment has been minimized.

Copy link

@stomachacheGE stomachacheGE commented Jun 5, 2018

@mitar I think you are right. As a minimal example, if I run python -c "import torch" from terminal, it works perfectly. However, if I run system('python -c "import torch"') from Matlab console, the error occurs. I checked the PATH and LD_LIBRARY_PATH for both cases, and they are the same.

@he010103

This comment has been minimized.

Copy link

@he010103 he010103 commented Jun 26, 2018

@stomachacheGE @mitar I read your dicussion carefully. Do you solve the problem by running system('python -c "import torch"') with error?

@mitar

This comment has been minimized.

Copy link

@mitar mitar commented Jun 26, 2018

I could not solve the problem. I am staying at 0.3.1 for now.

@he010103

This comment has been minimized.

Copy link

@he010103 he010103 commented Jun 27, 2018

@mitar ok, but pytorch0.4.0 doesnet have fft.

@SsnL

This comment has been minimized.

Copy link
Collaborator

@SsnL SsnL commented Jul 2, 2018

@he010103 0.4 definitely has fft.

@zou3519

This comment has been minimized.

Copy link
Contributor

@zou3519 zou3519 commented Jul 2, 2018

Can any of you reproduce this in a docker image? That would really help with debugging.

@zou3519 zou3519 added the todo label Jul 2, 2018
@mitar

This comment has been minimized.

Copy link

@mitar mitar commented Jul 2, 2018

I use something like this to install Matlab into a Docker container:

RUN cd /usr/src && \
 wget --no-verbose http://ssd.mathworks.com/supportfiles/downloads/R2017b/deployment_files/R2017b/installers/glnxa64/MCR_R2017b_glnxa64_installer.zip && \
 unzip MCR_R2017b_glnxa64_installer.zip && \
 mkdir -p /opt/mcr && ./install -mode silent -agreeToLicense yes -destinationFolder /opt/mcr && \
 echo /opt/mcr/v93/runtime/glnxa64 > /etc/ld.so.conf.d/zzz-matlab.conf && \
 echo /opt/mcr/v93/bin/glnxa64 >> /etc/ld.so.conf.d/zzz-matlab.conf && \
 echo /opt/mcr/v93/sys/os/glnxa64 >> /etc/ld.so.conf.d/zzz-matlab.conf && \
 ldconfig && \
 cd / && rm -rf /usr/src/*

This is an older version, but new has also the same issues as mentioned in this thread.

So then just install pytorch and then import matlab first before torch and it will fail. Importing matlab is a bit tricky, I am seeing now in the code, I will see if I can extract it later. But it is generally importing the /opt/mcr/v93/bin/glnxa64/matlabruntimeforpython3_6.so file.

@SsnL

This comment has been minimized.

Copy link
Collaborator

@SsnL SsnL commented Jul 2, 2018

If matlab has a libstdc++ that is too old, can we do anything? @soumith

@mitar

This comment has been minimized.

Copy link

@mitar mitar commented Jul 2, 2018

From my testing I am not sure if it is the the question of old libstdc++, even if I force loading of system-wide libstdc++, error still happens.

My feeling is that it is something about threading. Does PyTorch uses threads?

@soumith

This comment has been minimized.

Copy link
Member

@soumith soumith commented Jul 27, 2018

Yes, PyTorch uses threads in general, via OpenMP, MKL and also has it's own threads for a few things like autograd

@josephnunn

This comment has been minimized.

Copy link

@josephnunn josephnunn commented Jul 27, 2018

Hi there, I've got the exact same problem.

from torch._C import * ModuleNotFoundError: No module named 'torch._C'

This from an empty file excepting
import torch

But of course, it works when run from the command line.

I build pyTorch from the github source against cuda 9.2 and intel mkl on Linux. I'm using Eclipse + pyDev as the IDE. Clearly something is getting set wrong in the environment when called one way vs the other but have not identified yet what that is.

@BestSonny

This comment has been minimized.

Copy link
Contributor

@BestSonny BestSonny commented Aug 23, 2018

Is there any progress on this issue?

@aespielberg

This comment has been minimized.

Copy link
Author

@aespielberg aespielberg commented Aug 23, 2018

This seems to still be an issue in 0.4. I am running Ubuntu 16.04. glibc version:
ldd (Ubuntu GLIBC 2.23-0ubuntu10) 2.23

This happens only when trying to use PyTorch from inside a MATLAB environment by invoking a system call.

@juanitoroter

This comment has been minimized.

Copy link

@juanitoroter juanitoroter commented Aug 30, 2018

Also I reproduce the same error when run system(['source activate Py36Torch; python -c "import torch"']) on Ubuntu 16.04.4 Matlab2016a and Matlab2018a (with and without CUDA8.0). Even if I force to Matab to use my system libraries or my Python environment libraries: LD_PRELOAD=~/.conda/envs/Py36Torch/lib/libstdc++.so.6 /home2/Softwares/MATLAB/R2016b/bin/matlab

It is really a pity if the only solution is just degrading pytorch (it is not possible for me!)

@ezyang

This comment has been minimized.

Copy link
Contributor

@ezyang ezyang commented Aug 30, 2018

Can you run the crashing version of Python under gdb and give us a backtrace?

@SsnL

This comment has been minimized.

Copy link
Collaborator

@SsnL SsnL commented Aug 30, 2018

@juanitoroter Also, what is the stdc++lib version (including minor) your matlab uses? What is the version of ~/.conda/envs/Py36Torch/lib/libstdc++.so.6 ?

@kidanger

This comment has been minimized.

Copy link

@kidanger kidanger commented Oct 22, 2018

I had the same error. I fixed it using: system('env -i /usr/bin/python3 -c "import torch"').
Some environment variable set by matlab is messing with pytorch, but I don't know which.

@yalaudah

This comment has been minimized.

Copy link

@yalaudah yalaudah commented Nov 1, 2018

@kidanger I've had the same issue as others here, and your solution worked. Thanks!!

@skyoung

This comment has been minimized.

Copy link

@skyoung skyoung commented Nov 8, 2018

Seems like autograd.grad does not work well even in Pytorch 0.3.x. Does anyone have any idea?

@hwh3304

This comment has been minimized.

Copy link

@hwh3304 hwh3304 commented Nov 10, 2018

@kidanger Thank you. This works. Just add 'env -i' before the previous command.

@wmasonku

This comment has been minimized.

Copy link

@wmasonku wmasonku commented Dec 27, 2018

Instead of 'env -i' try 'unset MKL_NUM_THREADS' before running. ex: system('unset MKL_NUM_THREADS; /usr/bin/python3 -c "import torch"')

@messcode

This comment has been minimized.

Copy link

@messcode messcode commented Jan 21, 2019

@wmasonku 's solution works two. Many thanks. I am using Matlab 2015b and torch 0.4.1.

@xi-mao

This comment has been minimized.

Copy link

@xi-mao xi-mao commented Mar 7, 2019

Also I reproduce the same error when run system(['source activate Py36Torch; python -c "import torch"']) on Ubuntu 16.04.4 Matlab2016a and Matlab2018a (with and without CUDA8.0). Even if I force to Matab to use my system libraries or my Python environment libraries: LD_PRELOAD=~/.conda/envs/Py36Torch/lib/libstdc++.so.6 /home2/Softwares/MATLAB/R2016b/bin/matlab

It is really a pity if the only solution is just degrading pytorch (it is not possible for me!)

can i ask how to change matlab env PATH?

@xi-mao

This comment has been minimized.

Copy link

@xi-mao xi-mao commented Mar 7, 2019

@juanitoroter Also, what is the stdc++lib version (including minor) your matlab uses? What is the version of ~/.conda/envs/Py36Torch/lib/libstdc++.so.6 ?

did you solved the problem?

@xi-mao

This comment has been minimized.

Copy link

@xi-mao xi-mao commented Mar 7, 2019

Hi all,

I just recently installed pytorch. I'm using 3.5 but also tried it with 3.6 import torch gives me the following error:

>>> import torch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/aespielberg/anaconda3/lib/python3.5/site-packages/torch/__init__.py", line 78, in <module>
    from torch._C import *
RuntimeError: stoi

I've tried upgrading conda, numpy, and pytorch.

  • Pytorch
  • conda
  • PyTorch version:
  • Python version: Python 3.5.5 :: Anaconda custom (64-bit)
  • CUDA/cuDNN version: 8.0

anyone solved the problem?

@LCAR979

This comment has been minimized.

Copy link

@LCAR979 LCAR979 commented Mar 14, 2019

Same problem here.

  • Ubuntu 16.04
  • Matlab R2017a
  • PyTorch 0.4.1 (conda env)
  • anconda2
    Any progress on this issue?
@xi-mao

This comment has been minimized.

Copy link

@xi-mao xi-mao commented Mar 14, 2019

Same problem here.

  • Ubuntu 16.04
  • Matlab R2017a
  • PyTorch 0.4.1 (conda env)
  • anconda2
    Any progress on this issue?

I have solved the problem. you should down grade your pytorch version to 0.3, and use python2. The issue will be solved. Besides I didn't use conda env. I didn't suggest use conda environment。

@LCAR979

This comment has been minimized.

Copy link

@LCAR979 LCAR979 commented Mar 14, 2019

@xi-mao. Thanks, but downgrading is not an option for me since I strongly need the fft module in 0.4.

@bilylee

This comment has been minimized.

Copy link

@bilylee bilylee commented Apr 15, 2019

Thanks to @wmasonku . It works!

By the way, if you are calling a python script from matlab, you can achieve the same effects by putting the following line before importing torch.

# in a python script
del os.environ['MKL_NUM_THREADS']
import torch
@colesbury

This comment has been minimized.

Copy link
Member

@colesbury colesbury commented Apr 15, 2019

This looks like an issue with the code here:

if (const char *env_p = std::getenv("MKL_NUM_THREADS")) {
at::set_num_threads(std::stoi(env_p));
}

We should fix this before the next release

@gchanan

This comment has been minimized.

Copy link
Contributor

@gchanan gchanan commented May 9, 2019

I believe this should be fixed after #19011 -- please reopen if that is not the case.

@gchanan gchanan closed this May 9, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.