Cannot find/open tsne_multicore shared library #32

SumNeuron · 2017-11-08T09:46:40Z

I installed this.
It works inside the directory.
When I move it too dist-packages, it breaks.
It is on sys.path

guenteru · 2017-11-12T15:46:56Z

I just tried to replicate this and it works fine for me (python 3.6.2).
Which python version are you running?

SumNeuron · 2017-11-12T17:41:36Z

3.6

On Nov 12, 2017, at 4:46 PM, guenteru ***@***.***> wrote: I just tried to replicate this and it works fine for me (python 3.6.2). Which python version are you running? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#32 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AVzyadLeNCHVr7fo5wI7GobHEjFZbsoKks5s1xLxgaJpZM4QWGoT>.

guenteru · 2017-11-12T20:03:10Z

which command did you use to install?

SumNeuron · 2017-11-14T08:04:35Z

git clone https://github.com/DmitryUlyanov/Multicore-TSNE.git
cd Multicore-TSNE/
pip install --no-cache-dir .

guenteru · 2017-11-14T10:25:07Z

try without --no-cache-dir

SumNeuron · 2017-11-18T13:42:55Z

I did, then it fails

…

On Nov 14, 2017, at 11:25 AM, guenteru ***@***.***> wrote: try without --no-cache-dir — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#32 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AVzyafNq2YCIuROE7WXfAjAT_Yi2YJZZks5s2WqDgaJpZM4QWGoT>.

AndrewCurran · 2017-11-21T13:12:47Z

I am having the same problem on 3.6.2.

heurainbow · 2017-11-23T09:11:55Z

same problem

overdrivek · 2017-12-07T14:40:56Z

any updates regarding this? i have the same issue.

DmitryUlyanov · 2017-12-10T12:45:33Z

Please try to install with python setup.py install

akupemula · 2018-01-14T09:29:19Z

I solved the problem by compiling the shared .so file manually using g++

mpenagar · 2018-01-23T14:14:31Z

I got the same problem with python 3.4 when running the example inside the installation directory "Multicore-TSNE"

Worked if I run python outside the installation directory.

LukasMosser · 2018-02-01T18:52:19Z

Python 2.7
Compiled using windows build tools 2017
Installed via pip install. and python setup.py install
Neither work.
Any idea why?

arianhosseini · 2018-02-26T17:50:03Z

I had the same problem. This is how I fixed it:

go to the parent directory
run pip install Multicore-TSNE/
then the egg is added to /usr/local/lib/python2.7/dist-packages/. Same can be done on python3.x with pip3.

darthdeus · 2018-03-15T19:09:58Z

Tried both python setup.py install, running pip install . and running pip install Multicore-TSNE from the parent dir, neither works on Windows 10 with 2017 devtools.

abrahamnunes · 2018-04-04T22:05:01Z

I was experiencing this on one of my machines and so I just updated numpy in my Anaconda distribution. That seemed to fix it.

jroakes · 2018-04-22T14:29:05Z

The problem I am having is that the file tsne_multicore.dll has been compiled with i386 architecture (32-bit) so it will not work on 64-bit systems, thus the "error 0xc1". I am not sure if this is the correct answer for everyone, but I was able to add:

    if 0 != execute(['cmake',
		 '-GMinGW Makefiles',  <----------This
                     '-DCMAKE_BUILD_TYPE={}'.format(build_type),
                     '-DCMAKE_VERBOSE_MAKEFILE={}'.format(int(self.verbose)),

to the setup.py file to get it to compile with MinGW for WIndows which works and produces the right files for the architecture.

haltuf · 2018-06-08T08:46:52Z

I have same problem on Win 10, tried to install using all methods mentioned.

RichardMeyes · 2018-07-11T18:08:23Z

@jroakes THX! That worked perfectly!

fingoldo · 2018-07-13T16:31:58Z

@DmitryUlyanov could you kindly take a look, there is still a problem on Windows...
UPD
After server reboot somehow it started working.. )

adrisede · 2018-07-31T18:06:21Z

Hi, I also have the same problem on Win 10, and also tried with all the methods mentioned and couldn't make it to work.

RebekkaWegmann · 2018-08-08T12:01:02Z

I also have the same issue on Win 10.

IrisStark · 2018-08-09T04:46:41Z

Same issue on Win 10. Tried everything above.

rajbhatti · 2018-08-29T16:16:09Z

Did anyone find any solution? I am getting the same error

worm6206 · 2018-09-19T04:15:57Z

If you are on Windows,

Install MinGW 64. I had it already, but you can install from http://mingw-w64.org/doku.php.
Then add MinGW flag as Cannot find/open tsne_multicore shared library #32 (comment) describes.
Next, build with python setup.py install
and install with pip install .

bscully27 · 2018-10-17T01:38:37Z

I spent an hour trying different installs with cmd, mingw, pip, setup.py. I'm using python 2.7.10 on windows7. Nothing worked.

I then tried on python3.6, worked immediately.

mosynaq · 2018-11-04T12:52:55Z

I built the lib using MSVS 2017 and it worked. I can share the file. But there is an easier way: use pypi wheel package (pip regular installation process)! It simply works!

eardic · 2019-01-09T20:42:08Z

I added '-DCMAKE_GENERATOR_PLATFORM=x64' flag to the setup.py and the problem solved in Win64 (Win10) and Python 3.5.4.

if 0 != execute(['cmake',
'-DCMAKE_GENERATOR_PLATFORM=x64', <----- THIS
'-DCMAKE_BUILD_TYPE={}'.format(build_type),
'-DCMAKE_VERBOSE_MAKEFILE={}'.format(int(self.verbose)),

ZippoML · 2019-01-11T10:10:08Z

I added '-DCMAKE_GENERATOR_PLATFORM=x64' flag to the setup.py and the problem solved in Win64 (Win10) and Python 3.5.4.

if 0 != execute(['cmake',
'-DCMAKE_GENERATOR_PLATFORM=x64', <----- THIS
'-DCMAKE_BUILD_TYPE={}'.format(build_type),
'-DCMAKE_VERBOSE_MAKEFILE={}'.format(int(self.verbose)),

thx, solved my problem. win 10

Po-Shin · 2019-02-10T12:17:54Z

Hi, I used pip install MulticoreTSNE and got the same error while running my code. Anyone got the solution to fix it?

thanks!

ghayth82 · 2019-02-15T12:41:20Z

I added '-DCMAKE_GENERATOR_PLATFORM=x64' flag to the setup.py and the problem solved in Win64 (Win10) and Python 3.5.4.
if 0 != execute(['cmake',
'-DCMAKE_GENERATOR_PLATFORM=x64', <----- THIS
'-DCMAKE_BUILD_TYPE={}'.format(build_type),
'-DCMAKE_VERBOSE_MAKEFILE={}'.format(int(self.verbose)),

thx, solved my problem. win 10

where I can find the setup.py file, please?

ghayth82 · 2019-02-15T14:24:41Z

If you are on Windows,

Install MinGW 64. I had it already, but you can install from http://mingw-w64.org/doku.php.

Then add MinGW flag as #32 (comment) describes.

Next, build with python setup.py install

and install with pip install .

hi
can you help me , what do you mean by "build with python setup.py install" then install with pip install .` ?
can you provide me with full commands please

ZippoML · 2019-02-19T08:28:18Z

If you are on Windows,

Install MinGW 64. I had it already, but you can install from http://mingw-w64.org/doku.php.

Then add MinGW flag as #32 (comment) describes.

Next, build with python setup.py install

and install with pip install .

hi
can you help me , what do you mean by "build with python setup.py install" then install with pip install .` ?
can you provide me with full commands please

I alrdy forgot what i did to solve the problem. But i think the steps are:

download package from here https://github.com/DmitryUlyanov/Multicore-TSNE
open setup.py install with notepad or jupyter
change the code as was written above
install the pakage by command in cmd. python setup.py install

arikanev · 2019-02-21T21:03:19Z

Also having this issue on OSX, @akupemula, can you expand on how to compile the .so with g++ manually? Thanks!

ntr34g · 2019-03-27T07:38:31Z

i've done everything mentioned here #32 (comment)
compiled with cmake x64 and mingw-w64\x86_64-8.1.0-posix-sjlj-rt_v6-rev0

but still getting
"RuntimeError: Cannot find/open tsne_multicore shared library"

bscully27 · 2019-03-27T13:02:10Z

I used to have a working multicoreTSNE on python3.6 in windows7 but uninstalled then tried re-installing, same problem "RuntimeError: Cannot find/open tsne_multicore shared library". tried both pip and clone installs.

…

On Wed, Mar 27, 2019 at 3:38 AM P.S. ***@***.***> wrote: i've done everything mentioned here #32 (comment) <#32 (comment)> compiled withcmake x64 and mingw-w64\x86_64-8.1.0-posix-sjlj-rt_v6-rev0 but still getting "RuntimeError: Cannot find/open tsne_multicore shared library" — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#32 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ApcfAVaaYWKDCLpxmg45CVbaj2CnaclFks5vax_5gaJpZM4QWGoT> .

ntr34g · 2019-03-31T12:16:49Z

I added '-DCMAKE_GENERATOR_PLATFORM=x64' flag to the setup.py and the problem solved in Win64 (Win10) and Python 3.5.4.

if 0 != execute(['cmake',
'-DCMAKE_GENERATOR_PLATFORM=x64', <----- THIS
'-DCMAKE_BUILD_TYPE={}'.format(build_type),
'-DCMAKE_VERBOSE_MAKEFILE={}'.format(int(self.verbose)),

this helped me on win 10 x64 python 3.6.8

Applenice · 2019-04-15T07:17:39Z

I added '-DCMAKE_GENERATOR_PLATFORM=x64' flag to the setup.py and the problem solved in Win64 (Win10) and Python 3.5.4.

if 0 != execute(['cmake',
'-DCMAKE_GENERATOR_PLATFORM=x64', <----- THIS
'-DCMAKE_BUILD_TYPE={}'.format(build_type),
'-DCMAKE_VERBOSE_MAKEFILE={}'.format(int(self.verbose)),

Thank you very much, this solved my problem，this helped me on windows 10 x64 python 3.6.5

mukesh5 · 2019-04-21T12:05:01Z

I added '-DCMAKE_GENERATOR_PLATFORM=x64' flag to the setup.py and the problem solved in Win64 (Win10) and Python 3.5.4.

if 0 != execute(['cmake',
'-DCMAKE_GENERATOR_PLATFORM=x64', <----- THIS
'-DCMAKE_BUILD_TYPE={}'.format(build_type),
'-DCMAKE_VERBOSE_MAKEFILE={}'.format(int(self.verbose)),

Thanx this solved my problem on win10 & python 3.5.4

jonimatix · 2019-04-21T17:53:07Z

I added '-DCMAKE_GENERATOR_PLATFORM=x64' flag to the setup.py and the problem solved in Win64 (Win10) and Python 3.5.4.
if 0 != execute(['cmake',
'-DCMAKE_GENERATOR_PLATFORM=x64', <----- THIS
'-DCMAKE_BUILD_TYPE={}'.format(build_type),
'-DCMAKE_VERBOSE_MAKEFILE={}'.format(int(self.verbose)),

Thanx this solved my problem on win10 & python 3.5.4

This works for me as well. If you had already installed the package, just type pip uninstall MulticoreTSNE and then reinstall it..

michaelthwan · 2019-05-10T08:01:12Z

The above solution works for me. Let me summarize the solution:

If you installed package before, pip uninstall MulticoreTSNE
Clone the source git clone https://github.com/DmitryUlyanov/Multicore-TSNE.git
Edit /Multicore-TSNE/setup.py,

Find these piece of code and add one line

if 0 != execute(['cmake',
'-DCMAKE_GENERATOR_PLATFORM=x64', <----- THIS
'-DCMAKE_BUILD_TYPE={}'.format(build_type),
'-DCMAKE_VERBOSE_MAKEFILE={}'.format(int(self.verbose)),

install by source

cd /Multicore-TSNE
pip install .

xDarkLemon · 2020-05-13T06:33:09Z

Does anyone know hot to fix it on Mac OS?

zfhxi · 2021-09-26T08:44:44Z

Problem：

demo：

from sklearn.datasets import load_digits
from MulticoreTSNE import MulticoreTSNE as TSNE
from matplotlib import pyplot as plt

digits = load_digits()
embeddings = TSNE(n_jobs=4).fit_transform(digits.data)
vis_x = embeddings[:, 0]
vis_y = embeddings[:, 1]
plt.scatter(vis_x, vis_y, c=digits.target, cmap=plt.cm.get_cmap("jet", 10), marker='.')
plt.colorbar(ticks=range(10))
plt.clim(-0.5, 9.5)
plt.show()

If you run this demo, the error may occurr：

Traceback (most recent call last):
  File "/xxxx/miniconda3/xxx/test_MultiCore_TSNE.py", line 10, in <module>
    embeddings = TSNE(n_jobs=4).fit_transform(digits.data)
  File "/xxxx/miniconda3/xxx/site-packages/MulticoreTSNE/__init__.py", line 103, in __init__
    raise RuntimeError('Cannot find/open tsne_multicore shared library')
RuntimeError: Cannot find/open tsne_multicore shared library

Explore：

Open /xxx//miniconda3/xxx/site-packages/MulticoreTSNE/__init__.py to check what makes it happen，and locate to around line 103：

path = os.path.dirname(os.path.realpath(__file__))
try:
    sofile = (glob(os.path.join(path, "libtsne*.so")) + glob(os.path.join(path, "*tsne*.dll")))[0]
    self.C = self.ffi.dlopen(os.path.join(path, sofile))
except (IndexError, OSError):
    raise RuntimeError('Cannot find/open tsne_multicore shared library')

To get more concrete information about this error, we can modified this source code as follow：

path = os.path.dirname(os.path.realpath(__file__))
try:
    sofile = (glob(os.path.join(path, "libtsne*.so")) + glob(os.path.join(path, "*tsne*.dll")))[0]
    self.C = self.ffi.dlopen(os.path.join(path, sofile))
# except (IndexError, OSError):
# raise RuntimeError('Cannot find/open tsne_multicore shared library')

except IndexError as e:
    print(e)
    exit(-1)

except OSError as e1:
    print(e1)
    exit(-1)

Then run the demo again, errors may occurr as follow：

cannot load library '/xxxx/miniconda3/xxx/site-packages/MulticoreTSNE/libtsne_multicore.so':
 /xxxx/miniconda3/xxx/site-packages/scipy/_lib/_uarray/../../../../../libstdc++.so.6: version `GLIBCXX_3.4.29' not found
 (required by /xxxx/miniconda3/xxx/site-packages/MulticoreTSNE/libtsne_multicore.so)

From this information, we can see that there's no GLIBCXX_3.4.29 in libstdc++.so.6. Test it as follow:

$ real path /xxxxx/miniconda3/xxx/scipy/_lib/_uarray/../../../../../libstdc++.so.6
$ /xxxx/miniconda3/xxx/lib/libstdc++.so.6.0.26
$ strings /xxxx/miniconda3/xxx/lib/libstdc++.so.6.0.26  | grep GLIBCXX_3.4.29
$ # return nothing

Solution

Reference：https://blog.csdn.net/qq_39295044/article/details/86685729

We can find /usr/lib/libstdc++.so.6.0.29 contains GLIBCXX_3.4.29, test it:

$ strings /usr/lib/libstdc++.so.6 | grep GLIBCXX_3.4.29
GLIBCXX_3.4.29
GLIBCXX_3.4.29

This lib file satisfies the requirement, so we can replace /xxxx/miniconda3/xxx/lib/libstdc++.so.6.0.26 with /usr/lib/libstdc++.so.6.0.29:

$ mv /xxxx/miniconda3/xxx/lib/libstdc++.so.6.0.26 /xxxx/miniconda3/xxx/lib/libstdc++.so.6.0.26_bak
$ ln -s /usr/lib/libstdc++.so.6.0.29 /xxxx/miniconda3/xxx/lib/libstdc++.so.6.0.26

Run the demo again, good lucky!

shayantabatabaee · 2023-06-28T09:44:13Z

I also encountered a similar issue, but I managed to resolve it by following these steps:

First, I downloaded the Mingw64 compiler for Windows from the official website: https://winlibs.com/#download-release.
Once the download was complete, I installed the Mingw64 compiler on my system.
After installing Mingw64, I changed the following code to build the project:

if 0 != execute(['cmake',  
                         '-G', 'MinGW Makefiles',  <----- This Line
                         '-DCMAKE_BUILD_TYPE={}'.format(build_type),

I compiled the project using the Mingw64 compiler, and fortunately, it worked perfectly fine.

By following these steps and utilizing the Mingw64 compiler, I was able to overcome the problem.

MulticoreTSNE-0.1-cp37-cp37m-win_amd64.zip

zdwhite mentioned this issue Jun 24, 2019

MulticoreTSNE on python 3.7.3 using conda #67

Open

Cannot find/open tsne_multicore shared library #32

Cannot find/open tsne_multicore shared library #32

Comments

SumNeuron commented Nov 8, 2017

guenteru commented Nov 12, 2017

SumNeuron commented Nov 12, 2017 via email

guenteru commented Nov 12, 2017

SumNeuron commented Nov 14, 2017

guenteru commented Nov 14, 2017

SumNeuron commented Nov 18, 2017 via email

AndrewCurran commented Nov 21, 2017

heurainbow commented Nov 23, 2017

overdrivek commented Dec 7, 2017

DmitryUlyanov commented Dec 10, 2017

akupemula commented Jan 14, 2018

mpenagar commented Jan 23, 2018

LukasMosser commented Feb 1, 2018

arianhosseini commented Feb 26, 2018

darthdeus commented Mar 15, 2018

abrahamnunes commented Apr 4, 2018

jroakes commented Apr 22, 2018 • edited

haltuf commented Jun 8, 2018

RichardMeyes commented Jul 11, 2018

fingoldo commented Jul 13, 2018 • edited

adrisede commented Jul 31, 2018

RebekkaWegmann commented Aug 8, 2018

IrisStark commented Aug 9, 2018

rajbhatti commented Aug 29, 2018 • edited

worm6206 commented Sep 19, 2018

bscully27 commented Oct 17, 2018

mosynaq commented Nov 4, 2018 • edited

eardic commented Jan 9, 2019 • edited

ZippoML commented Jan 11, 2019

Po-Shin commented Feb 10, 2019

ghayth82 commented Feb 15, 2019

ghayth82 commented Feb 15, 2019

ZippoML commented Feb 19, 2019

arikanev commented Feb 21, 2019

ntr34g commented Mar 27, 2019 • edited

bscully27 commented Mar 27, 2019 via email

ntr34g commented Mar 31, 2019

Applenice commented Apr 15, 2019

mukesh5 commented Apr 21, 2019

jonimatix commented Apr 21, 2019

michaelthwan commented May 10, 2019

xDarkLemon commented May 13, 2020

zfhxi commented Sep 26, 2021 • edited

Problem：

Explore：

Solution

shayantabatabaee commented Jun 28, 2023 • edited

jroakes commented Apr 22, 2018 •

edited

fingoldo commented Jul 13, 2018 •

edited

rajbhatti commented Aug 29, 2018 •

edited

mosynaq commented Nov 4, 2018 •

edited

eardic commented Jan 9, 2019 •

edited

ntr34g commented Mar 27, 2019 •

edited

zfhxi commented Sep 26, 2021 •

edited

shayantabatabaee commented Jun 28, 2023 •

edited