Installation Question #32
Labels
Comments
|
Hi Sandeep, Just so I'm clear here, the steps involved in this issue are:
Is that right? If so, I think the issue is with paths / Conda. I don't know that anyone's ever tested First things to check:
For example, on my MinGW installed Conda env, the command
I think the issue here is that subprocess doesn't inherit the conda path when looking for gcc (but it would, once it actually launches gcc). For instance, yields: So the quick fix is to put I'll investigate a longer-term fix |
|
Nick,
Spot on with everything. This is exactly my scenario.
Sounds like the quick fix will work since you show that python can find the
path. I'll update again when this works.
Thanks again for your help,
Sandeep.
…On Fri, May 25, 2018 at 10:35 AM, arghdos ***@***.***> wrote:
Hi Sandeep,
Just so I'm clear here, the steps involved in this issue are:
1. Install pyJac on windows
2. Generate the c-code
3. use pyjac.pywrap to generate a python wrapper for the generated
C-code
Is that right?
If so, I think the issue is with paths / Conda. I don't know that anyone's
ever tested pyJac with MinGW, but this is something we should support.
First things to check:
1. Can you call gcc via the Anaconda prompt (that you run python from)?
For example, on my MinGW installed Conda env, the command gcc -v yields:
gcc -v
Using built-in specs.
COLLECT_GCC=C:\ProgramData\Miniconda3\Scripts\gcc.bat\..\..\MinGW\bin\gcc.exe
COLLECT_LTO_WRAPPER=c:/programdata/miniconda3/mingw/bin/../libexec/gcc/x86_64-w64-mingw32/4.7.0/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../../../build/gcc/src/configure --target=x86_64-w64-mingw32 --prefix=/c/bb/vista64-mingw32/mingw-x86-x86_64/build/build/root --with-sysroot=/c/bb/vista64-mingw32/mingw-x86-x86_64/build/build/root --enable-languages=all,obj-c++ --enable-fully-dynamic-string --disable-multilib
Thread model: win32
gcc version 4.7.0 20111220 (experimental) (GCC)
1. However, when I call gcc within python:
python
>>>import subprocess
>>>subprocess.call(['gcc', '-v'])
>>> import sys
>>> import subprocess
>>> subprocess.call(['gcc', '-v'])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\ProgramData\Miniconda3\lib\subprocess.py", line 267, in call
with Popen(*popenargs, **kwargs) as p:
File "C:\ProgramData\Miniconda3\lib\subprocess.py", line 707, in __init__
restore_signals, start_new_session)
File "C:\ProgramData\Miniconda3\lib\subprocess.py", line 992, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
I think the issue here is that subprocess doesn't inherit the conda path
<https://bugs.python.org/msg166473> when *looking* for gcc (but it would,
once it actually launches gcc).
For instance,
subprocess.call(['gcc', '-v'], env=os.environ, shell=True)
yields:
COLLECT_GCC=C:\ProgramData\Miniconda3\Scripts\gcc.bat\..\..\MinGW\bin\gcc.exe
COLLECT_LTO_WRAPPER=c:/programdata/miniconda3/mingw/bin/../libexec/gcc/x86_64-w64-mingw32/4.7.0/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../../../build/gcc/src/configure --target=x86_64-w64-mingw32 --prefix=/c/bb/vista64-mingw32/mingw-x86-x86_64/build/build/root --with-sysroot=/c/bb/vista64-mingw32/mingw-x86-x86_64/build/build/root --enable-languages=all,obj-c++ --enable-fully-dynamic-string --disable-multilib
Thread model: win32
gcc version 4.7.0 20111220 (experimental) (GCC)
So the quick fix is to put shell=True in the subprocess call on lines 111
& 207 in libgen.py
<https://github.com/SLACKHA/pyJac/tree/master/pyjac/libgen>.
In order to do so, you should uninstall pyJac via conda, clone this repo,
make these changes and then install your modified version of pyJac via pip
I'll investigate a longer-term fix
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#32 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/Alvbc3ra-Ygeh15x2uRfzh1lbLefwB-Sks5t2Ba0gaJpZM4UNOVq>
.
|
|
Sandeep, I'm still running into some issues getting the python wrapper to fully compile here on windows (you have to do similar things to the linker and there's some Cython weirdness), but I should have a pull request for you to test later this afternoon |
|
OK, thanks for that. I'll wait to hear from you. GCC is now found by pyjac
and some object files are created but like you mention, there is some
weirdness happening.
Not sure if its of use but here's a screenshot of my output. Some crazy
path mangling is evident (Line 11)
Sandeep.
…On Fri, May 25, 2018 at 11:54 AM, arghdos ***@***.***> wrote:
Sandeep, I'm still running into some issues getting the python wrapper to
fully compile here on windows (you have to do similar things to the linker
and there's some Cython weirdness), but I should have a pull request for
you to test later this afternoon
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#32 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/Alvbc_6Spxiw7fT0TVqhXd-Gv8irv5sCks5t2ClIgaJpZM4UNOVq>
.
|
|
Ok, so this is actually a bit more complicated than I hoped. The mingwpy project was started to try and get around this type of thing, but has since been abandoned. I could get the C-library to compile, but linking it into a python module for import is proving difficult. So we have some options here:
I've also added a mingw branch you could try out. For me, this fails with: which if you run directly, yields: Pretty sure those "file format not recognized; treating as a linker script" is the root of the error here, but I can't find much info about it |
|
No worries, the Linux route is probably best. Thanks for looking at this in
detail,
Sandeep.
Sent from my phone.
On May 25, 2018 1:02 PM, "arghdos" <notifications@github.com> wrote:
Ok, so this is actually a bit more complicated than I hoped.
Unfortunately Conda compiles everything with MSVC compilers
<https://conda.io/docs/user-guide/tutorials/build-windows.html> on windows,
and there are some serious
<https://stackoverflow.com/questions/49088038/pyd-modules-build-failed-with-ld-exe-complaining-def-file-def1-syntax-error>
incompatibilities between the MinGW
<https://stackoverflow.com/questions/24291506/building-minimal-cython-file-with-python-3-3-anaconda-under-windows-7>
and MSVC python run-time libraries.
The mingwpy <https://github.com/mingwpy/mingwpy> project was started to try
and get around this type of thing, but has since been abandoned.
I could get the C-library to compile, but linking it into a python module
for import is proving difficult. So we have some options here:
1.
Implement a MSVC C target -- a bit annoying since it'll require you
either have MSVC installed (or the Microsoft SDK's)
2.
Go to linux where life becomes much less of an issue!
I've also added a mingw branch you could try out:
For me, this fails with:
python -m pyjac.pywrap -l c -so out
...
subprocess.CalledProcessError: Command
'['C:\\ProgramData\\Miniconda3\\python.exe',
'C:\\ProgramData\\Miniconda3\\lib\\site-packages\\pyjac\\pywrap\\pyjacob_setup.py',
'build_ext', '--build-lib', 'C:\\Users\\njc07003\\Desktop\\test',
'--compiler=mingw32']' returned non-zero exit status 1.
which if you run directly, yields:
C:\ProgramData\Miniconda3\python.exe
C:\ProgramData\Miniconda3\lib\site-packages\pyjac\pywrap\pyjacob_setup.py
build_ext --build-lib C:\Users\njc07003\Desktop\test
--compiler=mingw32
...
writing c:\programdata\miniconda3\lib\site-packages\pyjac\pywrap\pyjacob.cp36-win_amd64.def
C:\ProgramData\Miniconda3\pkgs\mingw-4.7-1\MinGW\bin\gcc.exe -shared
-s c:\programdata\miniconda3\lib\site-packages\pyjac\pywrap\pyjacob_wrapper.o
C:\Users\njc07003\Desktop\test\build\temp.win-amd64-3.6\libc_pyjac.a
c:\programdata\miniconda3\lib\site-packages\pyjac\pywrap\pyjacob.cp36-win_amd64.def
-LC:\ProgramData\Miniconda3\libs
-LC:\ProgramData\Miniconda3\PCbuild\amd64 -lpython36 -lmsvcr140 -o
C:\Users\njc07003\Desktop\test\pyjacob.cp36-win_amd64.pyd
c:/programdata/miniconda3/pkgs/mingw-4.7-1/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.7.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
c:\programdata\miniconda3\lib\site-packages\pyjac\pywrap\pyjacob.cp36-win_amd64.def:1:
syntax error
c:/programdata/miniconda3/pkgs/mingw-4.7-1/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.7.0/../../../../x86_64-w64-mingw32/bin/ld.exe:c:\programdata\miniconda3\lib\site-packages\pyjac\pywrap\pyjacob.cp36-win_amd64.def:
file format not recognized; treating as linker script
c:/programdata/miniconda3/pkgs/mingw-4.7-1/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.7.0/../../../../x86_64-w64-mingw32/bin/ld.exe:c:\programdata\miniconda3\lib\site-packages\pyjac\pywrap\pyjacob.cp36-win_amd64.def:1:
syntax error
collect2.exe: error: ld returned 1 exit status
error: command 'C:\\ProgramData\\Miniconda3\\pkgs\\mingw-4.7-1\\MinGW\\bin\\gcc.exe'
failed with exit status 1
Pretty sure those "file format not recognized; treating as a linker script"
is the root of the error here, but I can't find much info about it
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#32 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/Alvbc1xeY7DoESCfDpw7_4e3k6S2Ulswks5t2Dk_gaJpZM4UNOVq>
.
|
Open
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I'm using Cantera + Python in Anaconda and have installed pyjac. Works well and generates the C code but I am running into an issue trying to generate the C wrapper that works on that output for later use in Python (i.e. a cantera program).
It is fairly silly: Pyjac doesn't find the gcc compiler, which is installed through MingW in the Anaconda environment. Outside Pyjac's run, the conda prompt is able to find gcc.
I don't think this is a PyJac issue but perhaps someone could spot the obvious and put me out of my misery?
Best wishes,
Sandeep
The text was updated successfully, but these errors were encountered: