Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Compile issue - 'PyThreadState' #71

Closed
joshua-wx opened this issue Jul 2, 2019 · 18 comments
Closed

Compile issue - 'PyThreadState' #71

joshua-wx opened this issue Jul 2, 2019 · 18 comments
Labels
bug question Waiting for more information

Comments

@joshua-wx
Copy link

joshua-wx commented Jul 2, 2019 •
edited

Hi,

I've compiled CyLP before without any issues but just ran into a problem with Python 3.7
I'm using coincbc-2.10.2 (conda forge), python 3.7.3, numpy 1.16.4, gcc 9.1.0 and cython 0.29.11. Also the COIN_INSTALL_DIR has been correctly set.

I've seen similar errors from PyThreadState reported for pandas and numpy
link

Hope you can help!

Here's the error stack.

n0/jss548/miniconda3/envs/radar-dev/lib/python3.7/site-packages/numpy/core/include -I. -I/g/data/en0/jss548/miniconda3/envs/radar-dev/include/python3.7m -c cylp/cy/CyCbcModel.cpp -o build/temp.linux-x86_64-3.7/cylp/cy/CyCbcModel.o -w
cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
cylp/cy/CyCbcModel.cpp: In function 'void __Pyx_ExceptionSave(PyObject**, PyObject**, PyObject**)':
cylp/cy/CyCbcModel.cpp:10528:21: error: 'PyThreadState' {aka 'struct _ts'} has no member named 'exc_type'; did you mean 'curexc_type'?
10528 | type = tstate->exc_type;
| ^~~~~~~~
| curexc_type
cylp/cy/CyCbcModel.cpp:10529:22: error: 'PyThreadState' {aka 'struct _ts'} has no member named 'exc_value'; did you mean 'curexc_value'?
10529 | value = tstate->exc_value;
| ^~~~~~~~~
| curexc_value
cylp/cy/CyCbcModel.cpp:10530:19: error: 'PyThreadState' {aka 'struct _ts'} has no member named 'exc_traceback'; did you mean 'curexc_traceback'?
10530 | tb = tstate->exc_traceback;
| ^~~~~~~~~~~~~
| curexc_traceback
cylp/cy/CyCbcModel.cpp: In function 'void __Pyx_ExceptionReset(PyObject, PyObject, PyObject)':
cylp/cy/CyCbcModel.cpp:10542:24: error: 'PyThreadState' {aka 'struct _ts'} has no member named 'exc_type'; did you mean 'curexc_type'?
10542 | tmp_type = tstate->exc_type;
| ^~~~~~~~
| curexc_type
cylp/cy/CyCbcModel.cpp:10543:25: error: 'PyThreadState' {aka 'struct _ts'} has no member named 'exc_value'; did you mean 'curexc_value'?
10543 | tmp_value = tstate->exc_value;
| ^~~~~~~~~
| curexc_value
cylp/cy/CyCbcModel.cpp:10544:22: error: 'PyThreadState' {aka 'struct _ts'} has no member named 'exc_traceback'; did you mean 'curexc_traceback'?
10544 | tmp_tb = tstate->exc_traceback;
| ^~~~~~~~~~~~~
| curexc_traceback
cylp/cy/CyCbcModel.cpp:10545:13: error: 'PyThreadState' {aka 'struct _ts'} has no member named 'exc_type'; did you mean 'curexc_type'?
10545 | tstate->exc_type = type;
| ^~~~~~~~
| curexc_type
cylp/cy/CyCbcModel.cpp:10546:13: error: 'PyThreadState' {aka 'struct _ts'} has no member named 'exc_value'; did you mean 'curexc_value'?
10546 | tstate->exc_value = value;
| ^~~~~~~~~
| curexc_value
cylp/cy/CyCbcModel.cpp:10547:13: error: 'PyThreadState' {aka 'struct _ts'} has no member named 'exc_traceback'; did you mean 'curexc_traceback'?
10547 | tstate->exc_traceback = tb;
| ^~~~~~~~~~~~~
| curexc_traceback
cylp/cy/CyCbcModel.cpp: In function 'int __Pyx_GetException(PyObject**, PyObject**, PyObject**)':
cylp/cy/CyCbcModel.cpp:10590:24: error: 'PyThreadState' {aka 'struct _ts'} has no member named 'exc_type'; did you mean 'curexc_type'?
10590 | tmp_type = tstate->exc_type;
| ^~~~~~~~
| curexc_type
cylp/cy/CyCbcModel.cpp:10591:25: error: 'PyThreadState' {aka 'struct _ts'} has no member named 'exc_value'; did you mean 'curexc_value'?
10591 | tmp_value = tstate->exc_value;
| ^~~~~~~~~
| curexc_value
cylp/cy/CyCbcModel.cpp:10592:22: error: 'PyThreadState' {aka 'struct _ts'} has no member named 'exc_traceback'; did you mean 'curexc_traceback'?
10592 | tmp_tb = tstate->exc_traceback;
| ^~~~~~~~~~~~~
| curexc_traceback
cylp/cy/CyCbcModel.cpp:10593:13: error: 'PyThreadState' {aka 'struct _ts'} has no member named 'exc_type'; did you mean 'curexc_type'?
10593 | tstate->exc_type = local_type;
| ^~~~~~~~
| curexc_type
cylp/cy/CyCbcModel.cpp:10594:13: error: 'PyThreadState' {aka 'struct _ts'} has no member named 'exc_value'; did you mean 'curexc_value'?
10594 | tstate->exc_value = local_value;
| ^~~~~~~~~
| curexc_value
cylp/cy/CyCbcModel.cpp:10595:13: error: 'PyThreadState' {aka 'struct _ts'} has no member named 'exc_traceback'; did you mean 'curexc_traceback'?
10595 | tstate->exc_traceback = local_tb;
| ^~~~~~~~~~~~~
| curexc_traceback
error: command '/apps/gcc/9.1.0/bin/gcc' failed with exit status 1
----------------------------------------
ERROR: Command "/g/data/en0/jss548/miniconda3/envs/radar-dev/bin/python -u -c 'import setuptools, tokenize;file='"'"'/short/en0/jss548/tmp/pip-install-b93fks8q/cylp/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /short/en0/jss548/tmp/pip-record-4bnpy7e2/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /short/en0/jss548/tmp/pip-install-b93fks8q/cylp/
@joshua-wx joshua-wx changed the title Compile issue - 'PyThreadState Compile issue - 'PyThreadState' Jul 2, 2019
@tkralphs
Copy link
Member

tkralphs commented Jul 2, 2019

Hmm, strange, I did build successfully in Python 3.7 on Windows. I guess you are on Linux? I generated the files in Python 3.6, which I now realize might be a problem for Python 3.7 compatibility.

In any case, the .cpp files are auto-generated, so re-generating them will probably fix it. Can you try that? Just set

export CYLP_USE_CYTHON=

before building. I don't actually know exactly how Cython works vis a vis backwards compatibility, but I assume generated code works in all previous versions of Python. Let me know and I can re-generate the files in Python 3.7 and check them in.

@joshua-wx
Copy link
Author

Thanks for the advice! No luck yet though.

After running the following, it still fails with the same error. I've attached the full output.
export COIN_INSTALL_DIR=/g/data/en0/jss548/miniconda3/pkgs/coincbc-2.10.2-hab63836_0
export CYLP_USE_CYTHON=
pip install --pre cylp

I'm on a linux machine and installing in a clean miniconda environment.

conda_env_list.txt
output.txt

@tkralphs
Copy link
Member

tkralphs commented Jul 4, 2019

Looks like it did not re-generate the .cpp files because the file modification times of the .pyx files are older. Can you touch all the .pyx files?

@joshua-wx
Copy link
Author

Sorry - I'm still running in to problems and decided to try using a clean anaconda environment. See the commands below and the error.

[using gcc 6.2.0/python 3.7/Linux raijin3 3.10.0-957.21.3.el6.x86_64]

conda create -n test-cylp python=3.7
conda activate test-cylp
conda install -c conda-forge numpy scipy coincbc cython
git clone https://github.com/coin-or/CyLP
cd CyLP
export COIN_INSTALL_DIR="/g/data/en0/jss548/miniconda3/pkgs/coincbc-2.10.2-hab63836_0"
export PKG_CONFIG_PATH="/g/data/en0/jss548/miniconda3/pkgs/coincbc-2.10.2-hab63836_0/lib/pkgconfig"
export CYLP_USE_CYTHON=
find -type f -name "*.pyx" -exec touch {} +

(test-cylp) [jss548@raijin3 CyLP]$ pip install . --user
Processing /g/data1a/kl02/jss548/libs/CyLP
Requirement already satisfied: numpy>=1.5.0 in /g/data1a/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages (from cylp==0.9.0b1) (1.16.4)
Requirement already satisfied: scipy>=0.10.0 in /g/data1a/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages (from cylp==0.9.0b1) (1.3.0)
Building wheels for collected packages: cylp
Building wheel for cylp (setup.py) ... error
ERROR: Complete output from command /g/data/en0/jss548/miniconda3/envs/test-cylp/bin/python -u -c 'import setuptools, tokenize;file='"'"'/short/en0/jss548/tmp/pip-req-build-b01cpzuk/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /short/en0/jss548/tmp/pip-wheel-n_6sdv1j --python-tag cp37:
ERROR: running bdist_wheel
running build
running build_py
running build_ext
cythoning cylp/cy/CyClpPrimalColumnPivotBase.pyx to cylp/cy/CyClpPrimalColumnPivotBase.cpp
Traceback (most recent call last):
File "", line 1, in
File "/short/en0/jss548/tmp/pip-req-build-b01cpzuk/setup.py", line 448, in
package_data={"cylp": extra_files})
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/setuptools/init.py", line 145, in setup
return distutils.core.setup(**attrs)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/wheel/bdist_wheel.py", line 192, in run
self.run_command('build')
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 193, in build_extensions
ext.sources = self.cython_sources(ext.sources, ext)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 346, in cython_sources
full_module_name=module_name)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Compiler/Main.py", line 778, in compile
return compile_single(source, options, full_module_name)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Compiler/Main.py", line 727, in compile_single
return run_pipeline(source, options, full_module_name)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Compiler/Main.py", line 515, in run_pipeline
err, enddata = Pipeline.run_pipeline(pipeline, source)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Compiler/Pipeline.py", line 355, in run_pipeline
data = run(phase, data)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Compiler/Pipeline.py", line 335, in run
return phase(data)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Compiler/Pipeline.py", line 32, in parse
scope = context.find_module(full_module_name, pos = initial_pos, need_pxd = 0)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Compiler/Main.py", line 211, in find_module
pxd_pathname = self.find_pxd_file(qualified_name, pos, sys_path=need_pxd)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Compiler/Main.py", line 250, in find_pxd_file
pxd = self.search_include_directories(qualified_name, ".pxd", pos, sys_path=sys_path)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Compiler/Main.py", line 296, in search_include_directories
suffix, pos, include)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Utils.py", line 38, in wrapper
res = cache.get(args, uncomputed)
TypeError: unhashable type: 'list'

ERROR: Failed building wheel for cylp

@tkralphs
Copy link
Member

tkralphs commented Jul 8, 2019

Hmm, I can't really see what the actual error is there. Can you send the output of

python setup.py build

which is a lot more verbose?

@joshua-wx
Copy link
Author

Here it is!

(test-cylp) [jss548@raijin4 CyLP]$ python setup.py build
running build
running build_py
running build_ext
cythoning cylp/cy/CyClpPrimalColumnPivotBase.pyx to cylp/cy/CyClpPrimalColumnPivotBase.cpp
Traceback (most recent call last):
File "setup.py", line 448, in
package_data={"cylp": extra_files})
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/setuptools/init.py", line 145, in setup
return distutils.core.setup(**attrs)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 193, in build_extensions
ext.sources = self.cython_sources(ext.sources, ext)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 346, in cython_sources
full_module_name=module_name)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Compiler/Main.py", line 778, in compile
return compile_single(source, options, full_module_name)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Compiler/Main.py", line 727, in compile_single
return run_pipeline(source, options, full_module_name)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Compiler/Main.py", line 515, in run_pipeline
err, enddata = Pipeline.run_pipeline(pipeline, source)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Compiler/Pipeline.py", line 355, in run_pipeline
data = run(phase, data)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Compiler/Pipeline.py", line 335, in run
return phase(data)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Compiler/Pipeline.py", line 32, in parse
scope = context.find_module(full_module_name, pos = initial_pos, need_pxd = 0)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Compiler/Main.py", line 211, in find_module
pxd_pathname = self.find_pxd_file(qualified_name, pos, sys_path=need_pxd)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Compiler/Main.py", line 250, in find_pxd_file
pxd = self.search_include_directories(qualified_name, ".pxd", pos, sys_path=sys_path)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Compiler/Main.py", line 296, in search_include_directories
suffix, pos, include)
File "/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/Cython/Utils.py", line 38, in wrapper
res = cache.get(args, uncomputed)
TypeError: unhashable type: 'list'

@tkralphs
Copy link
Member

tkralphs commented Jul 8, 2019

Hmm, can't make much sense out of that. Are you using the latest master commit? I made some changes to the setup script and my best guess is that there is a bug there. Are you specifying COIN_INSTALL_DIR to specify the location of the install? This used to be required, but now you have the option of also just adding the path to the .pc files to your PKG_CONFIG_PATH or of adding Clp to your PATH. Maybe you can play around with some different options and see if you can get it to work that way? I did some testing here and it's all working. Anything you can do to help me replicate would be great.

@joshua-wx
Copy link
Author

Thanks for your patience! I'm using the latest master commit.
The setup.py error above was triggered by 'export CYLP_USE_CYTHON='
The only environmental variable I've set otherwise now is:
export PKG_CONFIG_PATH="/g/data/en0/jss548/miniconda3/pkgs/coincbc-2.10.2-hab63836_0/lib/pkgconfig"

Here's the current output of build (I get the same error using COIN_INSTALL_DIR)
It looks like the c++ header files weren't generated because I can't set the CYLP_USE_CYTHON variable?

(test-cylp) [jss548@raijin4 CyLP]$ python setup.py build
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/cylp
copying cylp/init.py -> build/lib.linux-x86_64-3.7/cylp
creating build/lib.linux-x86_64-3.7/cylp/cy
copying cylp/cy/init.py -> build/lib.linux-x86_64-3.7/cylp/cy
copying cylp/cy/createCythonInterface.py -> build/lib.linux-x86_64-3.7/cylp/cy
copying cylp/cy/CySolve.py -> build/lib.linux-x86_64-3.7/cylp/cy
creating build/lib.linux-x86_64-3.7/cylp/py
copying cylp/py/PySolve.py -> build/lib.linux-x86_64-3.7/cylp/py
copying cylp/py/test_PySolve.py -> build/lib.linux-x86_64-3.7/cylp/py
copying cylp/py/init.py -> build/lib.linux-x86_64-3.7/cylp/py
copying cylp/py/Constants.py -> build/lib.linux-x86_64-3.7/cylp/py
creating build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/MostFrequentPivot.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/PivotPythonBase.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/init.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/DualDantzigPivot.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/LIFOPivot.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/DualPivotPythonBase.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/DantzigPivot.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/PositiveEdgePivot.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/PositiveEdgeWolfePivot.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/WolfePivot.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/WolfePivotPE.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
creating build/lib.linux-x86_64-3.7/cylp/py/modeling
copying cylp/py/modeling/CyLPModel.py -> build/lib.linux-x86_64-3.7/cylp/py/modeling
copying cylp/py/modeling/init.py -> build/lib.linux-x86_64-3.7/cylp/py/modeling
copying cylp/py/modeling/test_modeling.py -> build/lib.linux-x86_64-3.7/cylp/py/modeling
creating build/lib.linux-x86_64-3.7/cylp/py/utils
copying cylp/py/utils/readSetcovering.py -> build/lib.linux-x86_64-3.7/cylp/py/utils
copying cylp/py/utils/init.py -> build/lib.linux-x86_64-3.7/cylp/py/utils
copying cylp/py/utils/util.py -> build/lib.linux-x86_64-3.7/cylp/py/utils
copying cylp/py/utils/sparseUtil.py -> build/lib.linux-x86_64-3.7/cylp/py/utils
creating build/lib.linux-x86_64-3.7/cylp/py/mip
copying cylp/py/mip/CyLPCutGenerator.py -> build/lib.linux-x86_64-3.7/cylp/py/mip
copying cylp/py/mip/SimpleNodeCompare.py -> build/lib.linux-x86_64-3.7/cylp/py/mip
copying cylp/py/mip/NodeCompareBase.py -> build/lib.linux-x86_64-3.7/cylp/py/mip
copying cylp/py/mip/init.py -> build/lib.linux-x86_64-3.7/cylp/py/mip
copying cylp/py/mip/GomoryCutGenerator.py -> build/lib.linux-x86_64-3.7/cylp/py/mip
creating build/lib.linux-x86_64-3.7/cylp/py/QP
copying cylp/py/QP/GQP.py -> build/lib.linux-x86_64-3.7/cylp/py/QP
copying cylp/py/QP/QPGen.py -> build/lib.linux-x86_64-3.7/cylp/py/QP
copying cylp/py/QP/QPSReader.py -> build/lib.linux-x86_64-3.7/cylp/py/QP
copying cylp/py/QP/init.py -> build/lib.linux-x86_64-3.7/cylp/py/QP
copying cylp/py/QP/QP.py -> build/lib.linux-x86_64-3.7/cylp/py/QP
creating build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICoinMP.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpSimplexPrimal.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/CbcCompareUser.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICbcNode.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpPrimalColumnPivotBase.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICoinPackedMatrix.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICoinIndexedVector.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICoinMpsIO.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpSimplex.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpDualRowPivotBase.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IOsiCuts.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICbcModel.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICglCutGeneratorBase.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpSimplexPrimal_Wolfe.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpPackedMatrix.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpPrimalColumnPivot.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICbc.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpPackedMatrix.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICoinIndexedVector.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpSimplex.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICoinMpsIO.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpSimplexPrimal_Wolfe.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICoinModel.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpSimplexPrimal.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/CbcCompareUser.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICoinPackedMatrix.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IOsiCuts.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICbc.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICbcModel.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICbcNode.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICoinMP.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpDualRowPivotBase.h -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/CyClpSimplex_api.h -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpPrimalColumnPivotBase.h -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICglCutGeneratorBase.h -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpPrimalColumnPivot.h -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/VERSION -> build/lib.linux-x86_64-3.7/cylp
running build_ext
building 'cylp.cy.CyClpPrimalColumnPivotBase' extension
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/cylp
creating build/temp.linux-x86_64-3.7/cylp/cpp
creating build/temp.linux-x86_64-3.7/cylp/cy
gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/anaconda1anaconda2anaconda3/include/coin -I['./cylp/cpp', './cylp/cy', '/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/numpy/core/include', '.'] -I['/g/data/en0/jss548/miniconda3/envs/test-cylp/include/coin'] -I/g/data/en0/jss548/miniconda3/envs/test-cylp/include/python3.7m -c cylp/cpp/IClpPrimalColumnPivotBase.cpp -o build/temp.linux-x86_64-3.7/cylp/cpp/IClpPrimalColumnPivotBase.o -w
Important Note. git/2.9.5 is not compatible with matlab.
Make sure you do not have a matlab module loaded when using this version of git
cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
In file included from cylp/cpp/IClpPrimalColumnPivotBase.cpp:1:0:
cylp/cpp/IClpPrimalColumnPivotBase.h:5:36: fatal error: ClpPrimalColumnPivot.hpp: No such file or directory
#include "ClpPrimalColumnPivot.hpp"
^
compilation terminated.
error: command 'gcc' failed with exit status 1

@joshua-wx
Copy link
Author

joshua-wx commented Jul 10, 2019 •
edited

I've worked through the issues with a software engineer in Australia. We now have CyLP compiling on the Australian national supercomputer :) The changes to setup.py are below:

diff --git a/setup.py b/setup.py
index a41002a..965c617 100644
--- a/setup.py
+++ b/setup.py
@@ -111,7 +111,7 @@ if 'CbcSolver' not in libs:
     else:
         libs.append('CbcSolver')
 
-libDirs.append(['.', join('.', cythonFilesDir)])
+libDirs.extend(['.', join('.', cythonFilesDir)])
 try:
     libDirs.append(join(CoinDir, 'lib'))
 except:
@@ -123,10 +123,10 @@ if operatingSystem == 'windows':
     except:
         pass
     
-incDirs.append([join('.', cppFilesDir), join('.', cythonFilesDir),
+incDirs.extend([join('.', cppFilesDir), join('.', cythonFilesDir),
                 numpy.get_include(), '.'])
 try:
-    incDirs.append([join(CoinDir, 'include', 'coin')])
+    incDirs.extend([join(CoinDir, 'include', 'coin')])
 except:
     pass
 
@@ -143,7 +143,7 @@ else:
     fileext = '.cpp'
 
 
-extra_compile_args = ['-w']
+extra_compile_args = ['-w', '-std=c++11']
 ext_modules = []
 
 if operatingSystem == 'mac':

@tkralphs
Copy link
Member

Ah, thank you so much! I figured it was something buggy in the changes I introduced in setup.py. Thanks for your efforts in fixing this!

@tkralphs
Copy link
Member

By the way, was the final solution for the original issue just to re-generate the .cpp files? Were you eventually able to do that?

@joshua-wx
Copy link
Author

Yes, this fixed the original issue :) All sorted now.

@BrannonKing
Copy link

BrannonKing commented Jul 20, 2019 •
edited

This error "‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_type’; did you mean ‘curexc_type’?" just occurred for me. It is not fixed in the current beta. The env and touch workarounds seem to have no effect for me.

@tkralphs
Copy link
Member

OK, let me re-generate the cpp files with the latest Cython in Python 3.7, then we'll see if this fixes the problem for you and go from there.

@tkralphs tkralphs reopened this Jul 20, 2019
@tkralphs
Copy link
Member

tkralphs commented Jul 20, 2019 •
edited

I re-generated the files now with Cython 0.29.12 in Python 3.7. Let me know if this fixes things

@tkralphs tkralphs added bug question Waiting for more information labels Jul 27, 2019
@tkralphs
Copy link
Member

tkralphs commented Aug 4, 2019

I'm assuming that this is fixed. I'm closing for now. Please re-open if it's still an issue.

@tkralphs tkralphs closed this as completed Aug 4, 2019
@cafunechen cafunechen mentioned this issue Nov 22, 2021
Closed
@cafunechen
Copy link

Thanks for your patience! I'm using the latest master commit. The setup.py error above was triggered by 'export CYLP_USE_CYTHON=' The only environmental variable I've set otherwise now is: export PKG_CONFIG_PATH="/g/data/en0/jss548/miniconda3/pkgs/coincbc-2.10.2-hab63836_0/lib/pkgconfig"

Here's the current output of build (I get the same error using COIN_INSTALL_DIR) It looks like the c++ header files weren't generated because I can't set the CYLP_USE_CYTHON variable?

(test-cylp) [jss548@raijin4 CyLP]$ python setup.py build
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/cylp
copying cylp/init.py -> build/lib.linux-x86_64-3.7/cylp
creating build/lib.linux-x86_64-3.7/cylp/cy
copying cylp/cy/init.py -> build/lib.linux-x86_64-3.7/cylp/cy
copying cylp/cy/createCythonInterface.py -> build/lib.linux-x86_64-3.7/cylp/cy
copying cylp/cy/CySolve.py -> build/lib.linux-x86_64-3.7/cylp/cy
creating build/lib.linux-x86_64-3.7/cylp/py
copying cylp/py/PySolve.py -> build/lib.linux-x86_64-3.7/cylp/py
copying cylp/py/test_PySolve.py -> build/lib.linux-x86_64-3.7/cylp/py
copying cylp/py/init.py -> build/lib.linux-x86_64-3.7/cylp/py
copying cylp/py/Constants.py -> build/lib.linux-x86_64-3.7/cylp/py
creating build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/MostFrequentPivot.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/PivotPythonBase.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/init.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/DualDantzigPivot.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/LIFOPivot.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/DualPivotPythonBase.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/DantzigPivot.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/PositiveEdgePivot.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/PositiveEdgeWolfePivot.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/WolfePivot.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
copying cylp/py/pivots/WolfePivotPE.py -> build/lib.linux-x86_64-3.7/cylp/py/pivots
creating build/lib.linux-x86_64-3.7/cylp/py/modeling
copying cylp/py/modeling/CyLPModel.py -> build/lib.linux-x86_64-3.7/cylp/py/modeling
copying cylp/py/modeling/init.py -> build/lib.linux-x86_64-3.7/cylp/py/modeling
copying cylp/py/modeling/test_modeling.py -> build/lib.linux-x86_64-3.7/cylp/py/modeling
creating build/lib.linux-x86_64-3.7/cylp/py/utils
copying cylp/py/utils/readSetcovering.py -> build/lib.linux-x86_64-3.7/cylp/py/utils
copying cylp/py/utils/init.py -> build/lib.linux-x86_64-3.7/cylp/py/utils
copying cylp/py/utils/util.py -> build/lib.linux-x86_64-3.7/cylp/py/utils
copying cylp/py/utils/sparseUtil.py -> build/lib.linux-x86_64-3.7/cylp/py/utils
creating build/lib.linux-x86_64-3.7/cylp/py/mip
copying cylp/py/mip/CyLPCutGenerator.py -> build/lib.linux-x86_64-3.7/cylp/py/mip
copying cylp/py/mip/SimpleNodeCompare.py -> build/lib.linux-x86_64-3.7/cylp/py/mip
copying cylp/py/mip/NodeCompareBase.py -> build/lib.linux-x86_64-3.7/cylp/py/mip
copying cylp/py/mip/init.py -> build/lib.linux-x86_64-3.7/cylp/py/mip
copying cylp/py/mip/GomoryCutGenerator.py -> build/lib.linux-x86_64-3.7/cylp/py/mip
creating build/lib.linux-x86_64-3.7/cylp/py/QP
copying cylp/py/QP/GQP.py -> build/lib.linux-x86_64-3.7/cylp/py/QP
copying cylp/py/QP/QPGen.py -> build/lib.linux-x86_64-3.7/cylp/py/QP
copying cylp/py/QP/QPSReader.py -> build/lib.linux-x86_64-3.7/cylp/py/QP
copying cylp/py/QP/init.py -> build/lib.linux-x86_64-3.7/cylp/py/QP
copying cylp/py/QP/QP.py -> build/lib.linux-x86_64-3.7/cylp/py/QP
creating build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICoinMP.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpSimplexPrimal.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/CbcCompareUser.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICbcNode.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpPrimalColumnPivotBase.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICoinPackedMatrix.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICoinIndexedVector.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICoinMpsIO.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpSimplex.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpDualRowPivotBase.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IOsiCuts.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICbcModel.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICglCutGeneratorBase.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpSimplexPrimal_Wolfe.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpPackedMatrix.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpPrimalColumnPivot.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICbc.cpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpPackedMatrix.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICoinIndexedVector.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpSimplex.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICoinMpsIO.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpSimplexPrimal_Wolfe.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICoinModel.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpSimplexPrimal.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/CbcCompareUser.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICoinPackedMatrix.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IOsiCuts.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICbc.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICbcModel.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICbcNode.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICoinMP.hpp -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpDualRowPivotBase.h -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/CyClpSimplex_api.h -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpPrimalColumnPivotBase.h -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/ICglCutGeneratorBase.h -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/cpp/IClpPrimalColumnPivot.h -> build/lib.linux-x86_64-3.7/cylp/cpp
copying cylp/VERSION -> build/lib.linux-x86_64-3.7/cylp
running build_ext
building 'cylp.cy.CyClpPrimalColumnPivotBase' extension
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/cylp
creating build/temp.linux-x86_64-3.7/cylp/cpp
creating build/temp.linux-x86_64-3.7/cylp/cy
gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/anaconda1anaconda2anaconda3/include/coin -I['./cylp/cpp', './cylp/cy', '/g/data/en0/jss548/miniconda3/envs/test-cylp/lib/python3.7/site-packages/numpy/core/include', '.'] -I['/g/data/en0/jss548/miniconda3/envs/test-cylp/include/coin'] -I/g/data/en0/jss548/miniconda3/envs/test-cylp/include/python3.7m -c cylp/cpp/IClpPrimalColumnPivotBase.cpp -o build/temp.linux-x86_64-3.7/cylp/cpp/IClpPrimalColumnPivotBase.o -w
Important Note. git/2.9.5 is not compatible with matlab.
Make sure you do not have a matlab module loaded when using this version of git
cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
In file included from cylp/cpp/IClpPrimalColumnPivotBase.cpp:1:0:
cylp/cpp/IClpPrimalColumnPivotBase.h:5:36: fatal error: ClpPrimalColumnPivot.hpp: No such file or directory
#include "ClpPrimalColumnPivot.hpp"
^
compilation terminated.
error: command 'gcc' failed with exit status 1

@tkralphs hi, Are these two problems the same？ #135

@tkralphs
Copy link
Member

hi, Are these two problems the same？ #135

No.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug question Waiting for more information
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@BrannonKing @tkralphs @joshua-wx @cafunechen