Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to build Python library from source on Linux #380

Closed
mxr opened this issue May 17, 2017 · 25 comments
Closed

Unable to build Python library from source on Linux #380

mxr opened this issue May 17, 2017 · 25 comments
Assignees
@Mizux
Labels
Bug Lang: Python Python wrapper issue OS: Linux GNU/Linux OS
Milestone
v8.0

Comments

@mxr
Copy link

mxr commented May 17, 2017 •
edited
Loading

Hello, I'm following the instructions here to build the Python library but I'm getting a compiler error.

Repro steps:

sudo apt-get install git bison flex python-setuptools python-dev autoconf libtool zlib1g-dev texinfo help2man gawk g++ curl texlive cmake subversion
sudo apt-get install zip unzip  # I installed this to fix another error that came up

make third_party
make python

The last command fails with

g++ -fPIC -std=c++0x -fwrapv -O4 -DNDEBUG -I. -Iexamples -Iortools/gen -I/home/mrozentsveyg/tmp/or-tools/dependencies/install/include -DARCH_K8 -Wno-deprecated -I/home/mrozentsveyg/tmp/or-tools/dependencies/install/include -I/home/mrozentsveyg/tmp/or-tools/dependencies/install/include -DUSE_CBC -I/home/mrozentsveyg/tmp/or-tools/dependencies/install/include -DUSE_CLP     -DUSE_GLOP -DUSE_BOP -I/include -I/usr/local/include/python2.7 -I/usr/lib/python2.7   -c ortools/gen/ortools/algorithms/knapsack_solver_python_wrap.cc -o objs/swig/knapsack_solver_python_wrap.o
/home/mrozentsveyg/tmp/or-tools/dependencies/install/bin/swig -I. -c++ -python  -o ortools/gen/ortools/graph/graph_python_wrap.cc -module pywrapgraph ./ortools/graph/python/graph.i
/home/mrozentsveyg/tmp/or-tools/dependencies/install/bin/swig  -I/home/mrozentsveyg/tmp/or-tools/dependencies/install/include -DUSE_CLP -I/home/mrozentsveyg/tmp/or-tools/dependencies/install/include -DUSE_CBC    -DUSE_GLOP -DUSE_BOP -I. -c++ -python  -o ortools/gen/ortools/linear_solver/linear_solver_python_wrap.cc -module pywraplp ./ortools/linear_solver/python/linear_solver.i
ortools/gen/ortools/algorithms/knapsack_solver_python_wrap.cc:173:21: fatal error: Python.h: No such file or directory
 # include <Python.h>
                     ^
compilation terminated.
make: *** [objs/swig/constraint_solver_python_wrap.o] Error 1

I'm on Ubuntu 14.04.5 with Python 2.7.6 and g++ version Ubuntu 4.8.4-2ubuntu1~14.04.3

Any advice is appreciated. Thanks!
@lperron
Copy link
Collaborator

lperron commented May 17, 2017

can you report Makefile.local ?
It looks like the python path is not found.

@mxr
Copy link
Author

mxr commented May 18, 2017

Here are its contents:

JDK_DIRECTORY =
UNIX_PYTHON_VER = 2.7
PATH_TO_CSHARP_COMPILER =
CLR_KEYFILE = bin/or-tools.snk

# Define UNIX_GLPK_DIR to point to a compiled version of GLPK to use it
# Define UNIX_SCIP_DIR to point to a compiled version of SCIP to use it
#   i.e.: <path>/scipoptsuite-4.0.0/scip-4.0.0
#   compile scip with GMP=false READLINE=false
# Define UNIX_GUROBI_DIR and GUROBI_LIB_VERSION to use Gurobi
# Define UNIX_CPLEX_DIR to use CPLEX

UNIX_GFLAGS_DIR = /home/mrozentsveyg/tmp/or-tools/dependencies/install
UNIX_PROTOBUF_DIR = /home/mrozentsveyg/tmp/or-tools/dependencies/install
UNIX_SWIG_BINARY = /home/mrozentsveyg/tmp/or-tools/dependencies/install/bin/swig
UNIX_CLP_DIR = /home/mrozentsveyg/tmp/or-tools/dependencies/install
UNIX_CBC_DIR = /home/mrozentsveyg/tmp/or-tools/dependencies/install

Let me know if I can give more information. Note that I was compiling on commit d2cd773

@mxr
Copy link
Author

mxr commented May 23, 2017

Hi @lperron, I'm still unable to compile (tried again on the latest master). I added some debug code to find out about my environment but I wasn't able to get to the bottom of the issue.

diff --git a/makefiles/Makefile.python.mk b/makefiles/Makefile.python.mk
index 04284ce..40c588a 100755
--- a/makefiles/Makefile.python.mk
+++ b/makefiles/Makefile.python.mk
@@ -19,6 +19,12 @@ ifeq ($(shell $(PYTHON_EXECUTABLE) -c "from sys import version_info as v; print
   SWIG_PYTHON3_FLAG=-py3 -DPY3
 endif

+debug_info_python:
+       @echo $(OR_TOOLS_PYTHONPATH)
+       @echo $(PYTHONPATH)
+       @echo $(PYTHON_EXECUTABLE)
+       @echo $(UNIX_PYTHON_VER)
+
 # Main target
 CANONIC_PYTHON_EXECUTABLE = $(subst $(SPACE),$(BACKSLASH_SPACE),$(subst \,/,$(subst \\,/,$(PYTHON_EXECUTABLE))))
 ifeq ($(wildcard  $(CANONIC_PYTHON_EXECUTABLE)),)
$ make debug_info_python
/home/mrozentsveyg/tmp/or-tools:/home/mrozentsveyg/tmp/or-tools/dependencies/sources/protobuf-3.2.0/python

/usr/bin/python2.7
2.7

Is there something else I could try?

Thanks!

@lperron
Copy link
Collaborator

lperron commented May 24, 2017 via email

@diegoalberione
Copy link

Hello, I am trying to install OrTools as the documentation says and I get the following error:

python check_python_deps.py --log=ERROR
[ERROR] You are using a ortools module that doesn't have a version attribute : /root/.local/lib/python2.7/site-packages/ortools-6.0.4217-py2.7-linux-x86_64.egg/
ortools/init.pyc"
Run "/usr/bin/python setup.py install --user" to upgrade.
If the problem persists, remove the site-package that contains "/root/.local/lib/python2.7/site-packages/ortools-6.0.4217-py2.7-linux-x86_64.egg/ortools/init.py
c". You can do so either manually or by using pip.

I have tried with python version 2.7, 3.5 and 3.6. I use Ubuntu 14.04

I appreciate any help !!

@lperron
Copy link
Collaborator

lperron commented May 24, 2017 via email

@mxr
Copy link
Author

mxr commented May 31, 2017

@lperron Thanks, that helped. However I ran into some more issues, with the workarounds posted below:

  1. fix_python_libraries_on_linux.sh depends on patchelf being at a hardcoded path. I fixed that by doing this before the build:
sudo apt-get install patchelf
mkdir -p dependencies/install/bin
ln -s `which patchelf` dependencies/install/bin/patchelf
  1. Compiling glog failed because of aclocal conflicts.
cd dependencies/sources/glog-0.3.5 && PATH=/home/maxr/src/or-tools/dependencies/install/bin:/home/maxr/bin:/home/maxr/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin  make install
make[1]: Entering directory '/home/maxr/src/or-tools/dependencies/sources/glog-0.3.5'
CDPATH="${ZSH_VERSION+.}:" && cd . && aclocal-1.14 -I m4
/bin/bash: aclocal-1.14: command not found
Makefile:957: recipe for target 'aclocal.m4' failed
make[1]: *** [aclocal.m4] Error 127
make[1]: Leaving directory '/home/maxr/src/or-tools/dependencies/sources/glog-0.3.5'
makefiles/Makefile.third_party.unix.mk:214: recipe for target 'dependencies/install/include/glog/logging.h' failed
make: *** [dependencies/install/include/glog/logging.h] Error 2

I worked around this by applying the below change:

--- dependencies/sources/glog-0.3.5/configure.old	2017-05-30 15:03:16.255717394 -0700
+++ dependencies/sources/glog-0.3.5/configure	2017-05-30 13:39:24.957294021 -0700
@@ -2760,7 +2760,7 @@
 # (for sanity checking)


-am__api_version='1.14'
+am__api_version='1.15'

 ac_aux_dir=
 for ac_dir in "$srcdir" "$srcdir/.." "$srcdir/../.."; do

Then ran autoreconf --force --install in the glog-0.3.5 folder, then make third_party from the root again.

  1. My Python.h path was /usr/include/python2.7 and adding the ADD_PYTHON_INC="-I/usr/include/python2.7" per your suggestion/docs unblocked me.

I was able to produce a .whl, but trying to use it showed that some modules are missing:

>>> from ortools.graph import pywrapgraph
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/ortools/graph/pywrapgraph.py", line 17, in <module>
    _pywrapgraph = swig_import_helper()
  File "/usr/local/lib/python2.7/dist-packages/ortools/graph/pywrapgraph.py", line 16, in swig_import_helper
    return importlib.import_module('_pywrapgraph')
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: No module named _pywrapgraph

Is this the same issue that you mentioned in #381 (comment)? Or related to the "ImportError: No module named pywraplp" issue at https://developers.google.com/optimization/introduction/installing?

If not, do you have advice for how to debug the missing modules?

@lperron
Copy link
Collaborator

lperron commented May 31, 2017 via email

@lperron
Copy link
Collaborator

lperron commented May 31, 2017 via email

@mxr
Copy link
Author

mxr commented May 31, 2017

@lperron thanks for the cleanup and quick response. I tried to compile but ran into an error with proto. make third_party succeeded but make python did not:

$ make python ADD_PYTHON_INC="-I/usr/include/python2.7"
cd dependencies/sources/protobuf-3.2.0/python && /usr/bin/python2.7 setup.py build
running build
running build_py
Generating google/protobuf/descriptor_pb2.py...
protoc is not installed nor found in ../src.  Please compile it or install the binary package.
makefiles/Makefile.python.mk:72: recipe for target 'dependencies/sources/protobuf-3.2.0/python/google/protobuf/descriptor_pb2.py' failed
make: *** [dependencies/sources/protobuf-3.2.0/python/google/protobuf/descriptor_pb2.py] Error 255

I do see protoc here:

$ find . -name protoc
./dependencies/sources/protobuf-3.2.0/cmake/build/protoc
./dependencies/install/bin/protoc

Perhaps I need to pass another flag?

@lperron
Copy link
Collaborator

lperron commented May 31, 2017 via email

@lperron
Copy link
Collaborator

lperron commented May 31, 2017 via email

@mxr
Copy link
Author

mxr commented May 31, 2017

Will merge master and try again. Thanks!

@mxr
Copy link
Author

mxr commented Jun 1, 2017 •
edited
Loading

@lperron I was able to produce a wheel with the above directions, but using it caused some issues.

First, the build machine is able to use or-tools fine

$ gcc --version | head -n1
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
$ python -c "from ortools.graph import pywrapgraph"; echo $?
0

On the machine that I want to use or-tools (Trusty), I hit a new import issue

$ python
Python 2.7.6 (default, Oct 26 2016, 20:30:19)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ortools.graph import pywrapgraph
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/ortools/graph/pywrapgraph.py", line 28, in <module>
    _pywrapgraph = swig_import_helper()
  File "/usr/local/lib/python2.7/dist-packages/ortools/graph/pywrapgraph.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrapgraph', fp, pathname, description)
ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /usr/local/lib/python2.7/dist-packages/ortools/graph/_pywrapgraph.so)

When the Python interpreter is started you can see it's using an older gcc (4.8.4). Also, libstd++.so.6 indeed doesn't have the required glibc:

$ strings /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | grep GLIBCXX | tail -n 5
GLIBCXX_3.4.16
GLIBCXX_3.4.17
GLIBCXX_3.4.18
GLIBCXX_3.4.19
GLIBCXX_DEBUG_MESSAGE_LENGTH

I tried to update it using these instructions.

$ sudo apt-get install software-properties-common python-software-properties
$ sudo apt-get update
$ sudo apt-get install gcc-5 g++-5
$ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-5 60 --slave /usr/bin/g++ g++ /usr/bin/g++-5

And verified the right GLIBC is in the .so file

$ strings /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | grep GLIBCXX_3.4.21
GLIBCXX_3.4.21

Retrying gave a different import error:

$ python
Python 2.7.6 (default, Oct 26 2016, 20:30:19)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ortools.graph import pywrapgraph
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/ortools/graph/pywrapgraph.py", line 28, in <module>
    _pywrapgraph = swig_import_helper()
  File "/usr/local/lib/python2.7/dist-packages/ortools/graph/pywrapgraph.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrapgraph', fp, pathname, description)
ImportError: /usr/local/lib/python2.7/dist-packages/ortools/graph/../../ortools/libortools.so: symbol _ZTVNSt7__cxx1115basic_stringbufIcSt11char_traitsIcESaIcEEE, version GLIBCXX_3.4.21 not defined in file libstdc++.so.6 with link time reference

Since it didn't work, I undid the glib change via

$ wget http://security.ubuntu.com/ubuntu/pool/main/g/gcc-4.8/libstdc++6_4.8.4-2ubuntu1~14.04.3_amd64.deb
$ dpkg -i libstdc++6_4.8.4-2ubuntu1~14.04.3_amd64.deb

Trying another fix, I found that I might be able to pass another CFLAG

diff --git a/makefiles/Makefile.unix.mk b/makefiles/Makefile.unix.mk
index c0ec077..2a8db3f 100644
--- a/makefiles/Makefile.unix.mk
+++ b/makefiles/Makefile.unix.mk
@@ -248,7 +248,7 @@ endif  # MAC OS X

 CFLAGS = $(DEBUG) -I$(INC_DIR) -I$(EX_DIR) -I$(GEN_DIR) $(GFLAGS_INC) $(ARCH) \
   -Wno-deprecated $(PROTOBUF_INC) $(CBC_INC) $(CLP_INC) $(GLPK_INC) \
-        $(SCIP_INC) $(GUROBI_INC) $(CPLEX_INC) -DUSE_GLOP -DUSE_BOP $(GLOG_INC)
+        $(SCIP_INC) $(GUROBI_INC) $(CPLEX_INC) -DUSE_GLOP -DUSE_BOP $(GLOG_INC) -D_GLIBCXX_USE_CXX11_ABI=0

 JNIFLAGS = $(JNIDEBUG) -I$(INC_DIR) -I$(EX_DIR) -I$(GEN_DIR) $(GFLAGS_INC) $(ARCH) \

But using wheel from that build gave the import error at the start of this post, and also seems to have broken the ortools installation on the build machine. So maybe this flag is a non-starter.

It seems like there is an incompatibility with the gcc on the desired machine & the one where the wheel was compiled. Do you have advice for ways to debug further? Is the ortools wheel not compatible with Trusty because of the glibc issue?

@lperron
Copy link
Collaborator

lperron commented Jun 1, 2017 via email

@mxr
Copy link
Author

mxr commented Jun 1, 2017 •
edited
Loading

Thanks @lperron, it's starting to make more sense to me now. We are really hoping to use the wheel since we use https://github.com/jazzband/pip-tools to catch dependency conflicts, and our infra is built around pip install, but maybe we'll have to add custom code to use the egg, and we'll have to catch the dependency conflicts another way.

Can you advise where I should insert the flags? I think I understand how to do it for libortools but not the _pywrapXXX libraries

--- a/makefiles/Makefile.unix.mk
+++ b/makefiles/Makefile.unix.mk
@@ -254,4 +254,4 @@ CFLAGS = $(DEBUG) -I$(INC_DIR) -I$(EX_DIR) -I$(GEN_DIR) $(GFLAGS_INC) $(ARCH) \
 JNIFLAGS = $(JNIDEBUG) -I$(INC_DIR) -I$(EX_DIR) -I$(GEN_DIR) $(GFLAGS_INC) $(ARCH) \
         -Wno-deprecated $(PROTOBUF_INC) $(CBC_INC) $(CLP_INC) $(GLPK_INC) $(SCIP_INC) $(GUROBI_INC) $(CPLEX_INC) -DUSE_GLOP -DUSE_BOP
 DEPENDENCIES_LNK = $(GLPK_LNK) $(CBC_LNK) $(CLP_LNK) $(SCIP_LNK) $(LM_LNK) $(GUROBI_LNK) $(CPLEX_LNK) $(GFLAGS_LNK) $(PROTOBUF_LNK) $(GLOG_LNK)
-OR_TOOLS_LD_FLAGS = $(ZLIB_LNK) $(SYS_LNK)
+OR_TOOLS_LD_FLAGS = $(ZLIB_LNK) $(SYS_LNK) -static -static-libgcc

@lperron
Copy link
Collaborator

lperron commented Jun 1, 2017 via email

@mxr
Copy link
Author

mxr commented Jun 1, 2017

@lperron wanted to note that I see there were some changes recently but my branch is still against 88bf683. Trying with the below changes:

diff --git a/makefiles/Makefile.python.mk b/makefiles/Makefile.python.mk
index 16ae0b8..80165a8 100755
--- a/makefiles/Makefile.python.mk
+++ b/makefiles/Makefile.python.mk
@@ -316,7 +316,7 @@ else
   ifeq ($(PLATFORM),MACOSX)
        cd $(PYPI_ARCHIVE_TEMP_DIR)/ortools && $(PYTHON_EXECUTABLE) setup.py bdist_egg bdist_wheel
   else
-       cd $(PYPI_ARCHIVE_TEMP_DIR)/ortools && $(PYTHON_EXECUTABLE) setup.py bdist_egg
+       cd $(PYPI_ARCHIVE_TEMP_DIR)/ortools && $(PYTHON_EXECUTABLE) setup.py bdist_egg bdist_wheel
   endif
 endif
        cd $(PYPI_ARCHIVE_TEMP_DIR)/ortools && twine upload dist/*
diff --git a/makefiles/Makefile.unix.mk b/makefiles/Makefile.unix.mk
index c0ec077..04a8f7c 100644
--- a/makefiles/Makefile.unix.mk
+++ b/makefiles/Makefile.unix.mk
@@ -253,4 +253,4 @@ CFLAGS = $(DEBUG) -I$(INC_DIR) -I$(EX_DIR) -I$(GEN_DIR) $(GFLAGS_INC) $(ARCH) \
 JNIFLAGS = $(JNIDEBUG) -I$(INC_DIR) -I$(EX_DIR) -I$(GEN_DIR) $(GFLAGS_INC) $(ARCH) \
         -Wno-deprecated $(PROTOBUF_INC) $(CBC_INC) $(CLP_INC) $(GLPK_INC) $(SCIP_INC) $(GUROBI_INC) $(CPLEX_INC) -DUSE_GLOP -DUSE_BOP
 DEPENDENCIES_LNK = $(GLPK_LNK) $(CBC_LNK) $(CLP_LNK) $(SCIP_LNK) $(LM_LNK) $(GUROBI_LNK) $(CPLEX_LNK) $(GFLAGS_LNK) $(PROTOBUF_LNK) $(GLOG_LNK)
-OR_TOOLS_LD_FLAGS = $(ZLIB_LNK) $(SYS_LNK)
+OR_TOOLS_LD_FLAGS = $(ZLIB_LNK) $(SYS_LNK) -static -static-libgcc

Gave the below error

g++ -shared \
  -o lib/libortools.so \
  objs/base/bitmap.o objs/base/callback.o objs/base/file.o objs/base/filelinereader.o objs/base/join.o objs/base/mutex.o objs/base/numbers.o objs/base/random.o objs/base/recordio.o objs/base/split.o objs/base/stringpiece.o objs/base/stringprintf.o objs/base/sysinfo.o objs/base/threadpool.o objs/base/timer.o objs/base/time_support.o \
  objs/util/bitset.o objs/util/cached_log.o objs/util/fp_utils.o objs/util/graph_export.o objs/util/piecewise_linear_function.o objs/util/proto_tools.o objs/util/range_query_function.o objs/util/rational_approximation.o objs/util/rcpsp_parser.o objs/util/sorted_interval_list.o objs/util/stats.o objs/util/time_limit.o objs/util/xml_helper.o \
  objs/lp_data/lp_data.o objs/lp_data/lp_decomposer.o objs/lp_data/lp_print_utils.o objs/lp_data/lp_types.o objs/lp_data/lp_utils.o objs/lp_data/matrix_scaler.o objs/lp_data/matrix_utils.o objs/lp_data/mps_reader.o objs/lp_data/proto_utils.o objs/lp_data/sparse.o objs/lp_data/sparse_column.o \
  objs/glop/basis_representation.o objs/glop/dual_edge_norms.o objs/glop/entering_variable.o objs/glop/initial_basis.o objs/glop/lp_solver.o objs/glop/lu_factorization.o objs/glop/markowitz.o objs/glop/preprocessor.o objs/glop/primal_edge_norms.o objs/glop/reduced_costs.o objs/glop/revised_simplex.o objs/glop/status.o objs/glop/update_row.o objs/glop/variables_info.o objs/glop/variable_values.o objs/glop/parameters.pb.o \
  objs/graph/assignment.o objs/graph/astar.o objs/graph/bellman_ford.o objs/graph/cliques.o objs/graph/dijkstra.o objs/graph/linear_assignment.o objs/graph/max_flow.o objs/graph/min_cost_flow.o objs/graph/shortestpaths.o objs/graph/util.o objs/graph/flow_problem.pb.o \
  objs/algorithms/dynamic_partition.o objs/algorithms/dynamic_permutation.o objs/algorithms/find_graph_symmetries.o objs/algorithms/hungarian.o objs/algorithms/knapsack_solver.o objs/algorithms/sparse_permutation.o \
  objs/sat/boolean_problem.o objs/sat/clause.o objs/sat/cp_constraints.o objs/sat/cp_model_checker.o objs/sat/cp_model_presolve.o objs/sat/cp_model_solver.o objs/sat/cp_model_utils.o objs/sat/cumulative.o objs/sat/disjunctive.o objs/sat/drat.o objs/sat/encoding.o objs/sat/flow_costs.o objs/sat/integer.o objs/sat/integer_expr.o objs/sat/intervals.o objs/sat/linear_programming_constraint.o objs/sat/lp_utils.o objs/sat/no_cycle.o objs/sat/optimization.o objs/sat/overload_checker.o objs/sat/pb_constraint.o objs/sat/precedences.o objs/sat/sat_solver.o objs/sat/simplification.o objs/sat/symmetry.o objs/sat/table.o objs/sat/theta_tree.o objs/sat/timetable.o objs/sat/timetable_edgefinding.o objs/sat/util.o objs/sat/boolean_problem.pb.o objs/sat/cp_model.pb.o objs/sat/sat_parameters.pb.o \
  objs/bop/bop_base.o objs/bop/bop_fs.o objs/bop/bop_lns.o objs/bop/bop_ls.o objs/bop/bop_portfolio.o objs/bop/bop_solution.o objs/bop/bop_solver.o objs/bop/bop_util.o objs/bop/complete_optimizer.o objs/bop/integral_solver.o objs/bop/bop_parameters.pb.o \
  objs/linear_solver/bop_interface.o objs/linear_solver/cbc_interface.o objs/linear_solver/clp_interface.o objs/linear_solver/cplex_interface.o objs/linear_solver/glop_interface.o objs/linear_solver/glop_utils.o objs/linear_solver/glpk_interface.o objs/linear_solver/gurobi_interface.o objs/linear_solver/linear_expr.o objs/linear_solver/linear_solver.o objs/linear_solver/model_exporter.o objs/linear_solver/model_validator.o objs/linear_solver/scip_interface.o objs/linear_solver/linear_solver.pb.o \
  objs/constraint_solver/ac4_mdd_reset_table.o objs/constraint_solver/ac4r_table.o objs/constraint_solver/alldiff_cst.o objs/constraint_solver/assignment.o objs/constraint_solver/collect_variables.o objs/constraint_solver/constraints.o objs/constraint_solver/constraint_solver.o objs/constraint_solver/count_cst.o objs/constraint_solver/default_search.o objs/constraint_solver/demon_profiler.o objs/constraint_solver/deviation.o objs/constraint_solver/diffn.o objs/constraint_solver/element.o objs/constraint_solver/expr_array.o objs/constraint_solver/expr_cst.o objs/constraint_solver/expressions.o objs/constraint_solver/gcc.o objs/constraint_solver/graph_constraints.o objs/constraint_solver/hybrid.o objs/constraint_solver/interval.o objs/constraint_solver/io.o objs/constraint_solver/local_search.o objs/constraint_solver/model_cache.o objs/constraint_solver/nogoods.o objs/constraint_solver/pack.o objs/constraint_solver/range_cst.o objs/constraint_solver/resource.o objs/constraint_solver/routing.o objs/constraint_solver/routing_flags.o objs/constraint_solver/routing_neighborhoods.o objs/constraint_solver/routing_search.o objs/constraint_solver/sat_constraint.o objs/constraint_solver/sched_constraints.o objs/constraint_solver/sched_expr.o objs/constraint_solver/sched_search.o objs/constraint_solver/search.o objs/constraint_solver/softgcc.o objs/constraint_solver/table.o objs/constraint_solver/timetabling.o objs/constraint_solver/trace.o objs/constraint_solver/tree_monitor.o objs/constraint_solver/utilities.o objs/constraint_solver/visitor.o objs/constraint_solver/assignment.pb.o objs/constraint_solver/demon_profiler.pb.o objs/constraint_solver/model.pb.o objs/constraint_solver/routing_enums.pb.o objs/constraint_solver/routing_parameters.pb.o objs/constraint_solver/search_limit.pb.o objs/constraint_solver/solver_parameters.pb.o \
   /home/maxr/src/or-tools/dependencies/install/lib/libCbcSolver.a /home/maxr/src/or-tools/dependencies/install/lib/libCbc.a /home/maxr/src/or-tools/dependencies/install/lib/libCgl.a /home/maxr/src/or-tools/dependencies/install/lib/libOsi.a /home/maxr/src/or-tools/dependencies/install/lib/libOsiCbc.a /home/maxr/src/or-tools/dependencies/install/lib/libOsiClp.a /home/maxr/src/or-tools/dependencies/install/lib/libClp.a /home/maxr/src/or-tools/dependencies/install/lib/libCoinUtils.a     /home/maxr/src/or-tools/dependencies/install/lib/libgflags.a /home/maxr/src/or-tools/dependencies/install/lib/libprotobuf.a /home/maxr/src/or-tools/dependencies/install/lib/libglog.a \
  -lz -lrt -lpthread -static -static-libgcc
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/5/crtbeginT.o: relocation R_X86_64_32 against `__TMC_END__' can not be used when making a shared object; recompile with -fPIC
/usr/lib/gcc/x86_64-linux-gnu/5/crtbeginT.o: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
makefiles/Makefile.cpp.mk:628: recipe for target 'lib/libortools.so' failed
make: *** [lib/libortools.so] Error 1

Should I add -fPIC to LINK_CMD in Makefile.unix.mk and retry, or another flag/config? Should I pass on the current route and try the manylinux approach you mentioned? Thanks for all the help so far!

@lperron
Copy link
Collaborator

lperron commented Jun 1, 2017 via email

@lperron
Copy link
Collaborator

lperron commented Jun 1, 2017 via email

@mxr
Copy link
Author

mxr commented Jun 1, 2017 •
edited
Loading

I'm also able to compile once passing the required flags to make python, however, producing a wheel on Xenial that works on Trusty continues to have some issues.

re: egg, I tried installing the egg from pypi on Trusty and that fails with a similar error

$ easy_install ortools
Searching for ortools
Reading https://pypi.python.org/simple/ortools/
Downloading https://pypi.python.org/packages/27/49/f7546bdea4aaf05722b6a7796ee2fd69272ea869334720b04703ec29274d/ortools-6.0.4217-py2.7-linux-x86_64.egg#md5=4eb71b07792640f26b21e3470ab5c19f
Best match: ortools 6.0.4217
Processing ortools-6.0.4217-py2.7-linux-x86_64.egg
creating /usr/local/lib/service_venv/lib/python2.7/site-packages/ortools-6.0.4217-py2.7-linux-x86_64.egg
Extracting ortools-6.0.4217-py2.7-linux-x86_64.egg to /usr/local/lib/service_venv/lib/python2.7/site-packages
Adding ortools 6.0.4217 to easy-install.pth file

Installed /usr/local/lib/service_venv/lib/python2.7/site-packages/ortools-6.0.4217-py2.7-linux-x86_64.egg
Processing dependencies for ortools
Finished processing dependencies for ortools

$ python
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/service_venv/local/lib/python2.7/site-packages/ortools-6.0.4217-py2.7-linux-x86_64.egg/ortools/graph/pywrapgraph.py", line 17, in <module>
    _pywrapgraph = swig_import_helper()
  File "/usr/local/lib/service_venv/local/lib/python2.7/site-packages/ortools-6.0.4217-py2.7-linux-x86_64.egg/ortools/graph/pywrapgraph.py", line 16, in swig_import_helper
    return importlib.import_module('_pywrapgraph')
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: No module named _pywrapgraph

Is that expected? The README says Trusty is supported, but maybe that's only if you build from source?

@lperron
Copy link
Collaborator

lperron commented Jun 2, 2017 via email

@mxr
Copy link
Author

mxr commented Jun 2, 2017 •
edited
Loading

@lperron good news, I was able to build an egg (and a wheel!) on Trusty that didn't have the import error by running a combination of the steps above. However we are wary to use/host the wheel I built internally, because it depends on me building it locally with custom work. Specifically here is what I had to work around to build a wheel on Trusty that I could use on a different machine:

  1. I still had trouble with patchelf. I had to manually do make dependencies/install/bin/patchelf. Without this, I see errors that say dependencies/install/bin/patchelf: not found and the produces egg gives the No module named _pywrapgraph error. But an egg and wheel with this change work fine. So I think patchelf is not being built right, at least on Trusty.
  2. bdist_wheel is not an option to setup.py for Linux. I remember it was tried in d2cd773 and it caused other issues. We would need to re-attempt that change.
  3. The egg on pypi is built on Xenial which isn't compatible with Trusty. We would need a build to be produced on Trusty & uploaded that to pypi.
  4. HEAD produces a build that depends on protobuf==3.3.0. We are still on 3.2.0 so being able to produce a Python library that uses >=3.2.0 would be great for us.

Would fixing these issues and uploading a wheel to pypi be on your roadmap? We can also try to contribute upstream fixes for these issues (though we might need some guidance)

Thanks!

@lperron
Copy link
Collaborator

lperron commented Jul 14, 2017

Hi,

Please check my last mail on docker support.
You can now build python2 and python3 egg and wheel file for ubuntu 14.04, 16.04 and 17.04 with little effort.

This should simplify everything a lot.

@lperron
Copy link
Collaborator

lperron commented Aug 1, 2017

I have pushed version 6.3 with support for multiple linux distributions.
I have changed the name of the modules to reflect the linux distribution.
So you can install ortools-ubuntu-14.04, ortools-ubuntu-16.04 on your machines.

Thanks

@Mizux Mizux self-assigned this Jan 9, 2018
@Mizux Mizux added the Bug label Jan 9, 2018
@Mizux Mizux added OS: Linux GNU/Linux OS Lang: Python Python wrapper issue labels Jan 31, 2018
@Mizux Mizux closed this as completed Jan 31, 2018
@Mizux Mizux added this to the v8.0 milestone Feb 12, 2021
@google google deleted a comment from robin48gx Sep 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Mizux Mizux

Labels
Bug Lang: Python Python wrapper issue OS: Linux GNU/Linux OS
Projects
None yet
Milestone
v8.0
Development

No branches or pull requests
4 participants
@mxr @Mizux @lperron @diegoalberione