Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S2 compatible with python3? #46

Closed
winash12 opened this issue Sep 13, 2018 · 27 comments

Comments

@winash12
Copy link

@winash12 winash12 commented Sep 13, 2018

I followed the instructions as given here - http://s2geometry.io/about/platforms.html but I am getting these errors when I run cmake

-- The following features have been enabled:

  • GFLAGS, allows changing command line flags.
  • GLOG, provides logging configurability.
  • SHARED_LIBS, builds shared libraries instead of static.

GTEST_ROOT: /usr/src/googletest
File "/usr/lib/python3.6/site.py", line 183
file=sys.stderr)
^
SyntaxError: invalid syntax
CMake Error at src/python/CMakeLists.txt:30 (install):
install TARGETS given no LIBRARY DESTINATION for module target "_pywraps2".

CMake Error at src/python/CMakeLists.txt:31 (install):
install FILES given no DESTINATION!

-- Configuring incomplete, errors occurred!
See also "/home/winash12/S2source/s2geometry/CMakeFiles/CMakeOutput.log".
See also "/home/winash12/S2source/s2geometry/CMakeFiles/CMakeError.log".
This is on Ubuntu 18.04 LTS.

@jmr

This comment has been minimized.

Copy link
Member

@jmr jmr commented Sep 13, 2018

What did cmake say for "Found PythonInterp" and "Found PythonLibs"? It looks like there's a python2 vs 3 problem.

@winash12

This comment has been minimized.

Copy link
Author

@winash12 winash12 commented Sep 14, 2018

I have PYTHONPATH defined and I have only used the default Ubuntu Python installation. Not added anything of my own. How can I check for Found PythonInterp with cmake ? Right now it is saying anything regarding that.

@jmr

This comment has been minimized.

Copy link
Member

@jmr jmr commented Sep 14, 2018

Just paste the entire CMake output, there will be lines like "Found PythonInterp: ..." in there.

What do you have $PYTHONPATH set to?

@winash12

This comment has been minimized.

Copy link
Author

@winash12 winash12 commented Sep 14, 2018

PYTHONPATH is set to /usr/lib/python3.6

Whatever I gave you i.e. the cmake output was the result of running cmake . in the build directory.
There was no "Found PythonInterp:" at all in it.

@jmr

This comment has been minimized.

Copy link
Member

@jmr jmr commented Sep 14, 2018

Can you:

  1. Paste the entire CMake output, starting with an empty build/ directory?
  2. Try again without setting $PYTHONPATH, and tell me if that works.
@winash12

This comment has been minimized.

Copy link
Author

@winash12 winash12 commented Sep 14, 2018

When I unset PYTHONPATH CMake went through without any problems but it is using Python 2.7.15. Is that OK ?
Now I am proceeding to make.

@jmr

This comment has been minimized.

Copy link
Member

@jmr jmr commented Sep 14, 2018

Yes, that's ok.

What does python --version say for you? If it says "Python 2.7.15", then it's probably not a good idea to point PYTHONPATH at a python3 directory.

@winash12

This comment has been minimized.

Copy link
Author

@winash12 winash12 commented Sep 14, 2018

It says 2.7.15rc1. Does this mean I cannot use S2 geometry within Python3.6 scripts ?

@jmr

This comment has been minimized.

Copy link
Member

@jmr jmr commented Sep 14, 2018

You might be the first one to try to use s2 with python3.

To try this, you could either:

  1. Use update-alternatives to point python at python3.
  2. Add something like set(Python_ADDITIONAL_VERSIONS 3.6) to CMakeLists.txt before the find_package() calls.
@jmr jmr changed the title Unable to compile S2 S2 compatible with python3? Sep 14, 2018
@winash12

This comment has been minimized.

Copy link
Author

@winash12 winash12 commented Sep 15, 2018

I added it like this in CMakeLists.txt
set(Python_ADDITIONAL_VERSIONS 3.6)
find_package(OpenSSL REQUIRED)

pthreads isn't used directly, but this is still required for std::thread.

find_package(Threads REQUIRED)
find_package(SWIG)
find_package(PythonInterp)
find_package(PythonLibs)
-- The following features have been enabled:

  • GFLAGS, allows changing command line flags.
  • GLOG, provides logging configurability.
  • SHARED_LIBS, builds shared libraries instead of static.

GTEST_ROOT: /usr/src/googletest
File "/usr/lib/python3.6/site.py", line 183
file=sys.stderr)
^
SyntaxError: invalid syntax
CMake Error at src/python/CMakeLists.txt:30 (install):
install TARGETS given no LIBRARY DESTINATION for module target "_pywraps2".

CMake Error at src/python/CMakeLists.txt:31 (install):
install FILES given no DESTINATION!

-- Configuring incomplete, errors occurred!
See also "/home/aswin/S2source/s2geometry/CMakeFiles/CMakeOutput.log".
See also "/home/aswin/S2source/s2geometry/CMakeFiles/CMakeError.log".

Do you want both the log files ?

@winash12

This comment has been minimized.

Copy link
Author

@winash12 winash12 commented Sep 16, 2018

This comment by Al Piszcz suggests that I am not the first to try S2 with Python3 - http://blog.christianperone.com/2015/08/googles-s2-geometry-on-the-sphere-cells-and-hilbert-curve/

import s2sphere as s2

import matplotlib.pyplot as plt
from shapely.geometry import Polygon
import cartopy.crs as ccrs
import cartopy.io.img_tiles as cimgt
@jmr

This comment has been minimized.

Copy link
Member

@jmr jmr commented Sep 17, 2018

Do you want both the log files ?

Yes, please.

This comment by Al Piszcz suggests that I am not the first to try S2 with Python3 - http://blog.christianperone.com/2015/08/googles-s2-geometry-on-the-sphere-cells-and-hilbert-curve/#comment-117962

I'm not sure what he did to make it work with python 3.

Maybe try to put the version numbers in the find_package instead:
https://stackoverflow.com/a/34580995

find_package(PythonInterp 3.6)
find_package(PythonLibs 3.6)
@akhenakh

This comment has been minimized.

Copy link

@akhenakh akhenakh commented Sep 17, 2018

I've been using s2 with Python 3 for months without any modifications.

Arch defaulted to Python3 month ago https://aur.archlinux.org/packages/s2geometry-git

Python 3.7.0 (default, Jul 15 2018, 10:44:58) 
[GCC 8.1.1 20180531] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pywraps2 as s2

Since it's working I assume this is just an issue with cmake detecting the wrong interpreter, you can probably trick it with -DPYTHON_EXECUTABLE:FILEPATH

@jmr

This comment has been minimized.

Copy link
Member

@jmr jmr commented Sep 17, 2018

@akhenakh Thanks. Is CMake finding python2 or 3 for you? If CMake finds python2, will it be possible to use pywraps2 with python3?

@akhenakh

This comment has been minimized.

Copy link

@akhenakh akhenakh commented Sep 17, 2018

cmake is finding my default python interpreter which is Python3.
pywraps2 must be compiled with Python3 to be loaded in Python 3.

You can force it with: cmake -DPYTHON_EXECUTABLE=/usr/bin/python3

@winash12

This comment has been minimized.

Copy link
Author

@winash12 winash12 commented Sep 18, 2018

@akhenakh Congratulations ! That worked smoothly. It all works with Python3.6 and I had to install s2sphere but after that it is all a home run. Thanks to @jmr and @akhenakh for their excellent support !

@jmr

This comment has been minimized.

Copy link
Member

@jmr jmr commented Sep 18, 2018

I'm confused. s2sphere is a Python reimplementation of a subset of this library. You shouldn't need s2geometry at all if you use s2sphere.

@winash12

This comment has been minimized.

Copy link
Author

@winash12 winash12 commented Sep 19, 2018

I do not know. I was just implementing that example as is given in that external blog as is.

@akhenakh

This comment has been minimized.

Copy link

@akhenakh akhenakh commented Sep 19, 2018

s2sphere is an implementation in Python (at the time I've checked it was not relying on the c++ lib and was incompatible in term of results)

You should import import pywraps2 as s2 or the whole thing you compiled is useless

@winash12

This comment has been minimized.

Copy link
Author

@winash12 winash12 commented Sep 20, 2018

Excellent thanks.

@winash12 winash12 closed this Sep 20, 2018
@beyoung

This comment has been minimized.

Copy link

@beyoung beyoung commented Jun 13, 2019

I have made a docker image s2geometry_docker for s2 compatible with python3 @winash12 , you can download docker image by command docker pull beyouth/s2geometry @winash12

@winash12 winash12 reopened this Aug 30, 2019
@winash12 winash12 closed this Aug 30, 2019
@winash12 winash12 reopened this Aug 30, 2019
@winash12

This comment has been minimized.

Copy link
Author

@winash12 winash12 commented Aug 30, 2019

Sorry to reopen this after a year but I am getting a ModuleNotFoundError when I type

import pywraps2 as s2

This is the output of my make install command and I think it is putting pywraps2 in the wrong directory. How can I correct it by mentioning the right -DCMAKE_INSTALL_PREFIX ?

My python3.6 installation directory is here
/usr/local/lib/python3.6/dist-packages and that is where my egg files are located.

My CMAKE build command went like this

cmake ../ _DPYTHON_EXECUTABLE=/usr/bin/python3.6 -DCMAKE_INSTALL_PREFIX=/usr/local/lib/python3.6

Is that incorrect ? Apparently yes because I am still getting a ModuleNotFoundError.

-- Up-to-date: /usr/local/lib/python3.6/include/s2/util/bits/bits.h -- Up-to-date: /usr/local/lib/python3.6/include/s2/util/coding/coder.h -- Up-to-date: /usr/local/lib/python3.6/include/s2/util/coding/varint.h -- Up-to-date: /usr/local/lib/python3.6/include/s2/util/endian/endian.h -- Up-to-date: /usr/local/lib/python3.6/include/s2/util/gtl/btree.h -- Up-to-date: /usr/local/lib/python3.6/include/s2/util/gtl/btree_container.h -- Up-to-date: /usr/local/lib/python3.6/include/s2/util/gtl/btree_map.h -- Up-to-date: /usr/local/lib/python3.6/include/s2/util/gtl/btree_set.h -- Up-to-date: /usr/local/lib/python3.6/include/s2/util/gtl/compact_array.h -- Up-to-date: /usr/local/lib/python3.6/include/s2/util/gtl/container_logging.h -- Up-to-date: /usr/local/lib/python3.6/include/s2/util/gtl/dense_hash_set.h -- Up-to-date: /usr/local/lib/python3.6/include/s2/util/gtl/densehashtable.h -- Up-to-date: /usr/local/lib/python3.6/include/s2/util/gtl/hashtable_common.h -- Up-to-date: /usr/local/lib/python3.6/include/s2/util/gtl/layout.h -- Up-to-date: /usr/local/lib/python3.6/include/s2/util/hash/mix.h -- Up-to-date: /usr/local/lib/python3.6/include/s2/util/math/mathutil.h -- Up-to-date: /usr/local/lib/python3.6/include/s2/util/math/matrix3x3.h -- Up-to-date: /usr/local/lib/python3.6/include/s2/util/math/vector.h -- Up-to-date: /usr/local/lib/python3.6/include/s2/util/math/vector3_hash.h -- Up-to-date: /usr/local/lib/python3.6/include/s2/util/units/length-units.h -- Up-to-date: /usr/local/lib/python3.6/include/s2/util/units/physical-units.h -- Up-to-date: /usr/local/lib/python3.6/lib/libs2.so -- Up-to-date: /usr/local/lib/python3.6/lib/libs2testing.a -- Up-to-date: /usr/local/lib/python3.6/lib/python3/dist-packages/_pywraps2.so -- Up-to-date: /usr/local/lib/python3.6/lib/python3/dist-packages/pywraps2.py

@jwhiting

This comment has been minimized.

Copy link

@jwhiting jwhiting commented Sep 10, 2019

@winash12 I just opened a related issue, #83 - the problem I found was that the default make install prefix put the libs2.so file in the wrong place, but then changing the prefix then puts the pywraps2 module in the wrong place. The workaround I am currently using is to set the make install prefix to just /usr (which makes libs2.so discoverable) and then adding /usr/local/lib/python3.7/site-packages to PYTHONPATH environment argument before launching python (which makes the module discoverable). I'm on ubuntu linux w/ the python 3 environment as built by the official docker team. FWIW my functional dockerfile is here https://gist.github.com/jwhiting/05d20afab735355fb0c3972f3f8ad910

@winash12

This comment has been minimized.

Copy link
Author

@winash12 winash12 commented Sep 12, 2019

@jwhiting In my case libs2.so is already present under /usrlocal/lib. So I am not sure how this helps me. I am indeed using Ubuntu Linux 18.04 LTS. So the key question is location of pywraps module. I think I will ask this as a question on Unix Stack Exchange.

@winash12

This comment has been minimized.

Copy link
Author

@winash12 winash12 commented Sep 12, 2019

@jwhiting Wait a moment. Are you saying libs2.so needs to be under /usr and NOT /usr/local/lib ? And that will put pywraps2 at the right location ? I am totally confused now.

@dmarcous

This comment has been minimized.

Copy link

@dmarcous dmarcous commented Sep 14, 2019

Script I used to install on debian 9 with python 3.6 managed by conda:

#!/bin/bash

pip install --upgrade pip

sudo apt-get install libgflags-dev libgoogle-glog-dev libgtest-dev libssl-dev -y
sudo apt-get install swig -y
sudo apt-get install cmake -y

mkdir Source
cd Source
git clone https://github.com/google/s2geometry.git
cd s2geometry
mkdir build
cd build

sudo cmake -DCMAKE_INSTALL_PREFIX=/usr/lib/python/ -DGTEST_ROOT=/usr/src/gtest
-DPYTHON_INCLUDE_DIR=$(python3 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())")
-DPYTHON_LIBRARY=$(python3 -c "import distutils.sysconfig as sysconfig; print(sysconfig.get_config_var('LIBDIR'))")
-DPYTHON_EXECUTABLE:FILEPATH=which python3 ..

sudo make
sudo make install

cp /usr/lib/python/lib/python3.6/site-packages/_pywraps2.so /opt/conda/default/lib/python3.6/site-packages
cp /usr/lib/python/lib/python3.6/site-packages/pywraps2.py /opt/conda/default/lib/python3.6/site-packages
cp /usr/lib/python/lib/libs2.so /opt/conda/default/lib

@jmr

This comment has been minimized.

Copy link
Member

@jmr jmr commented Sep 20, 2019

Let's keep the discussion in #83; pwraps2 is compatible with python3.

@jmr jmr closed this Sep 20, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.