New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segfault with shapely operations using fiona and shapely wheels #20

Closed
perrygeo opened this Issue Oct 26, 2017 · 10 comments

Comments

Projects
None yet
2 participants
@perrygeo
Contributor

perrygeo commented Oct 26, 2017

@sgillies I'm getting a segfault on OSX using the latest Python 3.6 wheels for Fiona and Shapely. Here's the steps to reproduce:

Set up a virtual env with wheels

⭆ virtualenv -p python3.6 /tmp/segfault
⭆ source /tmp/segfault/bin/activate
⭆ pip install shapely fiona
Collecting shapely
  Downloading Shapely-1.6.2-cp36-cp36m-macosx_10_6_intel.whl (2.8MB)
Collecting fiona
  Using cached Fiona-1.7.9.post1-cp36-cp36m-macosx_10_6_intel.whl
...

Run the following python script

#!/usr/bin/env python3.6

# import fiona
from shapely.geometry import shape

a = shape(
    {'type': 'Polygon',
     'coordinates': ((
         (-179.95983122707207, 85.04766197798072),
         (-179.95983122707207, 85.04321428424322),
         (-179.90833283542793, 85.04321428424322),
         (-179.90833283542793, 85.04766197798072),
         (-179.95983122707207, 85.04766197798072)),)})

b = shape(
    {'type': 'Polygon',
     'coordinates': ((
         (-179.9222362115971, 85.04614454850956),
         (-179.9222362115971, 85.04728275377343),
         (-179.93883444063917, 85.04728275377343),
         (-179.93883444063917, 85.04614454850956),
         (-179.9222362115971, 85.04614454850956)),)})


print("Is b within a?")
print(b.within(a))

This works as expected. But if we import fiona before the shapely import, we get a Segmentation fault: 11.

Importing Fiona after Shapely seems to work. And when either Fiona or Shapely is installed from source (e.g. with pip install --no-binary :all: fiona shapely) the segfault disappears. So it appears to be some interaction between Shapely and Fiona wheels.

I haven't been able to reproduce on Linux yet, OSX only so far.

@sgillies

This comment has been minimized.

Show comment
Hide comment
@sgillies

sgillies Oct 26, 2017

Owner

@perrygeo the shapely and fiona wheels have different versions of GEOS. I'm trying to make this a non issue by statically linking GEOS in GDAL. See how there's no GEOS libs in the Fiona wheels:

$ tar tzvf dist/Fiona-1.7.10-cp36-cp36m-macosx_10_6_intel.whl | grep dylib
-rwxrwxrwx  0 0      0    58239176 Oct 26 16:24 fiona/.dylibs/libgdal.20.dylib
-rwxrwxrwx  0 0      0     6784404 Oct 26 16:24 fiona/.dylibs/libhdf5.101.dylib
-rwxrwxrwx  0 0      0      271552 Oct 26 16:24 fiona/.dylibs/libhdf5_hl.100.dylib
-rwxrwxrwx  0 0      0      119600 Oct 26 16:24 fiona/.dylibs/libjson-c.2.dylib
-rwxrwxrwx  0 0      0     3023200 Oct 26 16:24 fiona/.dylibs/libnetcdf.11.dylib
-rwxrwxrwx  0 0      0      641420 Oct 26 16:24 fiona/.dylibs/libopenjp2.2.3.0.dylib
-rwxrwxrwx  0 0      0     1031492 Oct 26 16:24 fiona/.dylibs/libproj.12.dylib

Clearly it remains an issue and when shapely is imported first and dlopens a GEOS shared lib that is not compatible with Fiona's GDAL 💥

The upcoming Fiona 1.7.10 wheels will avoid this problem, but will not truly solve it.

Owner

sgillies commented Oct 26, 2017

@perrygeo the shapely and fiona wheels have different versions of GEOS. I'm trying to make this a non issue by statically linking GEOS in GDAL. See how there's no GEOS libs in the Fiona wheels:

$ tar tzvf dist/Fiona-1.7.10-cp36-cp36m-macosx_10_6_intel.whl | grep dylib
-rwxrwxrwx  0 0      0    58239176 Oct 26 16:24 fiona/.dylibs/libgdal.20.dylib
-rwxrwxrwx  0 0      0     6784404 Oct 26 16:24 fiona/.dylibs/libhdf5.101.dylib
-rwxrwxrwx  0 0      0      271552 Oct 26 16:24 fiona/.dylibs/libhdf5_hl.100.dylib
-rwxrwxrwx  0 0      0      119600 Oct 26 16:24 fiona/.dylibs/libjson-c.2.dylib
-rwxrwxrwx  0 0      0     3023200 Oct 26 16:24 fiona/.dylibs/libnetcdf.11.dylib
-rwxrwxrwx  0 0      0      641420 Oct 26 16:24 fiona/.dylibs/libopenjp2.2.3.0.dylib
-rwxrwxrwx  0 0      0     1031492 Oct 26 16:24 fiona/.dylibs/libproj.12.dylib

Clearly it remains an issue and when shapely is imported first and dlopens a GEOS shared lib that is not compatible with Fiona's GDAL 💥

The upcoming Fiona 1.7.10 wheels will avoid this problem, but will not truly solve it.

@sgillies sgillies closed this Oct 26, 2017

@perrygeo

This comment has been minimized.

Show comment
Hide comment
@perrygeo

perrygeo Oct 27, 2017

Contributor

@sgillies still hitting the segfault with Fiona 1.7.10

⭆ pip install shapely fiona
Collecting shapely
  Using cached Shapely-1.6.2-cp36-cp36m-macosx_10_6_intel.whl
Collecting fiona
  Using cached Fiona-1.7.10-cp36-cp36m-macosx_10_6_intel.whl
...
⭆ python try.py
Is b within a?
Segmentation fault: 11
Contributor

perrygeo commented Oct 27, 2017

@sgillies still hitting the segfault with Fiona 1.7.10

⭆ pip install shapely fiona
Collecting shapely
  Using cached Shapely-1.6.2-cp36-cp36m-macosx_10_6_intel.whl
Collecting fiona
  Using cached Fiona-1.7.10-cp36-cp36m-macosx_10_6_intel.whl
...
⭆ python try.py
Is b within a?
Segmentation fault: 11
@sgillies

This comment has been minimized.

Show comment
Hide comment
@sgillies

sgillies Oct 27, 2017

Owner

Argh.

Here's the stack trace using Shapely 1.6.2 and Fiona 1.7.10. GEOS 3.6.2.

$ lldb python
(lldb) target create "python"
Current executable set to 'python' (x86_64).
(lldb) run try.py
Process 33823 launched: '/Users/sean/envs/162_1710/bin/python' (x86_64)
Is b within a?
Process 33823 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x7fff5afffff8)
    frame #0: 0x00007fffd934fdea libc++abi.dylib`__cxxabiv1::__vmi_class_type_info::search_below_dst(__cxxabiv1::__dynamic_cast_info*, void const*, int, bool) const + 12
libc++abi.dylib`__cxxabiv1::__vmi_class_type_info::search_below_dst:
->  0x7fffd934fdea <+12>: pushq  %rbx
    0x7fffd934fdeb <+13>: subq   $0x18, %rsp
    0x7fffd934fdef <+17>: movl   %r8d, %r13d
    0x7fffd934fdf2 <+20>: movl   %ecx, -0x2c(%rbp)
Target 0: (python) stopped.
(lldb) up
frame #1: 0x00007fffd934feaa libc++abi.dylib`__cxxabiv1::__vmi_class_type_info::search_below_dst(__cxxabiv1::__dynamic_cast_info*, void const*, int, bool) const + 204
libc++abi.dylib`__cxxabiv1::__vmi_class_type_info::search_below_dst:
    0x7fffd934feaa <+204>: addq   $0x10, %rbx
    0x7fffd934feae <+208>: cmpq   %r12, %rbx
    0x7fffd934feb1 <+211>: jb     0x7fffd934fe8b            ; <+173>
    0x7fffd934feb3 <+213>: jmp    0x7fffd9350066            ; <+648>
(lldb) up
frame #2: 0x00007fffd934fb2c libc++abi.dylib`__dynamic_cast + 281
libc++abi.dylib`__dynamic_cast:
    0x7fffd934fb2c <+281>: movq   -0x48(%rbp), %rax
    0x7fffd934fb30 <+285>: movq   %rax, %rcx
    0x7fffd934fb33 <+288>: shrq   $0x20, %rcx
    0x7fffd934fb37 <+292>: orl    %eax, %ecx
(lldb) up
frame #3: 0x00000001043766dc libgdal.20.dylib`geos::operation::predicate::RectangleContains::isContainedInBoundary(geos::geom::Geometry const&) + 92
libgdal.20.dylib`geos::operation::predicate::RectangleContains::isContainedInBoundary:
    0x1043766dc <+92>:  movq   %rax, -0x58(%rbp)
    0x1043766e0 <+96>:  jmp    0x1043766f2               ; <+114>
    0x1043766e5 <+101>: xorl   %eax, %eax
    0x1043766e7 <+103>: movl   %eax, %ecx

I'm going to find Shapely and Fiona versions using a different GEOS.

Owner

sgillies commented Oct 27, 2017

Argh.

Here's the stack trace using Shapely 1.6.2 and Fiona 1.7.10. GEOS 3.6.2.

$ lldb python
(lldb) target create "python"
Current executable set to 'python' (x86_64).
(lldb) run try.py
Process 33823 launched: '/Users/sean/envs/162_1710/bin/python' (x86_64)
Is b within a?
Process 33823 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x7fff5afffff8)
    frame #0: 0x00007fffd934fdea libc++abi.dylib`__cxxabiv1::__vmi_class_type_info::search_below_dst(__cxxabiv1::__dynamic_cast_info*, void const*, int, bool) const + 12
libc++abi.dylib`__cxxabiv1::__vmi_class_type_info::search_below_dst:
->  0x7fffd934fdea <+12>: pushq  %rbx
    0x7fffd934fdeb <+13>: subq   $0x18, %rsp
    0x7fffd934fdef <+17>: movl   %r8d, %r13d
    0x7fffd934fdf2 <+20>: movl   %ecx, -0x2c(%rbp)
Target 0: (python) stopped.
(lldb) up
frame #1: 0x00007fffd934feaa libc++abi.dylib`__cxxabiv1::__vmi_class_type_info::search_below_dst(__cxxabiv1::__dynamic_cast_info*, void const*, int, bool) const + 204
libc++abi.dylib`__cxxabiv1::__vmi_class_type_info::search_below_dst:
    0x7fffd934feaa <+204>: addq   $0x10, %rbx
    0x7fffd934feae <+208>: cmpq   %r12, %rbx
    0x7fffd934feb1 <+211>: jb     0x7fffd934fe8b            ; <+173>
    0x7fffd934feb3 <+213>: jmp    0x7fffd9350066            ; <+648>
(lldb) up
frame #2: 0x00007fffd934fb2c libc++abi.dylib`__dynamic_cast + 281
libc++abi.dylib`__dynamic_cast:
    0x7fffd934fb2c <+281>: movq   -0x48(%rbp), %rax
    0x7fffd934fb30 <+285>: movq   %rax, %rcx
    0x7fffd934fb33 <+288>: shrq   $0x20, %rcx
    0x7fffd934fb37 <+292>: orl    %eax, %ecx
(lldb) up
frame #3: 0x00000001043766dc libgdal.20.dylib`geos::operation::predicate::RectangleContains::isContainedInBoundary(geos::geom::Geometry const&) + 92
libgdal.20.dylib`geos::operation::predicate::RectangleContains::isContainedInBoundary:
    0x1043766dc <+92>:  movq   %rax, -0x58(%rbp)
    0x1043766e0 <+96>:  jmp    0x1043766f2               ; <+114>
    0x1043766e5 <+101>: xorl   %eax, %eax
    0x1043766e7 <+103>: movl   %eax, %ecx

I'm going to find Shapely and Fiona versions using a different GEOS.

@sgillies

This comment has been minimized.

Show comment
Hide comment
@sgillies

sgillies Oct 27, 2017

Owner

Same problem as above with Shapely 1.6.0 + Fiona 1.7.9.post1 (frs-wheel-builds 1.4.0 – GEOS 3.4.2). At least with Python 3.6. And Python 3.5.

I can't blame GEOS. Manual bisection continues...

Owner

sgillies commented Oct 27, 2017

Same problem as above with Shapely 1.6.0 + Fiona 1.7.9.post1 (frs-wheel-builds 1.4.0 – GEOS 3.4.2). At least with Python 3.6. And Python 3.5.

I can't blame GEOS. Manual bisection continues...

@sgillies

This comment has been minimized.

Show comment
Hide comment
@sgillies

sgillies Oct 27, 2017

Owner

Results of analysis of Python 2.7 macosx wheels on PyPI are in. I'm almost certain they apply to the other Python versions.

The first (in time) wheel combination that breaks is Shapely 1.6b5 and Fiona 1.7.7. Shapely 1.6b4 works with all Fiona wheels from 1.7.1 to 1.7.10. Fiona 1.7.6 works with all Shapely wheels from 1.6b4 to 1.6.2.

Now to track down the actual build bug now that we've found where it lives in time.

Owner

sgillies commented Oct 27, 2017

Results of analysis of Python 2.7 macosx wheels on PyPI are in. I'm almost certain they apply to the other Python versions.

The first (in time) wheel combination that breaks is Shapely 1.6b5 and Fiona 1.7.7. Shapely 1.6b4 works with all Fiona wheels from 1.7.1 to 1.7.10. Fiona 1.7.6 works with all Shapely wheels from 1.6b4 to 1.6.2.

Now to track down the actual build bug now that we've found where it lives in time.

@sgillies

This comment has been minimized.

Show comment
Hide comment
@sgillies

sgillies Oct 27, 2017

Owner

The most recent change to frs-wheel-builds before Shapely 1.6b5 or Fiona 1.7.7 is this: https://github.com/sgillies/frs-wheel-builds/blob/master/CHANGES.txt#L18-L28. My hypothesis is that it's the macosx deployment target. I'm testing by building Fiona 1.7.10 using the old 10.6 target (if possible.

Owner

sgillies commented Oct 27, 2017

The most recent change to frs-wheel-builds before Shapely 1.6b5 or Fiona 1.7.7 is this: https://github.com/sgillies/frs-wheel-builds/blob/master/CHANGES.txt#L18-L28. My hypothesis is that it's the macosx deployment target. I'm testing by building Fiona 1.7.10 using the old 10.6 target (if possible.

@sgillies

This comment has been minimized.

Show comment
Hide comment
@sgillies

sgillies Oct 27, 2017

Owner

Good news! GDAL 2.2.2 builds with XCode 8.2.1 MACOSX_DEPLOYMENT_TARGET=10.6 and --without-cp11 where GDAL 2.2.0 did not.

In hindsight, I should have checked on this much earlier, like at GDAL 2.2.1.

Owner

sgillies commented Oct 27, 2017

Good news! GDAL 2.2.2 builds with XCode 8.2.1 MACOSX_DEPLOYMENT_TARGET=10.6 and --without-cp11 where GDAL 2.2.0 did not.

In hindsight, I should have checked on this much earlier, like at GDAL 2.2.1.

@sgillies

This comment has been minimized.

Show comment
Hide comment
@sgillies

sgillies Oct 27, 2017

Owner

Same crash with the MACOSX_DEPLOYMENT_TARGET=10.6 wheels :(.

Next thing to try is reverting to GDAL 2.1.3.

Note: I tried GDAL 2.1.4 and hit a build bug involving isnan.

Owner

sgillies commented Oct 27, 2017

Same crash with the MACOSX_DEPLOYMENT_TARGET=10.6 wheels :(.

Next thing to try is reverting to GDAL 2.1.3.

Note: I tried GDAL 2.1.4 and hit a build bug involving isnan.

@sgillies

This comment has been minimized.

Show comment
Hide comment
@sgillies

sgillies Oct 30, 2017

Owner

There's a chance this has almost nothing to do with GDAL and my build configuration.

After finding that reverting to frs-wheel-builds 1.3.0 didn't help (same error), I noticed my Xcode and command line tools were out of sync. How this happened, I'm not sure, but by reverting my command line tools to match Xcode 8.2.1 I'm able to get pairs of Shapely and Fiona wheels that don't crash each other. More details soon.

Owner

sgillies commented Oct 30, 2017

There's a chance this has almost nothing to do with GDAL and my build configuration.

After finding that reverting to frs-wheel-builds 1.3.0 didn't help (same error), I noticed my Xcode and command line tools were out of sync. How this happened, I'm not sure, but by reverting my command line tools to match Xcode 8.2.1 I'm able to get pairs of Shapely and Fiona wheels that don't crash each other. More details soon.

@sgillies sgillies reopened this Oct 31, 2017

@sgillies

This comment has been minimized.

Show comment
Hide comment
@sgillies

sgillies Oct 31, 2017

Owner

I can't find any evidence that the problem was anything other than Xcode mis-configuration.

Summary here: https://sgillies.net/2017/10/31/trued-macosx-wheels.html.

  • Delete macosx wheels for Shapely 1.6b5 through 1.6.2
  • Delete macosx wheels for Fiona 1.7.7 through 1.7.10
  • Delete macosx wheels for Rasterio 1.0a10
Owner

sgillies commented Oct 31, 2017

I can't find any evidence that the problem was anything other than Xcode mis-configuration.

Summary here: https://sgillies.net/2017/10/31/trued-macosx-wheels.html.

  • Delete macosx wheels for Shapely 1.6b5 through 1.6.2
  • Delete macosx wheels for Fiona 1.7.7 through 1.7.10
  • Delete macosx wheels for Rasterio 1.0a10

@sgillies sgillies closed this Oct 31, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment