Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sqlite Spatial: dotnet crashes when loading Spatial Extension - On Mac. #16667

Open
garfbradaz opened this issue Jul 19, 2019 · 23 comments

Comments

@garfbradaz
Copy link

commented Jul 19, 2019

I'm trying to test out the Spatial Types functionality in EF Core for SQLite In Memory provider.

Now when I run an Xunit test the dotnet executable crashes and I will receive a crash dump report (No application exceptions are thrown).

It seems to crash when trying to access the mod_spatialite extension. I followed the following advice from this issue #14402, as my original problem was with receiving a SQLite Exception with a no such function: InitSpatialMetaData error to begin with.

Steps to reproduce for Mac

  1. git clone https://github.com/StratumFive/EFCore.SQLite.SpatialCrash.
  2. brew install libspatialit
  3. Change directory to the root solution.
  4. dotnet test xunit
  5. Receive error message:

The active test run was aborted. Reason: Test host process crashed

There is even a console application there to prove it isn't just a Xunit issue.

Further technical details

EF Core version: 2.2.6
Database Provider: Microsoft.EntityFrameworkCore.Sqlite / Microsoft.EntityFrameworkCore.Sqlite.NetTopologySuite
Operating system: Mac Mojave 10.14.5
IDE: Visual Studio Code
SQLite version: 3.24.0 2018-06-04 14:10:15
dotnet info:

.NET Core SDK (reflecting any global.json):
Version: 2.2.300
Commit: 73efd5bd87

Runtime Environment:
OS Name: Mac OS X
OS Version: 10.14
OS Platform: Darwin
RID: osx.10.14-x64
Base Path: /usr/local/share/dotnet/sdk/2.2.300/

Host (useful for support):
Version: 2.2.5
Commit: 0a3c9209c0

.NET Core SDKs installed:
2.2.105 [/usr/local/share/dotnet/sdk]
2.2.106 [/usr/local/share/dotnet/sdk]
2.2.107 [/usr/local/share/dotnet/sdk]
2.2.300 [/usr/local/share/dotnet/sdk]

.NET Core runtimes installed:
Microsoft.AspNetCore.All 2.2.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.2.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.2.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.2.4 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.2.3 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 2.2.4 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 2.2.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

@ajcvickers

This comment has been minimized.

Copy link
Member

commented Jul 19, 2019

/cc @bricelam

@garfbradaz

This comment has been minimized.

Copy link
Author

commented Jul 22, 2019

Updated above with SQLite version.

@garfbradaz

This comment has been minimized.

Copy link
Author

commented Jul 22, 2019

Update

Added a Dockerfile to the repository here for my repo. It also crashes as well (Debian).

Steps to reproduce for Dockerfile (assumes docker is setup on machine)

  1. git clone https://github.com/StratumFive/EFCore.SQLite.SpatialCrash.
  2. Change directory to the root solution.
  3. Build the image docker build . -t xunit-crash
  4. Run the container and execute into it docker run --rm -it -p 5001:5001/tcp xunit-crash:latest
  5. Run tests dotnet test xunit/EFCore.Spatial.Crash.csproj -c Release -r /TestResults --logger "trx;LogFileName=TestResults.trx"
  6. Receive error:

The active test run was aborted. Reason:

@garfbradaz

This comment has been minimized.

Copy link
Author

commented Jul 22, 2019

Also, here is my Mac spatialite Mac info:

spatialite
SpatiaLite version ..: 4.3.0a Supported Extensions:
- 'VirtualShape' [direct Shapefile access]
- 'VirtualDbf' [direct DBF access]
- 'VirtualXL' [direct XLS access]
- 'VirtualText' [direct CSV/TXT access]
- 'VirtualNetwork' [Dijkstra shortest path]
- 'RTree' [Spatial Index - RTree]
- 'MbrCache' [Spatial Index - MBR cache]
- 'VirtualSpatialIndex' [R
Tree metahandler]
- 'VirtualElementary' [ElemGeoms metahandler]
- 'VirtualXPath' [XML Path Language - XPath]
- 'VirtualFDO' [FDO-OGR interoperability]
- 'VirtualGPKG' [OGC GeoPackage interoperability]
- 'VirtualBBox' [BoundingBox tables]
- 'SpatiaLite' [Spatial SQL - OGC]
PROJ.4 version ......: Rel. 6.1.1, July 1st, 2019
GEOS version ........: 3.7.2-CAPI-1.11.2 b55d2125
TARGET CPU ..........: x86_64-apple-darwin18.2.0
the SPATIAL_REF_SYS table already contains some row(s)
SQLite version ......: 3.28.0
Enter ".help" for instructions
SQLite version 3.28.0 2019-04-16 19:49:53
Enter ".help" for instructions
Enter SQL statements terminated with a ";"

@garfbradaz

This comment has been minimized.

Copy link
Author

commented Jul 23, 2019

@bricelam do you need anything else from me?

@bricelam

This comment has been minimized.

Copy link
Member

commented Jul 23, 2019

I don't think so. I just need to repro it locally and dig into the failure.

@ajcvickers ajcvickers added this to the Backlog milestone Jul 29, 2019

@garfbradaz

This comment has been minimized.

Copy link
Author

commented Aug 1, 2019

Can I help in anyway to dig into the failure @bricelam ? I know you guys are busy and this issue is blocking me.

@bricelam

This comment has been minimized.

Copy link
Member

commented Aug 1, 2019

Does it repro with something as trivial as this:

using static SQLitePCL.raw;

class Program
{
    static void Main()
    {
        SQLitePCL.Batteries_V2.Init();

        sqlite3_open(":memory:", out var db);
        sqlite3_enable_load_extension(db, 1);
        sqlite3_exec(db, "SELECT load_extension('mod_spatialite')");
    }
}
@bricelam

This comment has been minimized.

Copy link
Member

commented Aug 1, 2019

Looks like our CI has been hitting this too. 😁

@bricelam

This comment has been minimized.

Copy link
Member

commented Aug 1, 2019

It appears to be in issue with the latest revision of the Homebrew package.

Workaround

Use the previous version:

brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/835ae521c065524abdf66578e68032fa24bce514/Formula/libspatialite.rb

bricelam added a commit to bricelam/EFCore that referenced this issue Aug 1, 2019

@bricelam bricelam removed this from the Backlog milestone Aug 1, 2019

@bricelam

This comment has been minimized.

Copy link
Member

commented Aug 1, 2019

@fxcoudert What's the proper channel for reporting this?

@bricelam

This comment has been minimized.

Copy link
Member

commented Aug 2, 2019

It may (or may not) be related to the fact that our native SQLite library is named libe_sqlite3.dylib not libsqlite3.dylib

@garfbradaz

This comment has been minimized.

Copy link
Author

commented Aug 2, 2019

It may (or may not) be related to the fact that our native SQLite library is named libe_sqlite3.dylib not libsqlite3.dylib

Amazing @bricelam - looking now :)

@garfbradaz

This comment has been minimized.

Copy link
Author

commented Aug 2, 2019

Feedback

@bricelam - sorry for the delay.

I reverted the version back using the workaround, but this has caused a new set of errors explained now.

New Errors

  • This runs fine without error.
using static SQLitePCL.raw;

class Program
{
    static void Main()
    {
        SQLitePCL.Batteries_V2.Init();

        sqlite3_open(":memory:", out var db);
        sqlite3_enable_load_extension(db, 1);
        sqlite3_exec(db, "SELECT load_extension('mod_spatialite')");
    }
}
  • The following Xunit code found here now throws the following exception:

EFCore.Spatial.Crash.TestCrash.Test_Crash_for_EF_Team:
Outcome: Failed
Error Message:
Microsoft.Data.Sqlite.SqliteException : SQLite Error 1: 'dlopen(mod_spatialite.dylib.dylib, 10): image not found'.
Stack Trace:
at Microsoft.Data.Sqlite.SqliteException.ThrowExceptionForRC(Int32 rc, sqlite3 db)
at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader(CommandBehavior behavior)
at Microsoft.Data.Sqlite.SqliteCommand.ExecuteNonQuery()
at Microsoft.EntityFrameworkCore.Infrastructure.SpatialiteLoader.Load(DbConnection connection)
at EFCore.Spatial.Crash.TestCrash.Test_Crash_for_EF_Team() in /Users/garethb/code/EFCore.Spatial.Crash/xunit/TestCrash.cs:line 18
--- End of stack trace from previous location where exception was thrown ---

  • The following console code found here now produces the following stack dump here.

Interestingly now when I run spatialite on the commandline I get:

dyld: Library not loaded: /usr/local/opt/proj/lib/libproj.13.dylib
Referenced from: /usr/local/opt/libspatialite/lib/libspatialite.7.dylib
Reason: image not found
[1] 21039 abort spatialite

Could it be some $PATH is missing or is expected? Here is my echo'd $PATH:

/usr/local/opt/sqlite/bin
/Users/garethb/.dotnet/tools
/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin
/usr/local/share/dotnet
~/.dotnet/tools
/Library/Frameworks/Mono.framework/Versions/Current/Commands
/Applications/Xamarin Workbooks.app/Contents/SharedSupport/path-bin

I even updated my .csproj to reflect your test suite as your pipeline was successful, but the same error:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netcoreapp3.0</TargetFramework>

    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.Data.Sqlite" Version="3.0.0-preview7.19362.6" />
    <PackageReference Include="Microsoft.EntityFrameworkCore" Version="3.0.0-preview7.19362.6" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite" Version="3.0.0-preview7.19362.6" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite.NetTopologySuite" Version="3.0.0-preview7.19362.6" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.3.0-preview-20190715-02" />
    <PackageReference Include="xunit" Version="2.4.0" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.0" />
    <PackageReference Include="SQLitePCLRaw.bundle_e_sqlite3" Version="2.0.0-pre20190628101813" />
    <PackageReference Include="NETStandard.Library" Version="1.6.1" />
  </ItemGroup>

</Project>

bricelam added a commit to bricelam/EFCore that referenced this issue Aug 2, 2019

@ajcvickers

This comment has been minimized.

Copy link
Member

commented Aug 5, 2019

Unfortunately, we are out of ideas here. We are going to put this on the backlog, but we are likely to need expert help to make real progress here.

@ajcvickers ajcvickers added this to the Backlog milestone Aug 5, 2019

@pond

This comment has been minimized.

Copy link

commented Aug 16, 2019

This is a critical blocker for us.

As a community effort, even with everyone trying their best, crash bugs thrown into the "too hard" bucket mean it can't be taken seriously as something a company's income depends upon. Hard-crash issues need proper engineering resources from Microsoft dedicating to them so that people who are actually able to diagnose and fix the issue are promptly assigned. Perhaps the bottom line is that although EntityFrameworkCore is cross-platform, a crash on a non-Windows platform isn't where the money is and isn't going to get much traction.

If there's anything that can be done to escalate this I'm sure others would appreciate it.

@ajcvickers

This comment has been minimized.

Copy link
Member

commented Aug 16, 2019

@pond What version of EF Core are you using?

@bricelam

This comment has been minimized.

Copy link
Member

commented Aug 16, 2019

@pond the crash is caused by changes made to the libspatialite Homebrew package. I've filed Homebrew/homebrew-core#43203. See my comment above for a workaround.

@pond

This comment has been minimized.

Copy link

commented Aug 19, 2019

@bricelam The work-around doesn't help I'm afraid - it was the first thing I tried. It doesn't matter what version of that package I install; the crash happens anyway. It looks likely to be a combination of specific versions of several packages. I have isolated a single machine which works, and will try to replicate that particular configuration on another machine as soon as I have time.

@ajcvickers Unsure how to get a version for that particular component; .Net SDK 2.2.4 ("latest download" at the time of writing) and again "most recent" Nuget packages related to EFC & SQLite... Looks like there are references to Microsoft.EntityFrameworkCore.Sqlite 2.2.6 and Microsoft.EntityFrameworkCore.Sqlite.NetTopologySuite 2.2.6 in the .csproj file.

@pond

This comment has been minimized.

Copy link

commented Aug 20, 2019

By carefully unpicking all newer versions and installing specific versions of the runtime and related libraries, I have my own test suite running on SQLite that used to crash. Notably, the version of libspatialite is not in my case responsible. Using the most recent Homebrew build works fine. It's other dependencies that seem to be the issue - at least in my case.

  • However - even with rolling libspatialite back to 4.3.0a_6 as per the above workaround, and therefore having to roll proj back to 5.2.0 to avoid the related dylib load error, via brew install --ignore-dependencies https://raw.githubusercontent.com/Homebrew/homebrew-core/cc8b37d018e1dd29b9051fff899fa397239c59c4/Formula/proj.rb - I still can't get that tiny test case to pass. Our large company test suite runs now, strangely, but the small test case still crashes.

It's quite an ugly process. I imagine that most of these carefully chosen versions are not necessary, so I'll edit and update this answer as I narrow things down, until I run out of time for it anyway.

First:

Next, remove a few Homebrew packages:


brew uninstall postgis
brew uninstall gdal
brew uninstall libspatialite 
brew uninstall libgeotiff
brew uninstall proj
brew uninstall sqlite

...resolving dependencies (or ignoring them via --ignore-dependencies) with further uninstall commands if need be, taking note of them all for reinstallation later. Then, we cherry-pick versions that happen to work on my machine.

SQLite 3.28.0

brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/4fa421cfc34548eb48bc236460584c9f4fa16b22/Formula/sqlite.rb


Proj 6.1.0

brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/ca19c245aae7d6fa8c0d9d7314a4964a3a3b18c8/Formula/proj.rb

Libgeotiff seems unrelated, so we just install the most recent version. For this and further calls, Homebrew doesn't seem to recognise the from-SHA installations of dependencies so halts; you have to install with --ignore-dependencies. Rather unhelpfully, Homebrew will then warn you that this is an unsupported flag you're not supposed to use, but it's not as if there's any choice.

brew install --ignore-dependencies libgeotiff

Gdal 2.4.1_1

brew install --ignore-dependencies https://raw.githubusercontent.com/Homebrew/homebrew-core/37ce68573be9fe921ef65d421da4657e2f6d0270/Formula/gdal.rb

Libspatialite 4.3.0a_7 (the most recent version at the time of writing, but locking to SHA for safety here)

brew install --ignore-dependencies https://raw.githubusercontent.com/Homebrew/homebrew-core/1bbba12ca10e05e7194a6233d4bf5bf56677939f/Formula/libspatialite.rb


Postgis 2.5.2_2 (the most recent version at the time of writing, but locking to SHA for safety here)

brew install --ignore-dependencies https://raw.githubusercontent.com/Homebrew/homebrew-core/0a2e72798e72e3accead7e793d6cccf4ff89dd70/Formula/postgis.rb
@bricelam

This comment has been minimized.

Copy link
Member

commented Aug 20, 2019

I too am starting to think this is a problem with PROJ. It looks like it links directly to libspatialite3-0 which breaks SQLite extensions.

@pond

This comment has been minimized.

Copy link

commented Aug 20, 2019

@bricelam Should be able to go to 5.2.0 via

brew uninstall --ignore-dependencies proj
brew uninstall --ignore-dependencies libspatialite
brew install --ignore-dependencies  https://raw.githubusercontent.com/Homebrew/homebrew-core/cc8b37d018e1dd29b9051fff899fa397239c59c4/Formula/proj.rb
brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/835ae521c065524abdf66578e68032fa24bce514/Formula/libspatialite.rb

I tried this and while my own test suite continued to build, I didn't have much luck with the one in the Homebrew/homebrew-core#43203 example repository. Worth a try on your local machine though as there are many other things that might have caused problems on my old laptop.

@SMillerDev

This comment has been minimized.

Copy link

commented Aug 21, 2019

Just make sure not to file issues with homebrew if you install old versions this way since it isn't supported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.