MacOS ARM SQLite not found

[baileyheading]

Hi,

Has anyone succeeded in building Valhalla for Mac OS ARM?

After many attempts, starting with brew install, and then installing from source with the dynamic extension loading flag on, I couldn't get past this first error "sqlite_enable_load_extension is disabled.", as per the image below

I can see that it is pathing to my intel installs for sqlite, but I also tried manually pathing the cmake to the ARM ones with no success

[nilsnolde]

Did you follow the docs at https://valhalla.github.io/valhalla/building/#building-from-source-macos?

[baileyheading]

My attempt was based off of these docs yes. I think that most of this is about the mac os intel install using Rosetta 2, which I've already done a few times.

[nilsnolde]

No idea about any of that;) But I soon will, it looks like I'll have to spend lots of days & dev time on M1 soon-ish, ain't looking forward to that!

[baileyheading]

I'm very interested in being able to run it on M1 as it is my daily driver, and the performance should in theory be ridiculous. Happy to help out debugging any issues encountered.

[nilsnolde]

First thing I'd look at is https://github.com/valhalla/valhalla/blob/master/cmake/FindSQLite3.cmake. I bet it doesn't look in the right paths. Feel free to PR a fix.

[baileyheading]

I attempted to make modifications to this file already but hit a dead end. I was possibly pathing to the right file as I did it with a manual path, but the enable_load_extension thing might have still been disabled. I even tried installing sqlite3 it from source with dynamic load extension set to true, but my tests suggested it wasn't enabled still (assuming this test was even pathing to the right place)

[baileyheading]

I think the path should be to "/opt/homebrew/opt/sqlite3" (i think" /opt/homebrew/opt/sqlite" goes to the same place though). I had no luck with adding this though. Would path to homebrew because the system version has dynamic extension loading off for security reasons.

[nilsnolde]

OK, that does sound painful.. I had a similar problem once for a MinGW build but memory eludes me..

If you're not strongly set on building data on your M1, you don't need SQLite (or any of the -DDATA_TOOLS dependencies. In theory tiles built on x64 should work on arm64 right @kevinkreiser ?

[nilsnolde]

That's what I'll do if M1 pisses me off for longer than a few mins;)

[baileyheading]

I'm not a tile expert but I don't think they're platform dependent because I built mine on a linux docker, and then I run them on mac os intel (emulated with Rosetta 2 on M1), so building them is definitely a low priority.

[baileyheading]

wait, that would both be simular architectures wouldn't it.. anyway, I don't think it matters?

[nilsnolde]

yeah.. still, I think it works. try it, if it works, then do cmake with -DDATA_TOOLS=OFF and you won't have that pain.

[nilsnolde]

and -DENABLE_TESTS=OFF (might not be necessary though, I think our cmake logic respects DATA_TOOLS as well)

[baileyheading]

I ran "cmake -B build -DCMAKE_BUILD_TYPE=Release -DDATA_TOOLS=OFF -DENABLE_TESTS=OFF" and got the exact same problem

[nilsnolde]

did you nuke your build folder before?

[baileyheading]

yes. and I did a fresh clone.

[baileyheading]

With "-DDATA_TOOLS=OFF", it shouldn't even look for it as far as I can see in the cmake. I'll try commenting that out.

[baileyheading]

as expected, it zoomed straight past where the error occurred, but then it hit the tile building part and got dependency complaints. I don't know why the "-DDATA_TOOLS=OFF" was ignored for me

[baileyheading]

I got to 65% and then errors

[nilsnolde]

oops, it's -DENABLE_DATA_TOOLS 😅 shouldn't go by memory;)

add a -DENABLE_SINGLE_FILES_WERROR=OFF and you'll be good

[baileyheading]

I had to rebuild prime_server for Mac OS ARM because I had only done it for intel x86, but everything has now built. time to test it.

[baileyheading]

libc++abi: terminating due to uncaught exception of type boost::wrapexceptboost::property_tree::ptree_bad_path: No such node (httpd.service.timeout_seconds)
zsh: abort ./valhalla_service ./valhalla.json 2

I don't seem to have boost on homebrew for ARM. maybe I missed that error because of the single files with error flag. Made no difference anyway.

[nilsnolde]

create a new JSON config, you just don't have that key in there, it's been added a few months ago

[baileyheading]

Ok, I'm using the same config from the one I built a year or so ago, so that would be it. thanks

[baileyheading]

How do I create a new one of these (config.json)? Did I get it when I built my tiles, I can't remember anymore

[baileyheading]

Nevermind, I added that parameter manually

[baileyheading]

Routing seems to work fine, but isochrones are mega broken right now

I'm wondering if it's because of parameters in the json.config or if it's something else

[baileyheading]

I ran isochrones for 5,10,15 minutes and it looks quite bizarre. There are only about 6 coordinates each or less.

[kevinkreiser]

yes the tiles are platform independent, they have to be binary compatible on any architecture

yes i think we should get m1 building the whole stack even if it means screwing around with cmake dependency stuff. if i had access to an m1 i could help but i would never willingly give apple my money.

you said you manually added the config parameter, perhaps its best that you use valhalla_build_config to recreate it to make sure all the values in there are properly defaulted.

[baileyheading]

ok, I replaced with a new one built from "valhalla_build_config". Routing seems works perfectly, but the isochrones are still just as broken as before

[baileyheading]

I've got the python bindings working for Mac OS ARM, which is really nice. I just need to work out what is wrong with these isochrones

[kevinkreiser]

well we'll need to debug it a little closer then i suspect. tell us what data you are using, tell us what request you are sending and tell us the reseponse you are getting, that should be the first stage in debugging this

[baileyheading]

1. I'm using routing tiles made from global OSM data
2. input
   {
   "locations": [
   {
   "lat":40.44323081946625, "lon":-79.98759559595668
   },
   ],
   "costing": "auto",
   "contours": [{"time": 10.0}]
   }
   3)response

{'features': [{'properties': {'fill-opacity': 0.33,
'fill': '#bf4040',
'fillColor': '#bf4040',
'color': '#bf4040',
'fillOpacity': 0.33,
'opacity': 0.33,
'contour': 10,
'metric': 'time'},
'geometry': {'coordinates': [[-80.012893, 40.516934],
[-80.012101, 40.514737],
[-80.013596, 40.514472],
[-80.014596, 40.513549],
[-80.016849, 40.513485],
[-80.017019, 40.512231]],
'type': 'LineString'},
'type': 'Feature'}],
'type': 'FeatureCollection'}

not enough coordinates in the geometry. Similar result for all locations I've tried.

[nilsnolde]

feel free to open another issue

[kevinkreiser]

why did we close this if it doesn't build? i get there is some other problem with isochrones but it not finding sqlite stuff should be fixed before closing

[nilsnolde]

I'd guess we can close this finally:)