Generate planet country by country causing duplicities

[marfig2005]

Hello,
as I don't have a machine with enough RAM (just 16GB) to generate the whole planet I had an idea, to generate it country by country and then use tile-join to merge it into one final planet.mbtiles, finally servered to browser by docker maptiler/tileserver-gl
I did an exercise for Europe. And it's almost perfectly working.

Just ocean areas are corrupted, my guess is, that it's included in multiple countries and as such duplicated, see:

Am I completely wrong? And if not, is there any way how to improve it? I've tried removing duplicated tiles via SQL, but it was without any difference.

Thanks
Regards
Martin

[msbarry]

Hello! This shouldn't be necessary, I've generated planet tiles on a machine as small as 2cpu/8gb ram in about 18 hours so you should be fine with 16gb. Just use storage=mmap and -Xmx8g.

[marfig2005]

OK I tried to reproduce your exact setup with latest release on a 32-core machine a few times and finished in a little over an hour without getting stuck at the archive step so I'm not quite sure what's going on. It might be worth trying again, possibly with version 0.10.1 ? If it gets to a point where it's not making forward progress during the archive step then just kill it.

thank you! running 0.10.1 now

[marfig2005]

OK I tried to reproduce your exact setup with latest release on a 32-core machine a few times and finished in a little over an hour without getting stuck at the archive step so I'm not quite sure what's going on. It might be worth trying again, possibly with version 0.10.1 ? If it gets to a point where it's not making forward progress during the archive step then just kill it.

thank you! running 0.10.1 now

:( something must be wrong somewhere, even the previous version got stuck in a same way

[msbarry]

Got it. I think to debug any further I'd need the full output logs from startup to where it starts to hang.

[marfig2005]

Got it. I think to debug any further I'd need the full output logs from startup to where it starts to

OK, understand, let's give it one more try, will start the latest release from a scratch and dump the complete console output to file.

[marfig2005]

Got it. I think to debug any further I'd need the full output logs from startup to where it starts to

OK, understand, let's give it one more try, will start the latest release from a scratch and dump the complete console output to file.

log.txt
here is the complete log

Thanks
Martin

[ImreSamu]

I think it’s very difficult (or rather impossible) to generate a good planet from geofabrik country extracts,
and it’s highly likely that the vectortiles won’t look good along the country borders due to overlaps.

A while back, I generated a planet based on 8x8 Quadtiles and just had to merge the final result. It’s not that simple, but I didn’t have any problems with it.
( of course, you have to cut out the 8x8 quadtiles first, which results in 64 smaller runs. )

<< Alternative solution : just download a full planet mbtiles >>

OpenFreeMap.org has a "weekly full planet downloads both in Btrfs and MBTiles formats"

• created with Planetiler tool
• "map schema is unmodified OpenMapTiles."

see: https://github.com/hyperknot/openfreemap#full-planet-downloads

• check https://btrfs.openfreemap.com/files.txt
• and find the latest weekly full planet ( tiles.mbtiles ) downloads

example:

....
areas/planet/20260429_001001_pt/SHA256SUMS  <--- checksum 
areas/planet/20260429_001001_pt/done
areas/planet/20260429_001001_pt/osm_date
areas/planet/20260429_001001_pt/planetiler.err
areas/planet/20260429_001001_pt/planetiler.out
areas/planet/20260429_001001_pt/tiles.btrfs.gz
areas/planet/20260429_001001_pt/tiles.mbtiles  <-----  you have to download this file , it is a big file !

download example:

wget https://btrfs.openfreemap.com/areas/planet/20260429_001001_pt/SHA256SUMS
wget https://btrfs.openfreemap.com/areas/planet/20260429_001001_pt/tiles.mbtiles

[marfig2005]

native solution : just download a full planet mbtiles >

wow, wow, I didn't know about this, I was using many years old mbtiles file and the commercial company, where I downloaded it, asked too much for an updated one, considering my purpose

Thank you!!