Custom tileserver

[otsaloma]

It seems that there are not many providers of good-quality retina map tiles and the only good option, Mapbox, seems to have dropped midlevels from their pricing plans, meaning that once use exceeds the free plan it would be a whopping $500/month. Also, the retina tiles that providers have are only @2x, while some devices, such as Fairphone 2, would really need @3x or @4x.

For future reference, once the user count gets past free plans, based on a quick review, the most promising option for a tileserver seems to be TileServer GL.

• http://blog.klokantech.com/2016/08/tileserver-gl-maps-with-gl-json-styles.html
• https://github.com/klokantech/tileserver-gl
• http://demo.tileserver.org/
• https://maps.klokantech.com/
• https://www.youtube.com/watch?v=rOg4VnSAnI4
• https://github.com/tobinbradley/mapbox-gl-style-editor

[otsaloma]

http://blog.klokantech.com/2017/01/openmaptiles-vector-tiles-from.html

[otsaloma]

A custom tileserver could also allow bulk downloading tiles for offline use, without having to worry about breaking terms or exceeding limits of commercial service providers.

[otsaloma]

Our CEO Sami Pienimäki commented that we’re aiming to make the Sony Xpreria X a follower community device for Jolla C.

https://blog.jolla.com/wrapping-up-mwc-2017-jolla/

Resolution: 1080 x 1920 pixels (~441 ppi pixel density)

http://www.gsmarena.com/sony_xperia_x-7948.php

[rinigus]

I would be happy to contribute to resolving this issue by extending OSM Scout Server to serve data in some other way. We have just setup maps distribution service that allows you to get the maps into mobile. I can imagine that we could use this to deliver some additional formats, if needed, and then feed the data to the map applications (Poor Maps and modRana, for example).

If we offload the rendering into some widget on the client (OpenGL?), we would immediately get labels aligned according to the axis. Alternatively, we could also specify the angle showing direction of the top, so we could render labels accordingly.

In addition to MapBox GL there is https://github.com/tangrams/tangram-es that could serve for similar purpose.

[otsaloma]

I don't quite follow. Are you suggesting OSM Scout Server to run on a real server and return vector map tiles as requested? Or just to generate vector tiles and put them as files on a server?

And where do you see the rendering widget? A QtLocation plugin? A QML component different from QtLocation Map? A program/library separate from Qt/QML and called from Python? Currently Poor Maps uses raster tiles which are downloaded in Python and in QML overlaid on a QtLocation Map.

[rinigus]

I must say that I still have to educate myself on vector maps. Information is rather sparse or I look in the wrong places.

What I was suggesting was that we can probably get MBTiles (or similar data) to mobile and serve that to Poor Maps. Now that immediately bumps into your other questions. So, let me try to address them.

Rendering widget is a major question, I presume. How do you envision rendering vector tiles? As far as I can see, rendering can be done on the server side, as it is now, and served as images to feed the map viewers. Alternative, and I presume that's a real vector tile rendering, is that you are served vector tile (aka data) and its rendered in GUI. That's what I think MapBox GL / Tangram ES are doing and that would allow you to rotate the screen with all the labels rotating smoothly. How to integrate that into python application, I don't know.

If rendering has to be done on the server side, either for now and allowing us to develop other components first, then map viewing can be kept as long as we are able to generate the tiles. Either your proposed solution via separate server or OSM Scout Server on mobile could probably fit the bill. I am looking at present on how to improve rendering quality of the tiles in OSM Scout Server, hence the suggestions regarding this issue.

As a side note, you can use OSM Scout Server on real server as well. But rendering of the maps is done better at this stage by MapBox-based solution.

[otsaloma]

I haven't looked into vector tiles myself much yet either, just saved some links for later use. My general impression is that is that it's rather fragmented -- from an application author's perspective there's a danger of vendor lock-in -- you can't know if files of a particular vector tile format will be available in the future or if some particular renderer will run on future versions of an obscure OS.

If you basically replace the rendering engine of OSM Scout Server, then from my perspective it's business as usual. Live rendering in a QML component would of course be better. A QtLocation plugin would be an obvious choice (and there seems to already be a Mapbox GL plugin). There's a small problem of the Map's plugin property being write-once, i.e. one can't switch between plugins, but that's something we could adapt to, one way or another. A different component than QtLocation's Map would be difficult for Poor Maps to use as all current code of interaction and overlaid objects are written for the QtLocation Map API. And, of course, vice versa, for the same reasons, apps such as Modrana would have trouble using a QtLocation plugin.

I do expect to test Tileserver GL at some point, but I figure it might be a lot of work, so I haven't started yet. What I like about it is that it serves both raster and vector tiles, which would allow fixing the ppi problem with higher ppi tiles and at the same time getting vector tiles to experiment with, leaving the solution of how to use them for later.

[rinigus]

I have the same impression of provider lock-in, as you do. It also seem to be in a great flux right now with no standard really dominating.

I am focusing now on improving rendering of the maps provided by OSM Scout Server. At present, I am looking into different options: improving libosmscout or add an additional backend for it. Improving libosmscout would probably be rather long process, taking into account a big science around labeling placing, for example. Since adding backends is relatively simple with the latest server version, I am looking deeper into adding new backend for now, but its early days.

Let's see if I can pull it off and we can get high quality maps served via OSM Scout Server. If it would work, users would be able to adjust font and some other parameters on their devices and find the balance between PPI and their eyes. And I hope that would allow us to use the same approach on the client side for a time being.

[rinigus]

Hi!

I managed to compile Mapbox GL Native library for SFOS. For that, I used gcc 6.4.0 that I compiled earlier and which is compatible with the current SFOS compiler version (I used the same ABI switches). As a result, getting Mapbox vector tiles rendered using GL is a possibility that could come to us.

I will work on integration of Mapbox GL to SFOS QML, there are several bugs right now, and then release a demo RPM for you to test. In general, if it all works, we could make a separate vector-tile version of Poor Maps, I think.

Such development with Mapbox GL is related with the server that you are setting up. Namely, its a new format and, when you configure your server, maybe you could find a simple way to import the planet PBFs into the vector format as well. As far as I understood, its a part of your server setup...

Best wishes,

Rinigus

[otsaloma]

The thought of running Tileserver GL, or whatever underlying Mapbox technologies on mobile did occur to me as well. Glad to head of your progress.

I'm currently in progress of setting up hosting for the server, just today I got it running, but still need to work on running tileserver-gl as a service (pm2 or something) as well upper layers and caching, likely varnish and maybe nginx. It's all looking good and feasible so far. I'm using the planet as vector tiles (55 GB mbtiles file) from openmaptiles.com. While I'm now focusing on raster tiles, the server will output lot of different formats: http://tileserver.readthedocs.io/en/latest/endpoints.html

[rinigus]

Looks good, I can probably use this server as a reference in future if we manage to get vector tiles rendered in SFOS.

I presume that we'll need later to get planet import handled as well. As far as I remember, openmaptiles were not updating their dataset too frequently. But its an issue for future...

[rinigus]

I managed to get the first demo build ready for mapbox gl native (there were few technical issues). The projects are under my repos (one is qml bindings and another is sfos demo app). I compiled demo app at OBS at https://build.merproject.org/package/show/home:rinigus:maps/mapbox-demo-sfos . Pan and pinch-to-zoom works, screen rotation works, but that's it at present. I plan to build up the QML API to make it easy to use.

[otsaloma]

Done. Leaflet and Mapbox GL JS demo pages below.

• http://cartago.io/maps/raster/
• http://cartago.io/maps/vector/

Notes:

• The server is in Frankfurt and should be fast in Europe, but slow in Asia and Australia. I think that's fine for the current user base.

• There's a slight "warm-up" delay. When you first open a map, it takes a while to load, but as you pan further it get fast. I'm not sure about the reason for that, but I think it's in Tileserver GL itself.

• The map style is optimized for raster use. There's a zoom level offset difference of one between raster and vector (see tileserver-gl #167, #144). I might do something about that later if those vector tiles are used somewhere.

• @rinigus If you want to access the vector tiles, you need an API key. I'll send you one by email. Note that you need it for all requests, also any implicit requests to font glyphs etc. e.g. with Mapbox GL JS, I did that with transformRequest, see the vector demo source.

• Server endpoints are documented in tileserver-gl docs.

• There's two map styles: "cartago-streets" and "cartago-streets-english" with the former having labels in native language and the latter in English where available.

I'll make a new release after I check if I can (easily) get @3x tiles to work.

And, rinigus, I tested your mapbox-demo-sfos, but it crashed as soon as I interacted with the map. I assume it's still very much a work in progress. When I have more time, I'll test again and we can continue on that at the TMO thread.

[rinigus]

@otsaloma: first of all - congratulations! Its a great news!

Yes, I am working on mapbox-demo-sfos now, but it shouldn't crash, as far as I know. There were some crashes, but it seemed that I managed to resolve them by moving to another OpenGL encapsulation. But we can discuss it a bit later in the TMO thread. Would be great to get details :)

I will use for now Mapbox API, but surely we will get cartago tested on vector tiles in no time.