custom layers in postgis

[rishabhjain30]

I have custom layers stored in postgres with geometry. How can I use tileserver to serve mvt from these layers?
Also, Is addition of more layers dynamically supported?

[rmarianski]

I have custom layers stored in postgres with geometry. How can I use tileserver to serve mvt from these layers?

We don't have docs for this, but you can get started by looking at queries.yaml. If you pick one of the layers that's already there and look through how it's configured and set up, you should be able to do the same to add in your own layer. You'll need to add a jinja2 template file too that will become the sql that gets used.

Also, Is addition of more layers dynamically supported?

If you mean by being able to add to an already running tileserver, then no. But if you go through the configuration steps and restart then yes.

If you're looking for something more out of the box, have a look at tilestache.

[radumas]

Hi @rishabhjain30, we're doing this as well. Starting with our city's streets centerline layer. We'll update this thread with our discoveries in the process.

[nvkelso]

We also recommend checking out the new PostGIS functionality to export MVT vector tiles directly without (also) making the extra Postgres > Tileserver hop. (Still may make sense for you to use Tilezen stack, but just FYI):

• https://postgis.net/docs/manual-dev/ST_AsMVT.html

[radumas]

Woah. Very cool feature from PostGIS.

So we're trying to modify the configs to work with our streets layer based on the ne-roads layer. I'm a little puzzled about some of the contents of vector-datasource/queries.yaml. Specifically the following section (commented out lines were me):

layers:
  roads:
    template: roads.jinja2
    geometry_types: [LineString, MultiLineString]
    simplify_start: 8
    transform:
      - vectordatasource.transform.tags_create_dict
#      - vectordatasource.transform.tags_name_i18n
#      - vectordatasource.transform.add_id_to_properties
#      - vectordatasource.transform.detect_osm_relation
      - vectordatasource.transform.parse_layer_as_float
#      - vectordatasource.transform.road_classifier
#      - vectordatasource.transform.road_oneway
      - vectordatasource.transform.road_abbreviate_name
      - vectordatasource.transform.route_name
#      - vectordatasource.transform.normalize_aerialways
#      - vectordatasource.transform.normalize_cycleway
#      - vectordatasource.transform.add_is_bicycle_related
#      - vectordatasource.transform.merge_networks_from_tags
#      - vectordatasource.transform.extract_network_information
#      - vectordatasource.transform.choose_most_important_network
      - vectordatasource.transform.road_trim_properties
      - vectordatasource.transform.remove_feature_id
#      - vectordatasource.transform.tags_remove
      - vectordatasource.transform.truncate_min_zoom_to_2dp
    sort: vectordatasource.sort.roads
    area-inclusion-threshold: 1

First of all I couldn't find a roads.jinja2 file in that repository. There's a yaml/roads.yaml file, there's the ne-roads.jinja2 which I modified to fit the our streets layer's schema, but no file I could find with that name.

Second, I found that the lines like:

    transform:
      - vectordatasource.transform.tags_create_dict
      - etc...

Correspond to python functions under vectordatasource/transform.py and a number seem to be osm property specific. Is there an obvious way to know which should be applied to my data?

Thanks!

[nvkelso]

The simple answer is the transforms are optional. Try it without the transforms, and then try adding a few back in. I'll comment about a few that might be interesting to you:

• vectordatasource.transform.tags_name_i18n - allows / normalizes for English and French names on the same road feature.
• vectordatasource.transform.parse_layer_as_float - this is OSM specific to turn a string value into a float value, you probably don't need it.
• vectordatasource.transform.road_abbreviate_name - is more optimized for English name abbreviations (like Street to St.), probably useful for you.
• vectordatasource.transform.route_name - OSM specific, but generally deals with things like the signed maple leaf route versus province highways.
• vectordatasource.transform.remove_feature_id - you probably want to remove. It's helpful to us for simplifying (merging) roads at low- and mid-zooms to reduce file size.
• vectordatasource.transform.truncate_min_zoom_to_2dp - probably don't need this.

@rmarianski has been reorganizing the repo a bit. I suspect the roads.jinja file you want to reference is this now:

• https://github.com/tilezen/vector-datasource/blob/master/queries/ne-roads.jinja2

Did you see it referenced in code or in the documentation?

We generally create tags for things that didn't necessarily start off with tags as they're easier way to pass properties around.

[rmarianski]

The background on the queries is that they are now organized by table rather than by layer, and we're in a moment of transition now where things will likely be moving around a bit.

Depending on what you're doing, you might find it easier to use the latest release as of this time and the corresponding versions, which have the queries organized by layer. It would probably be easier to add a new layer using that as a base, but we'll be moving away from that ourselves so longer term you'd fall out of sync.

In terms of using the latest code on master, if you're adding a new table you'd want to add it the sources configuration in queries.yaml, and then you'd have to update the code in tilequeue.process which is what takes that and converts it back into a per layer structure. The layer list itself isn't dynamically set through configuration, and I expect you would have to edit the code in a couple of places to add in a new layer throughout the system. If you're generating a new layer from the existing tables in use, grep for that table in the queries directory to see where it's used.
Eventually we'd like to get to a position where this is more dynamically configurable, but we'd only get to that once all the movement on the pieces has settled down more. It's also worth mentioning that the configuration and apis aren't stable at this point, and we may be making further changes to them in the near future.

Hope that helps! If you have any questions about where something ends up happening now, I can help point you in that direction.

[radumas]

Thanks @rmarianski, I'll come back to this thread when I start tweaking things.

[radumas]

We've started working on setting up map in this repository (definitely still a work in progress). Our version of the installation docs is in tilezen/README.md

Still working on styling and interactivity.

[nvkelso]

Closing for inactivity.