-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can tilequeue regenerate tiles based on osmosis diffs? #186
Comments
Rob or Matt can provide more details, but we use a different process
described here:
- https://github.com/tilezen/chef-minutely_mapnik
…On Wed, Apr 12, 2017 at 9:38 PM, Taras Vozniuk ***@***.***> wrote:
How do you guys regenerate the tiles based on OSM updates?
I have found that I can pre-generate tiles with tilequeue, however I
haven't found any info on how do you guys refresh them. I only found that
config specifies expired-location, but it didn't give me any further
clues.
1.
What do you use to invalidate/delete outdated tiles? I doesn't seem to
be a tilequeue's job since I haven't found any .yamls that specify
when certain zoom-level tiles become outdated?
2.
Can I feed osmosis generated .osm.gz to tilequeue so it will look on
query config, do a bunch of postgres queries and figure out what tilesets
need to be recursively updated?
If there are no such mechanisms, writing tile refresh scripts is not a big
deal, but doing tile updates based on osmosis diffs is bit more complex. I
technically can wait until osmosis applies .osm.gz, parse the .osm.gz
xmls, query-config .yamls and osmosis styles and then it is technically
possible to understand what features corresponsd to this osm object, so I
can then recursively regenerate the tileset within this quad subtree. What
do you recommend here?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#186>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/AA0EO1f3MKZi6G5X8r9sFXrxgdINWt5kks5rvaaqgaJpZM4M8QA3>
.
|
awesome, thanks! |
I'm going to close this issue for now, but if you have followup questions please reopen it here. |
chef-Minutely_mapnik seems like a helper to run the osmosis and consume it into postgres. It doesn't look like it calls any other tool to update the tiles based on the |
Yes, we run The results of the TOI intersection are put into an SQS queue and get re-rendered eventually on our If you wanted to run this without our TOI steps, then I think it's possible to feed the output of |
@zerebubuth: Thanks a lot for a flow explanation. It's great to know it is available out-of-the box. One questions regarding TOI. So TOI are these tiles which are often requested + extra tilesets manually configured in |
Yes, that's almost it. The current behaviour is that we take tiles which are often requested and extra tiles that we think are important to make the initial TOI. Then, as the system is running, the However, this leads to a TOI which always grows bigger and bigger, and effort might be spent rendering updates for tiles which were only viewed once and have not been viewed since. Controlling that is the idea behind #176, so that tiles which are not viewed often will be removed from the TOI (and deleted from S3), and we'll save any further effort needed to keep them up-to-date. Depending on how you want to use the tiles, using a TOI might or might not be the right workflow. When the size of the TOI set is small this can be a very effective technique to keep the tiles up-to-date. Unfortunately, when the TOI gets into the millions or tens of millions, re-rendering updates in the TOI takes up quite a lot of computer time! |
@zerebubuth I got it, thanks a lot. Great, #176 has been merged, we can play around with it! Thanks! |
How do you guys regenerate the tiles based on OSM updates?
I have found that I can pre-generate tiles with tilequeue, however I haven't found any info on how do you guys refresh them. I only found that config specifies
expired-location
, but it didn't give me any further clues.What do you use to invalidate/delete outdated tiles? It doesn't seem to be a
tilequeue
's job since I haven't found any.yaml
s that specify when certain zoom-level tiles become outdated?Can I feed osmosis generated
.osm.gz
to tilequeue so it will look on query config, do a bunch of postgres queries and figure out what tilesets need to be recursively updated?If there are no such mechanisms, writing tile refresh scripts is not a big deal, but doing tile updates based on osmosis diffs is bit more complex. I technically can wait until osmosis applies
.osm.gz
, parse the.osm.gz
xmls, query-config.yaml
s and osmosis styles and then it is technically possible to understand what features corresponsd to this osm object, so I can then recursively regenerate the tileset within this quad subtree. What do you recommend here?The text was updated successfully, but these errors were encountered: