New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New openstreetmap-carto release, v4.5.0 #138

Closed
kocio-pl opened this Issue Nov 17, 2017 · 19 comments

Comments

Projects
None yet
5 participants
@kocio-pl

kocio-pl commented Nov 17, 2017

A new version of openstreetmap-carto, v4.5.0, has been released.

@matthijsmelissen

This comment has been minimized.

Show comment
Hide comment
@matthijsmelissen

matthijsmelissen Nov 23, 2017

Thanks for updating!

Could it be there is something wrong with the London tile server? See gravitystorm/openstreetmap-carto#2955 .

matthijsmelissen commented Nov 23, 2017

Thanks for updating!

Could it be there is something wrong with the London tile server? See gravitystorm/openstreetmap-carto#2955 .

@kocio-pl

This comment has been minimized.

Show comment
Hide comment
@kocio-pl

kocio-pl Nov 23, 2017

I have the same issue with z5 in Western Europe as reported, I also have similar problem on z7 (for example eastern part of Poland and other eastern countries still show natural reserves and railways).

$ host a.tile.openstreetmap.org
a.tile.openstreetmap.org is an alias for tile.geo.openstreetmap.org.
tile.geo.openstreetmap.org is an alias for pl.tile.openstreetmap.org.
pl.tile.openstreetmap.org is an alias for germany.tile.openstreetmap.org.
germany.tile.openstreetmap.org has address 144.76.70.77
germany.tile.openstreetmap.org has address 81.7.11.83
germany.tile.openstreetmap.org has address 185.66.195.245

kocio-pl commented Nov 23, 2017

I have the same issue with z5 in Western Europe as reported, I also have similar problem on z7 (for example eastern part of Poland and other eastern countries still show natural reserves and railways).

$ host a.tile.openstreetmap.org
a.tile.openstreetmap.org is an alias for tile.geo.openstreetmap.org.
tile.geo.openstreetmap.org is an alias for pl.tile.openstreetmap.org.
pl.tile.openstreetmap.org is an alias for germany.tile.openstreetmap.org.
germany.tile.openstreetmap.org has address 144.76.70.77
germany.tile.openstreetmap.org has address 81.7.11.83
germany.tile.openstreetmap.org has address 185.66.195.245
@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes Nov 23, 2017

Member

Telling me what cache you're using is next to useless - if you want to help go to http://tile.openstreetmap.org/cgi-bin/debug and tell me which render server you're using.

Member

tomhughes commented Nov 23, 2017

Telling me what cache you're using is next to useless - if you want to help go to http://tile.openstreetmap.org/cgi-bin/debug and tell me which render server you're using.

@kocio-pl

This comment has been minimized.

Show comment
Hide comment
@kocio-pl

kocio-pl Nov 23, 2017

Render Server: orm.openstreetmap.org

kocio-pl commented Nov 23, 2017

Render Server: orm.openstreetmap.org

@matthijsmelissen

This comment has been minimized.

Show comment
Hide comment
@matthijsmelissen

matthijsmelissen Nov 23, 2017

I'm using scorch, and this server renders fine.

matthijsmelissen commented Nov 23, 2017

I'm using scorch, and this server renders fine.

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes Nov 23, 2017

Member

Right, and that server (orm) hasn't finished the low zoom re-render yet so this is expected.

Member

tomhughes commented Nov 23, 2017

Right, and that server (orm) hasn't finished the low zoom re-render yet so this is expected.

@matthijsmelissen

This comment has been minimized.

Show comment
Hide comment
@matthijsmelissen

matthijsmelissen Nov 23, 2017

Good to hear! Is there anywhere where we can see if the re-render process is finished?

matthijsmelissen commented Nov 23, 2017

Good to hear! Is there anywhere where we can see if the re-render process is finished?

@kocio-pl

This comment has been minimized.

Show comment
Hide comment
@kocio-pl

kocio-pl Nov 23, 2017

I thought that z0-z12 was rendered in one go, probably at the start of deploying new version of osm-carto - is it different?

kocio-pl commented Nov 23, 2017

I thought that z0-z12 was rendered in one go, probably at the start of deploying new version of osm-carto - is it different?

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes Nov 23, 2017

Member

It is rendered in one go yes. That "go" has not yet finished.

Member

tomhughes commented Nov 23, 2017

It is rendered in one go yes. That "go" has not yet finished.

@kocio-pl

This comment has been minimized.

Show comment
Hide comment
@kocio-pl

kocio-pl Nov 23, 2017

But it lasts for a few days already with no change, like 5 probably (both on z5 and z7). Once I remember it was like about few hours peak and that was all. That's why I'm curious what's going on.

kocio-pl commented Nov 23, 2017

But it lasts for a few days already with no change, like 5 probably (both on z5 and z7). Once I remember it was like about few hours peak and that was all. That's why I'm curious what's going on.

@DaveF63

This comment has been minimized.

Show comment
Hide comment
@DaveF63

DaveF63 Nov 23, 2017

For me, the problem is older than 6 days. Weeks probably, As it didn't resolve itself I only posted about it now.
Do low levels get re-rendered every update, even if the appearance of entities don't change?

DaveF63 commented Nov 23, 2017

For me, the problem is older than 6 days. Weeks probably, As it didn't resolve itself I only posted about it now.
Do low levels get re-rendered every update, even if the appearance of entities don't change?

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes Nov 23, 2017

Member

Eh? I assumed the point was that this stylesheet change had added or removed a lot of detail at z5? So I don't see how this can predate this change?

Am I missing something? Are the two versions of z5 we are seeing not the before and after versions for this stylesheet change?

Member

tomhughes commented Nov 23, 2017

Eh? I assumed the point was that this stylesheet change had added or removed a lot of detail at z5? So I don't see how this can predate this change?

Am I missing something? Are the two versions of z5 we are seeing not the before and after versions for this stylesheet change?

@kocio-pl

This comment has been minimized.

Show comment
Hide comment
@kocio-pl

kocio-pl Nov 23, 2017

This particular rendering artifact is new, of course. But I also remember that with last release there were some quirks with rendering of z0-z12, which was like few days later on some servers than some others: #131 (comment).

kocio-pl commented Nov 23, 2017

This particular rendering artifact is new, of course. But I also remember that with last release there were some quirks with rendering of z0-z12, which was like few days later on some servers than some others: #131 (comment).

@kocio-pl

This comment has been minimized.

Show comment
Hide comment
@kocio-pl

kocio-pl Nov 24, 2017

Looks like orm is rendering low/medium zoom at last (it's not clear what happened earlier as we have missing data for v4.5.0 deployment on 17-19.11):
https://munin.openstreetmap.org/openstreetmap/orm.openstreetmap/index.html#renderd

I also see my problems described here are gone and USA or China are also clean now on z5.

Te question is however what just happened and if that could be avoided in the future. Tom, what is the algorithm/rule for rerendering low and medium zoom levels on the OSMF servers?

kocio-pl commented Nov 24, 2017

Looks like orm is rendering low/medium zoom at last (it's not clear what happened earlier as we have missing data for v4.5.0 deployment on 17-19.11):
https://munin.openstreetmap.org/openstreetmap/orm.openstreetmap/index.html#renderd

I also see my problems described here are gone and USA or China are also clean now on z5.

Te question is however what just happened and if that could be avoided in the future. Tom, what is the algorithm/rule for rerendering low and medium zoom levels on the OSMF servers?

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes Nov 24, 2017

Member

The short version is that render_old had hung and I gave it a kick.

That's not unusual - it doesn't seem to be the most stable of things. I don't have any good ideas how to work around it though - we already restart it if it exits with an error which copes with the case where renderd is restarted and it bombs out but in this case it had seemingly just hung.

Member

tomhughes commented Nov 24, 2017

The short version is that render_old had hung and I gave it a kick.

That's not unusual - it doesn't seem to be the most stable of things. I don't have any good ideas how to work around it though - we already restart it if it exits with an error which copes with the case where renderd is restarted and it bombs out but in this case it had seemingly just hung.

@kocio-pl

This comment has been minimized.

Show comment
Hide comment
@kocio-pl

kocio-pl Nov 24, 2017

Thanks, Tom! So should we just write here in case something seems to be stuck next time?

I'm also interested in the schedule - and if it's fired automatically or manually?

kocio-pl commented Nov 24, 2017

Thanks, Tom! So should we just write here in case something seems to be stuck next time?

I'm also interested in the schedule - and if it's fired automatically or manually?

@gravitystorm

This comment has been minimized.

Show comment
Hide comment
@gravitystorm

gravitystorm Nov 24, 2017

Contributor

The updates to the tileservers are handled via chef. All the configuration is available in the chef repo, although I realise it can be a bit intimidating to read, it's understandable particularly if you're used to how tileservers work.

execute "#{style_directory}/project.mml" do
action :nothing
command "carto -a 3.0.0 project.mml > project.xml"
cwd style_directory
user "tile"
group "tile"
subscribes :run, "git[#{style_directory}]"
notifies :restart, "service[renderd]", :immediately
notifies :restart, "service[update-lowzoom@#{name}]"
end

The above link shows the heart of the update process. It subscribes for notifications that the "git style_directory" resource has run, which happens when the config is updated for a new release. The execute resource that I linked to then runs the carto compiler, and issues two notifications - one to restart renderd, and the other to trigger the restarting of the update-lowzoom service. That service definition is here

So in short, it triggers automatically when the style gets updated.

In theory, you can use the chef repo to set up a perfect clone of the production tileservers, so that you can tinker with it and see how it works locally. We've set up a few of the cookbooks to work with test-kitchen, but not all of them yet. Unfortunately the tile cookbook isn't one of the ones that currently works with test-kitchen but hopefully someone can get that sorted.

If anyone needs a more thorough explanation of how the chef cookbooks work I'm happy to help. I'm not actually familiar with the OSMF tileserver setup but I'm experienced enough at reading Chef cookbooks to figure it out :-)

Contributor

gravitystorm commented Nov 24, 2017

The updates to the tileservers are handled via chef. All the configuration is available in the chef repo, although I realise it can be a bit intimidating to read, it's understandable particularly if you're used to how tileservers work.

execute "#{style_directory}/project.mml" do
action :nothing
command "carto -a 3.0.0 project.mml > project.xml"
cwd style_directory
user "tile"
group "tile"
subscribes :run, "git[#{style_directory}]"
notifies :restart, "service[renderd]", :immediately
notifies :restart, "service[update-lowzoom@#{name}]"
end

The above link shows the heart of the update process. It subscribes for notifications that the "git style_directory" resource has run, which happens when the config is updated for a new release. The execute resource that I linked to then runs the carto compiler, and issues two notifications - one to restart renderd, and the other to trigger the restarting of the update-lowzoom service. That service definition is here

So in short, it triggers automatically when the style gets updated.

In theory, you can use the chef repo to set up a perfect clone of the production tileservers, so that you can tinker with it and see how it works locally. We've set up a few of the cookbooks to work with test-kitchen, but not all of them yet. Unfortunately the tile cookbook isn't one of the ones that currently works with test-kitchen but hopefully someone can get that sorted.

If anyone needs a more thorough explanation of how the chef cookbooks work I'm happy to help. I'm not actually familiar with the OSMF tileserver setup but I'm experienced enough at reading Chef cookbooks to figure it out :-)

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes Nov 24, 2017

Member

To follow up a little more I should probably explain how the update-lowzoom service works...

Firstly as @gravitystorm says by the time it starts the style has been updated and renderd has been restart so any new tiles rendered will be in the new style from that point on.

What update-lowzoom does is to run render_old in a loop until it completes, using the timestamp of the new style as the reference time so that any z0-12 tiles older than that will be rerendered. In particular note that this ensures that if it does get restarted it will quickly skip any done by a previous pass.

Once render_old has completed all z0-12 tiles the planet file is touched with the time of the style file meaning that all high zooms tiles older than the style will now be rerendered if possible when requested.

What I did on orm yesterday was just to kill render_old so that the loop would restart it and finish off the rest of the low zoom tiles.

There is a separate script called render-lowzoom which runs on the first sunday of each month and rerenders the low zoom tiles in much the same way.

Member

tomhughes commented Nov 24, 2017

To follow up a little more I should probably explain how the update-lowzoom service works...

Firstly as @gravitystorm says by the time it starts the style has been updated and renderd has been restart so any new tiles rendered will be in the new style from that point on.

What update-lowzoom does is to run render_old in a loop until it completes, using the timestamp of the new style as the reference time so that any z0-12 tiles older than that will be rerendered. In particular note that this ensures that if it does get restarted it will quickly skip any done by a previous pass.

Once render_old has completed all z0-12 tiles the planet file is touched with the time of the style file meaning that all high zooms tiles older than the style will now be rerendered if possible when requested.

What I did on orm yesterday was just to kill render_old so that the loop would restart it and finish off the rest of the low zoom tiles.

There is a separate script called render-lowzoom which runs on the first sunday of each month and rerenders the low zoom tiles in much the same way.

@kocio-pl

This comment has been minimized.

Show comment
Hide comment
@kocio-pl

kocio-pl Nov 24, 2017

Thanks, that explains a lot!

I've made a link on osm-carto Wiki page, but I think that you could probably put these things in a place that is more visible for the rest of OSM community than a closed ticket in some obscure subproject 😄 (maybe https://blog.openstreetmap.org/category/operations/ would do? ).

This might look like a boring nerdy stuff, but for me it's a precious knowledge and I think this is a great way to make OWG more open and approachable. I remember, Andy, that you wanted to invite more people to work there, and I guess it could really work this way.

kocio-pl commented Nov 24, 2017

Thanks, that explains a lot!

I've made a link on osm-carto Wiki page, but I think that you could probably put these things in a place that is more visible for the rest of OSM community than a closed ticket in some obscure subproject 😄 (maybe https://blog.openstreetmap.org/category/operations/ would do? ).

This might look like a boring nerdy stuff, but for me it's a precious knowledge and I think this is a great way to make OWG more open and approachable. I remember, Andy, that you wanted to invite more people to work there, and I guess it could really work this way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment