New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move to https by default #190

Open
tomhughes opened this Issue Jan 4, 2018 · 36 comments

Comments

Projects
None yet
10 participants
@tomhughes
Member

tomhughes commented Jan 4, 2018

The majority of our services are already forcing https by redirecting any http requests. I'd like to move the remaining services to do the same. I believe that to be:

  • www.openstreetmap.org
  • tile.openstreetmap.org
  • gps-tile.openstreetmap.org
  • nominatim.openstreetmap.org
  • planet.openstreetmap.org
  • wiki.openstreetmap.org
  • dev.openstreetmap.org
  • os.openstreetmap.org
  • ooc.openstreetmap.org
  • agri.openstreetmap.org

There are some technical issues with the main web site that I need to work on but the rest is easy enough I believe, unless there are concerns about load.

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes Jan 4, 2018

Member

Quick attempt at a check list of things to do:

  • www.openstreetmap.org
    • figure out a way to resolve oauth issues
    • add redirect
  • tile.openstreetmap.org
    • make sure render servers have suitable certificates
    • upgrade backhaul to use https
    • update leaflet-osm etc to always use https
    • add redirect (assuming we're happy with potential query increase)
  • gps-tile.openstreetmap.org
    • add redirect
  • nominatim.openstreetmap.org
    • check if @lonvia has any worries around load
    • add redirect
  • planet.openstreetmap.org
    • persuade users to switch to using https (osmium doesn't follow redirects)
    • add redirect
  • wiki.openstreetmap.org
    • check if @Firefishy knows of any reason we're not already doing it
    • add redirect
  • dev.openstreetmap.org
    • make sure server has suitable certificates
    • add redirect
  • os.openstreetmap.org
    • add redirect
  • ooc.openstreetmap.org
    • make sure server has suitable certificates
    • add redirect
  • agri.openstreetmap.org
    • add redirect

Once all that is in place we can think about deploying HSTS across everything.

Member

tomhughes commented Jan 4, 2018

Quick attempt at a check list of things to do:

  • www.openstreetmap.org
    • figure out a way to resolve oauth issues
    • add redirect
  • tile.openstreetmap.org
    • make sure render servers have suitable certificates
    • upgrade backhaul to use https
    • update leaflet-osm etc to always use https
    • add redirect (assuming we're happy with potential query increase)
  • gps-tile.openstreetmap.org
    • add redirect
  • nominatim.openstreetmap.org
    • check if @lonvia has any worries around load
    • add redirect
  • planet.openstreetmap.org
    • persuade users to switch to using https (osmium doesn't follow redirects)
    • add redirect
  • wiki.openstreetmap.org
    • check if @Firefishy knows of any reason we're not already doing it
    • add redirect
  • dev.openstreetmap.org
    • make sure server has suitable certificates
    • add redirect
  • os.openstreetmap.org
    • add redirect
  • ooc.openstreetmap.org
    • make sure server has suitable certificates
    • add redirect
  • agri.openstreetmap.org
    • add redirect

Once all that is in place we can think about deploying HSTS across everything.

@Firefishy

This comment has been minimized.

Show comment
Hide comment
@Firefishy

Firefishy Jan 4, 2018

Collaborator

I'm not aware of any reason why we shouldn't enabled HTTPS by default on wiki.openstreetmap.org (with a redirect). There is a likelihood that there will be some 3rd parties who many need to enable HTTPS support in their code eg: taginfo.

Collaborator

Firefishy commented Jan 4, 2018

I'm not aware of any reason why we shouldn't enabled HTTPS by default on wiki.openstreetmap.org (with a redirect). There is a likelihood that there will be some 3rd parties who many need to enable HTTPS support in their code eg: taginfo.

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes Jan 4, 2018

Member

Taginfo already redirects everything to https anyway.

Member

tomhughes commented Jan 4, 2018

Taginfo already redirects everything to https anyway.

@lonvia

This comment has been minimized.

Show comment
Hide comment
@lonvia

lonvia Jan 4, 2018

I have no big worries about load for nominatim.openstreetmap.org.

I suspect though that for nominatim and planet there are quite a few scripts in active use which cannot handle redirects. Maybe we should consider some kind of pre-announcement.

lonvia commented Jan 4, 2018

I have no big worries about load for nominatim.openstreetmap.org.

I suspect though that for nominatim and planet there are quite a few scripts in active use which cannot handle redirects. Maybe we should consider some kind of pre-announcement.

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes Jan 4, 2018

Member

Really? Are there many http clients that don't handle redirects?

I guess main shell scripts using curl, which for reasons I've never understood doesn't do redirects unless you add the -L option.

Member

tomhughes commented Jan 4, 2018

Really? Are there many http clients that don't handle redirects?

I guess main shell scripts using curl, which for reasons I've never understood doesn't do redirects unless you add the -L option.

@SomeoneElseOSM

This comment has been minimized.

Show comment
Hide comment
@SomeoneElseOSM

SomeoneElseOSM Jan 4, 2018

Taginfo already redirects everything to https anyway.

It does, but not all the country ones necessarily do (taginfo.openstreetmap.org.uk for example doesn't seem to have a valid certificate on it). Obviously that's a separate issue (and I'll mention it to the maintainer of that site), and a fairly low-level issue since it's not the main one.

SomeoneElseOSM commented Jan 4, 2018

Taginfo already redirects everything to https anyway.

It does, but not all the country ones necessarily do (taginfo.openstreetmap.org.uk for example doesn't seem to have a valid certificate on it). Obviously that's a separate issue (and I'll mention it to the maintainer of that site), and a fairly low-level issue since it's not the main one.

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes Jan 4, 2018

Member

Well whatever - wiki is already available over https and lots of people use it that way so it can't exactly have any major problems.

Member

tomhughes commented Jan 4, 2018

Well whatever - wiki is already available over https and lots of people use it that way so it can't exactly have any major problems.

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes Jan 5, 2018

Member

I now have that is (hopefully) a workaround the the OAuth issues on the main site and I have that running at https://tomh.apis.dev.openstreetmap.org/ if anybody wants to test OAuth against that (using an http URL that will redirect).

Member

tomhughes commented Jan 5, 2018

I now have that is (hopefully) a workaround the the OAuth issues on the main site and I have that running at https://tomh.apis.dev.openstreetmap.org/ if anybody wants to test OAuth against that (using an http URL that will redirect).

@pnorman

This comment has been minimized.

Show comment
Hide comment
@pnorman

pnorman Jan 5, 2018

Collaborator

Testing on planet.openstreetmap.org revealed that osmosis doesn't follow redirects: https://trac.openstreetmap.org/ticket/5483

This can be worked around by switching the http://planet.openstreetmap.org to https://planet.openstreetmap.org in the osmosis configuration.txt file, but users need time to make the change.

I'm writing an email to inform people.

Collaborator

pnorman commented Jan 5, 2018

Testing on planet.openstreetmap.org revealed that osmosis doesn't follow redirects: https://trac.openstreetmap.org/ticket/5483

This can be worked around by switching the http://planet.openstreetmap.org to https://planet.openstreetmap.org in the osmosis configuration.txt file, but users need time to make the change.

I'm writing an email to inform people.

@genodeftest

This comment has been minimized.

Show comment
Hide comment
@genodeftest

genodeftest Jan 17, 2018

Adding HSTS to the wiki would be nice. www. and dev. have it already. I don't know about the others.

genodeftest commented Jan 17, 2018

Adding HSTS to the wiki would be nice. www. and dev. have it already. I don't know about the others.

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes Jan 17, 2018

Member

It should have it already.

Member

tomhughes commented Jan 17, 2018

It should have it already.

@genodeftest

This comment has been minimized.

Show comment
Hide comment
@genodeftest

genodeftest Jan 17, 2018

@tomhughes wrote:

It should have it already.

I don't see it.

genodeftest commented Jan 17, 2018

@tomhughes wrote:

It should have it already.

I don't see it.

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes Jan 17, 2018

Member

openstreetmap/chef@5239c5f should have fixed it.

Member

tomhughes commented Jan 17, 2018

openstreetmap/chef@5239c5f should have fixed it.

@genodeftest

This comment has been minimized.

Show comment
Hide comment
@genodeftest

genodeftest Jan 19, 2018

@tomhughes:
Now I see it on the wiki too. Thanks!

genodeftest commented Jan 19, 2018

@tomhughes:
Now I see it on the wiki too. Thanks!

@mmd-osm

This comment has been minimized.

Show comment
Hide comment
@mmd-osm

mmd-osm Feb 1, 2018

Could we also add https://wiki.openstreetmap.org/wiki/Slippy_Map_MediaWiki_Extension to the todo list? It's still using http to fetch tiles, including mixed content warnings. Those seem to be hard coded here: https://www.openstreetmap.org/openlayers/OpenStreetMap.js

// ping: @harry-wood

mmd-osm commented Feb 1, 2018

Could we also add https://wiki.openstreetmap.org/wiki/Slippy_Map_MediaWiki_Extension to the todo list? It's still using http to fetch tiles, including mixed content warnings. Those seem to be hard coded here: https://www.openstreetmap.org/openlayers/OpenStreetMap.js

// ping: @harry-wood

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes Feb 1, 2018

Member

The only that extension needs is banishing to the pits of doom.

I have absolutely no plans to change that archived copy of OpenLayers which nobody (including that extension!) should be using anyway - we only keep it around because so many people in the early days decided it was a good idea to load OpenLayers from our site.

Member

tomhughes commented Feb 1, 2018

The only that extension needs is banishing to the pits of doom.

I have absolutely no plans to change that archived copy of OpenLayers which nobody (including that extension!) should be using anyway - we only keep it around because so many people in the early days decided it was a good idea to load OpenLayers from our site.

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes Feb 1, 2018

Member

Hmm are we actually the canonical source for that file in fact? or is that part of upstream OpenLayers?

Still changing it is scary, and breaking all the sites loading OL from us does have a certain appeal if it makes some of them upgrade ;-)

Member

tomhughes commented Feb 1, 2018

Hmm are we actually the canonical source for that file in fact? or is that part of upstream OpenLayers?

Still changing it is scary, and breaking all the sites loading OL from us does have a certain appeal if it makes some of them upgrade ;-)

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes Feb 1, 2018

Member

Looks like that file is actually our code - there is a generic OSM layer in upstream and it creates layer specific classes derived from that.

In any cases the images are passive content so won't currently be a problem - the big problem is that the js is active content so won't load over http from an https context. That will need fixing in the extension itself.

Member

tomhughes commented Feb 1, 2018

Looks like that file is actually our code - there is a generic OSM layer in upstream and it creates layer specific classes derived from that.

In any cases the images are passive content so won't currently be a problem - the big problem is that the js is active content so won't load over http from an https context. That will need fixing in the extension itself.

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes Feb 1, 2018

Member

Hmm those loads are protocol relative. That's weird then...

Member

tomhughes commented Feb 1, 2018

Hmm those loads are protocol relative. That's weird then...

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes Feb 1, 2018

Member

It's actually bombing out creating an instance of OpenLayers.Control.Button because there is no such class in that OL we are serving. That's why you just get pink tiles.

Member

tomhughes commented Feb 1, 2018

It's actually bombing out creating an instance of OpenLayers.Control.Button because there is no such class in that OL we are serving. That's why you just get pink tiles.

@mmd-osm

This comment has been minimized.

Show comment
Hide comment
@mmd-osm

mmd-osm Feb 1, 2018

@tomhughes : Thanks! Switch to https looks good here, Javascript warnings no longer show up in the console. I will also check with the user who originally reported the issue on the Forum, if the tiles are now visible in IE11, or if there's still some other issue.

mmd-osm commented Feb 1, 2018

@tomhughes : Thanks! Switch to https looks good here, Javascript warnings no longer show up in the console. I will also check with the user who originally reported the issue on the Forum, if the tiles are now visible in IE11, or if there's still some other issue.

@pnorman pnorman referenced this issue Mar 2, 2018

Closed

Redirect planet.openstreetmap.org to HTTPS #200

3 of 5 tasks complete

grischard added a commit to grischard/leaflet-osm that referenced this issue Apr 6, 2018

@grischard

This comment has been minimized.

Show comment
Hide comment
@grischard

grischard May 11, 2018

Quick update:

  • tile.openstreetmap.org

    • update leaflet-osm etc to always use https
  • planet.openstreetmap.org

    • persuade users to switch to using https (osmium doesn't follow redirects)
    • add redirect

grischard commented May 11, 2018

Quick update:

  • tile.openstreetmap.org

    • update leaflet-osm etc to always use https
  • planet.openstreetmap.org

    • persuade users to switch to using https (osmium doesn't follow redirects)
    • add redirect
@grischard

This comment has been minimized.

Show comment
Hide comment
@grischard

grischard May 11, 2018

Also, the tile.openstreetmap.org servers reply with a HSTS header even over HTTP, which will strongly encourage HTTPS already. I think a good next step would be to add that redirect there.

grischard commented May 11, 2018

Also, the tile.openstreetmap.org servers reply with a HSTS header even over HTTP, which will strongly encourage HTTPS already. I think a good next step would be to add that redirect there.

@Firefishy

This comment has been minimized.

Show comment
Hide comment
@Firefishy

Firefishy May 11, 2018

Collaborator

@grischard The HSTS spec makes clear clients MUST NOT trust HSTS header from HTTP sources. I don't think it causes any harm if we publish HSTS header on HTTP and clients ignore it.

Collaborator

Firefishy commented May 11, 2018

@grischard The HSTS spec makes clear clients MUST NOT trust HSTS header from HTTP sources. I don't think it causes any harm if we publish HSTS header on HTTP and clients ignore it.

@grischard

This comment has been minimized.

Show comment
Hide comment
@grischard

grischard May 11, 2018

@Firefishy yes, and many follow them anyway, which makes it a neat soft redirect hack. There's a discussion about it at #117 (comment)

grischard commented May 11, 2018

@Firefishy yes, and many follow them anyway, which makes it a neat soft redirect hack. There's a discussion about it at #117 (comment)

@grischard

This comment has been minimized.

Show comment
Hide comment
@grischard

grischard May 24, 2018

I see that wiki.openstreetmap.org and plain openstreetmap.org currently don't set the HSTS header. Should we enable it everywhere, add includeSubDomains, and add preload eventually if we're happy?

grischard commented May 24, 2018

I see that wiki.openstreetmap.org and plain openstreetmap.org currently don't set the HSTS header. Should we enable it everywhere, add includeSubDomains, and add preload eventually if we're happy?

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes May 25, 2018

Member

Both those domains seem to set it for me:

bericote [~] % HEAD -P https://wiki.openstreetmap.org/ | fgrep Strict-Transport-Security 
Strict-Transport-Security: max-age=31536000
bericote [~] % HEAD -P https://openstreetmap.org/ | fgrep Strict-Transport-Security     
Strict-Transport-Security: max-age=31536000

I think hot.openstreetmap.org is the main outstanding issue for preload?

Member

tomhughes commented May 25, 2018

Both those domains seem to set it for me:

bericote [~] % HEAD -P https://wiki.openstreetmap.org/ | fgrep Strict-Transport-Security 
Strict-Transport-Security: max-age=31536000
bericote [~] % HEAD -P https://openstreetmap.org/ | fgrep Strict-Transport-Security     
Strict-Transport-Security: max-age=31536000

I think hot.openstreetmap.org is the main outstanding issue for preload?

@grischard

This comment has been minimized.

Show comment
Hide comment
@grischard

grischard May 26, 2018

Ah, I see, it's set on https://wiki.openstreetmap.org/wiki/Main_Page but not on https://wiki.openstreetmap.org that redirects to it, and it's set on https://www.openstreetmap.org but not on https://openstreetmap.org that redirects to it. I don't know if that's moot or not for now, but we'll need it on https://openstreetmap.org the day we add preload, I think.

The *.hotosm.org certificate expires in two months, so now would be a good time for them to work on this.

grischard commented May 26, 2018

Ah, I see, it's set on https://wiki.openstreetmap.org/wiki/Main_Page but not on https://wiki.openstreetmap.org that redirects to it, and it's set on https://www.openstreetmap.org but not on https://openstreetmap.org that redirects to it. I don't know if that's moot or not for now, but we'll need it on https://openstreetmap.org the day we add preload, I think.

The *.hotosm.org certificate expires in two months, so now would be a good time for them to work on this.

@grischard

This comment has been minimized.

Show comment
Hide comment
@grischard

grischard May 26, 2018

CC'ing @smit1678 - would it please be possible to add a letsencrypt certificate for hot, humanitarian, hot-dev and hot-staging.openstreetmap.org on amelia.hotosm.org? Alternatively, if only hot.openstreetmap.org is being used, and it's being used only for a redirect, we can bring hot.openstreetmap.org back to the osm infrastructure, have a certificate there, and just redirect to hotosm.org from there.

grischard commented May 26, 2018

CC'ing @smit1678 - would it please be possible to add a letsencrypt certificate for hot, humanitarian, hot-dev and hot-staging.openstreetmap.org on amelia.hotosm.org? Alternatively, if only hot.openstreetmap.org is being used, and it's being used only for a redirect, we can bring hot.openstreetmap.org back to the osm infrastructure, have a certificate there, and just redirect to hotosm.org from there.

@bgirardot

This comment has been minimized.

Show comment
Hide comment
@bgirardot

bgirardot May 26, 2018

Hi all,

Stereo contacted HOT via IRC and we now have a ticket for getting this supported on our end.

I have created a ticket for it in our system as well, which was auto linked above in this issue.

@smith1678 or HOT's sys admin @dakotabenjamin will follow up.

cheers

bgirardot commented May 26, 2018

Hi all,

Stereo contacted HOT via IRC and we now have a ticket for getting this supported on our end.

I have created a ticket for it in our system as well, which was auto linked above in this issue.

@smith1678 or HOT's sys admin @dakotabenjamin will follow up.

cheers

@smit1678

This comment has been minimized.

Show comment
Hide comment
@smit1678

smit1678 May 29, 2018

Alternatively, if only hot.openstreetmap.org is being used, and it's being used only for a redirect, we can bring hot.openstreetmap.org back to the osm infrastructure, have a certificate there, and just redirect to hotosm.org from there.

@grischard Thanks for flagging this. It looks like it's also redirecting hot.osm.org as well. I think it's probably best if we do this on the OSM infrastructure since this is a openstreetmap.org subdomain and just a redirect. Having it continue to redirect to hotosm.org would be great. We've been slowly transitioning some things off of the server that it's currently pointing to and so I think for the long term it would be best for that subdomain managed with everything else for osm.org.

If that's not the case, we can certainly set up the certificate to do the redirect correctly.

smit1678 commented May 29, 2018

Alternatively, if only hot.openstreetmap.org is being used, and it's being used only for a redirect, we can bring hot.openstreetmap.org back to the osm infrastructure, have a certificate there, and just redirect to hotosm.org from there.

@grischard Thanks for flagging this. It looks like it's also redirecting hot.osm.org as well. I think it's probably best if we do this on the OSM infrastructure since this is a openstreetmap.org subdomain and just a redirect. Having it continue to redirect to hotosm.org would be great. We've been slowly transitioning some things off of the server that it's currently pointing to and so I think for the long term it would be best for that subdomain managed with everything else for osm.org.

If that's not the case, we can certainly set up the certificate to do the redirect correctly.

@grischard

This comment has been minimized.

Show comment
Hide comment
@grischard

grischard May 29, 2018

@tomhughes would hosting the hot.o.o redirect ourselves be ok?

Also, I don't know if you saw my message on IRC: I'm thinking of doing a PR for a full squid setup for the tiles, it should be enough for now, and less work than varnish while still making an eventual future switch possible. Good idea?

grischard commented May 29, 2018

@tomhughes would hosting the hot.o.o redirect ourselves be ok?

Also, I don't know if you saw my message on IRC: I'm thinking of doing a PR for a full squid setup for the tiles, it should be enough for now, and less work than varnish while still making an eventual future switch possible. Good idea?

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes May 29, 2018

Member

@smit1678 We are also pointing hot-dev.osm.org and hot-staging.osm.org at you but they appear, for some reason, to redirect to https://en.wikipedia.org/wiki/Amelia_Earhart so I'm guessing you don't use those any more?

Member

tomhughes commented May 29, 2018

@smit1678 We are also pointing hot-dev.osm.org and hot-staging.osm.org at you but they appear, for some reason, to redirect to https://en.wikipedia.org/wiki/Amelia_Earhart so I'm guessing you don't use those any more?

@smit1678

This comment has been minimized.

Show comment
Hide comment
@smit1678

smit1678 May 30, 2018

@tomhughes Yes, that's correct, we don't use those either. Amelia is the nickname of the server and that redirect was implemented a while ago.

smit1678 commented May 30, 2018

@tomhughes Yes, that's correct, we don't use those either. Amelia is the nickname of the server and that redirect was implemented a while ago.

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes May 30, 2018

Member

I have moved hot.openstreetmap.org to point at one of our machines that has a valid certificate and just redirects to www.hotosm.org now.

Member

tomhughes commented May 30, 2018

I have moved hot.openstreetmap.org to point at one of our machines that has a valid certificate and just redirects to www.hotosm.org now.

@tomhughes

This comment has been minimized.

Show comment
Hide comment
@tomhughes

tomhughes May 30, 2018

Member

@grischard by "full squid" you mean dropping nginx and using squid to terminate https connections as well as http? that sounds good to me...

Member

tomhughes commented May 30, 2018

@grischard by "full squid" you mean dropping nginx and using squid to terminate https connections as well as http? that sounds good to me...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment