Alternative use of https

[following5]

Opencaching.de is going to implement secure access. It is planned to enable parallel use of both https://www.opencaching.de and http://www.opencaching.de so that non-SSL-enabled or even non-redirect-enabled clients will not be broken.

OKAPI may be handled independently from this, but it would be nice to have it run with https, too. This would mean that SITE_URL would be set to either of both URLs. At a first glance, it looks like OKAPI can handle this, but I guess that it has not been fully tested in such an environment?

Are there any full URLs including the protocol stored in OKAPI tables or files?

[wrygiel]

I don't think that the code requires any changes to support SSL. But the documentation might. Developers might wonder if they should pick HTTP or HTTPS scheme - pros and cons.

[following5]

The usage of the SITE_URL setting needs verification. Generally, I think it should reflect the protocol used in the current request, so that any OKAPI links and redirects stay within the same protocol. At first glance I found one minor issue with that: here the protocol in SITE_URL should be replaced by 'http', to be consistent with the installations.xml file.

[wrygiel]

Perhaps we need to split it into two dynamically calculated variables - SITE_ID and SITE_HTTPX_URL? At the same time, the SITE_URL settings would be made private to avoid misuse.

BTW, I will be offline for the most of the next week.

[following5]

Perhaps we need to split it into two dynamically calculated variables - SITE_ID and SITE_HTTPX_URL? At the same time, the SITE_URL settings would be made private to avoid misuse.

Agree. I can do that.

I have scanned all direct and indirect uses of SITE_URL and found the following special cases; line numbers refer to ver. 1113:

Notice about adding cron5 to the crontab

• cronjobs.php, line 458
• update.php, 363

I think this should show always show http:....

Installations

• installation.php, lines 26 and 27
• installations.php, lines 69 and 71
• menu.inc.php, line 73

I think this should always reflecte the site ID, which always starts with 'http:'.

• installations_box.tpl.php, line 5
• introduction.tpl.php, line 48

As long as there are no OC nodes which redirect everything to https, I think it is fine to present the site ID here, too. It is unlikely that this situation will change in the upcoming few years.

Besides of this, it would be nice if the installations list informs the user if https is available on a site. I suggest to add a new settings value and an installations.xml attribute for this boolean flag.

Cache attribution note

• geocaches.php, line 1370

There are links to the origin site and to the cache listing. This must not depend on the request's protocol, due to replication. The site ID should do it.

Method documentation

• method_doc.tpl.php, line 45

A full URL is shown in the docs for each method. I think it should show 'http:...' for non-https-enabled sites, and either 'http[s]:...' or both URLs for https-enabled sites.

Site reference in GPX data

• gpxfile.tpl.php, lines 20 and 22

I am not sure what to do here. May it confuse client software if it receives changing OC URLs in GPX files?

[following5]

Maybe we should right on add a "primary site URL" setting, to get all links independent from the site ID. This would currently be "http:..." for all OC sites but may change to "https:..." if a site decides to do so. This "primary site URL" would be used for

• the link in the node selection box in the upper right corner
• the site links on the introduction page
• the attribution note links
• the URLs in GPX data

[wrygiel]

I think that we may return HTTPS in many cases, even when the client is making its request with HTTP. This is because - even if the client prefers HTTP over HTTPS - most of the URLs themselves are usually destined to end up in the user's browser location bar, and the browser will handle HTTPS just fine.

However, it's hard to discuss cases without the actual code in front of us. :) Perhaps just commit your ideas to a separate branch (in the common repository so that both of us will be able to commit there), and we will set up a chat (or Skype meeting) and talk about it later?

[wrygiel]

We will also need to include a new cronjob for verifying server configuration. It will need to verify that OKAPI methods can still be called with OKAPI's base_url and HTTP scheme (if the site responds with a HTTP redirect response, then there's an error and admins need to be notified).

[Can I really edit YOUR comments? Wow. -- following5]
[Yeap, that'a a bit weird of GitHub, but I guess it's okay - anyone with commit rights can edit any comment -- wrygiel]

[following5]

Redirecting an OC site to https is an option which is far in the future, and it looks like it will make the OKAPI implementation too expensive (for me). Another point is that I don't know if the OC.de site will run stable on https right from start and therefor ewill declare https as "experimental". Several issues have already popped up while testing, and more may come after launch (in September?).

This said, I would prefer a simple and consistently dual-protocol OKAPI implementation. What about this:

• implement your SITE_ID / SITE_HTTPX_URL suggestion
• add an 'https' field to settings and to installations.xml with one of the following three values: not available, available or recommended.

It's up to the maintainer of okapi_settings.php to ensure that these settings correspond to the server's capabilities. OKAPI may verify if the 'https' settings field is consistent with installations.xml.

An additional setting PREFERED_SITE_URL will be generated from this information, which will point to https if it is "recommended" and to http else. And then one of SITE_ID, SITE_HTTPX_URL or PREFERED_SITE_URL will be used for all the cases listed above.

If you can generally agree to this, I can do a sample implementation.

[wrygiel]

Okay, I see your point. Sounds okay! I'm not sure if I understand the difference between SITE_HTTPX_URL and PREFERRED_SITE_URL though.

[following5]

Ok, the PREFERRED_SITE_URL ist not needed. Here is the sample implementation, not fully tested: https://github.com/following5/okapi/tree/https

The definition of the site_url field as returned by services/apisrv/installation(s) made me some headache. After discarding several implementation experiments, I came to this definition:

• The SITE_URL in okapi_settings.php, the new internal setting SITE_ID and the site_url returned by installation(s) are the same. I will call it "site ID".
• This site ID always has the form of a valid and usable URL.
• After assigned once, the site ID for an OKAPI installation should never change. So if one of the existing nodes decides to redirect everything to https (which maybe will never happen), it will keep an 'http' OKAPI site ID.
• I am not sure if it should ever be allowed for a new node to have an 'https' site ID (e.g. if it starts right away redirecting everything to https), because that would render the site ID useless as URL if the certificate expires. Anyway, I assume that there may appear an https site ID.

Building on this definition, I was able to implement what we had discussed above:

• SITE_URL is replaced by SITE_ID and SITE_HTTPX_URL in the Settings class;
• an HTTPS setting is added with the options unavailable, available, recommended and enforced; I have added enforced for future extension, it is not implemented yet in OKAPI and will throw an exception;
• an https field is added to /etc/installations.xml and to the output of the installation(s) methods, which corresponds to the HTTPS setting;
• client developers are adviced to evaluate the https field and to replace the scheme in the site_url field by something else if desired and applicable.

Generally, SITE_HTTPX_URL will be used for references within and by OKAPI. For the special cases listed in my earlier post, I did this:

• The crojob docs output the SITE_ID for the wget string.
• When identifying and comparing installations, the SITE_ID is used.
• The topright site selector and the site list on the intro page will output an https site URL if https is recommended or enforced, else an http URL.
• In attribution notes, an http URL will be linked, except if HTTPS is enforced. This minimizes the need for a huge replication update when changing settings.
• The method documentations show the SITE_HTTPX_URL.
• The < url > field in GPX files currently shows the SITE_ID, while the cache link is based on the SITE_HTTPX_URL. The optimum for the < url > field may be same as with the third point above.

I hope that is makes some sense.

There is one drawback: Developers have to actively insert 'https' into the URLs to make use of it. If they don't, they will continue with http even if https is desirable and recommended. But I think that returning 'https://...' as installation site_url may break some existing clients, e.g. using wget without SSL support. Not to speak of the complications to the internal usage of the installation(s) methods if they return something else than the site ID.

[wrygiel]

Quite a complex subject! I am preparing my review of your commit, but it comes very slowly.

A quick side note: When are you planning on introducing SSL to your site? Have you managed to set up and use HTTPS protocol on the OCPL-DEVEL VM?

[following5]

Jep, it's complex. OCDE code had to be adjusted too, and I am doing this without https experience, hoping that it will work out. OC.de will move to a new webhoster soon (October? I thought it would be September, but takes some more time), and we plan to start SSL with the new server. We may start it with declaring OKAPI as non-https if it is not https-ready.

There is a (German) Explanation by bohrsty on setting up SSL/https on our old developer VM, which has a customized Apache. mambofive (Elmar) is preparing a new developer VM which will be SSL- and OCPL-enabled, but that may take some more weeks. You may contact him for more information or for assistance in setting up an OC.de devel VM.

[following5]

We have now started https operation: https://www.opencaching.de/
I have adviced the users to stay with http for OKAPI.

[wrygiel]

Totally forgot about this issue. I had unfinished commits in my repository
in regard to this issue, but AFAIR they are not "required" and can wait.

Did you manage to set it up without any changes to OKAPI?

On Wed, Dec 23, 2015 at 11:36 AM, following5 notifications@github.com
wrote:

We have now started https operation: https://www.opencaching.de/
I have adviced the users to stay with http for OKAPI.

—
Reply to this email directly or view it on GitHub
#343 (comment).

Tylko niewielu ludzi, którzy cierpią na kaszel, idzie do lekarza. Większość
udaje się do teatru. - Alec Guinness
http://rygielski.net/

[following5]

Yes, the OC.de https operation is independent from OKAPI. The OKAPI change would just give additional security to OKAPI users. (Also, we currently could not go for "complete SSL security", as OKAPI allows http access to some private data. But we don't consider https-only anyway.)

[following5]

Meanwhile, the SITE_URL and OKAPI_BASE_URL settings at www.opencaching.de are either https://... or http://..., depending on the protocol of the current request. So they are no longer directly comparable to the site_url and okapi_base_url entries of the installations list.

d6ad7a6 solves this problem for the top-right menu. I am not aware of any other functions which compare the two "site_url" strings.

Now OKAPI works pretty well in dual-mode http / https. Before proceeding, OCPL sites should decide how they want to deal with https (opencaching/opencaching-pl#363).

[wrygiel]

Generally, I think it should reflect the protocol used in the current request, so that any OKAPI links and redirects stay within the same protocol.

Won't the users just keep using HTTP, then? They won't even know that HTTPS is supported.

BTW: Browsers show warnings when navigating from HTTPS to HTTP, but not the other way around (afair, you can even have HTTPS images embedded on HTTP page without triggering any warnings). If you intend for your users to switch to HTTPS eventually, then isn't it better to return HTTPS links even when OKAPI method is called via HTTP?

[following5]

They won't even know that HTTPS is supported.

Some developers are already using it. For the other, we can announce it in the OKAPI changelog and blog.

Opencaching.de https is still declared "experimental". We will switch to another certificate in a few months, and when that is successfully done we may declare it "recommended".

If you intend for your users to switch to HTTPS eventually, then isn't it better to return HTTPS links even when OKAPI method is called via HTTP?

I would like to leave the choice up to the developers if they use http or https, like other APIs (e.g. Google Maps) do it. There may be applications which are not https-enabled and would break on https links.

[following5]

Meanwhile, PL, DE and UK sites are https-only. DE has done this step a week ago, and there are no reports of any broken OKAPI clients. So no need to implement the proposed dual-scheme operation in OKAPI.

[andrixnet]

NL and RO will transition towards https as well in the near future.