Add MAPS.ME authentication #1433

Open
wants to merge 1 commit into
from

Projects

None yet

7 participants

@Zverik
Contributor
Zverik commented Feb 9, 2017 edited

We, as in maps.me, have published the first beta version of our own authentication service, passport.maps.me (does not have the front page yet). The plan is to use it instead of osm.org oauth to authenticate users, and then register them on osm.org using that account.

It would give us an independent profile storage and, more important, an option to use native SDKs for facebook and google on mobile devices. I could not negotiate that for osm.org (re: #1114 and mailing lists), so we decided to make an alternative service. It might be possible for other mobile apps to use the same service. That would be a benefit for other authors of OSM mobile editors, simplifying their sign-up workflow.

Until it is possible to link multiple social accounts to a profile (see #1274), we plan to use this only for new users, directing existing users to osm.org for all kinds of linked accounts, including facebook and google. Nothing is changed for the osm side, besides a new icon on the login page.

This pull requests adds two auth providers: mapsme for a regular web-based authentication via the passport site, and mapsme_token for a token-based authentication, which would allow skipping the passport site when logging in on a mobile device. The latter rewrites the provider string to mapsme, so there is no difference to the database which one of these is used.

You can test this at http://test.osmz.ru (sendmail does not work there, so please remind me to activate your account when you've registered).

@woodpeck
Contributor
woodpeck commented Feb 9, 2017

Please clarify "the plan is to use it instead of osm.org oauth to authenticate users, and then register them on osm.org using that account" - I'm not sure I understand this fully. Does this workflow still ensure that the human being operating the account will always see our sign-on page, contributor terms, and everything, or could this be sacrificed for a "simplified workflow"?

@Zverik
Contributor
Zverik commented Feb 9, 2017 edited

Contributors are always directed to a proper OSM sign-up page: they check their e-mail and account name, read the terms and so on. Nothing is changed on that part. This PR only adds a new authentication provider, and the reason for that I tried to explain.

What I forgot to mention is that e-mail addresses from passport.maps.me are considered verified: we take these only from verified sources (google and facebook at the moment).

@tomhughes
Member

Hmm I'm surprised we are considering facebook provided emails as verified as I don't believe they are verified in any real sense - like most sites they will probably have verified it when the user signed up but that might have been years ago.

Google is different because they only provide gmail address and as they own those they know what is really valid at any given point in time.

@tomhughes
Member

Hmm apparently @Zverik did that ;-) Not sure how you got me to accept it though!

@tomhughes
Member

Before we go any further you're going to have to explain better what the second provider is all about - from a quick look at the patch it's defined as an extra omniauth provider but isn't hooked up to anything.

Can you give a detailed explanation of the flow that uses that, and why it needs to be a separate provider rather than piggybacking on the other one?

@Zverik
Contributor
Zverik commented Feb 9, 2017 edited

Hmm I'm surprised we are considering facebook provided emails as verified as I don't believe they are verified in any real sense - like most sites they will probably have verified it when the user signed up but that might have been years ago.

The same is true for OSM emails. In #1096 we found evidence that Facebook only provides verified emails, but if we take into account that all addresses become obsolete with time, then we should not treat anything to be verified, and even maybe ask users to re-verify their email addresess once a year.

Can you give a detailed explanation of the flow that uses [mapsme_token provider], and why it needs to be a separate provider rather than piggybacking on the other one?

The provider is included in omniauth-mapsme gem. After a successful authentication it replaces the :provider field with mapsme, imitating a sign-in with that provider. It is built after the omniauth-facebook-access-token, which is also invisible to the target app.

Regular OAuth2 flow from the mapsme provider is:

  1. User goes to %osm%/auth/mapsme
  2. User is redirected to https://passport.maps.me/oauth/authorize?...&state=399whatever99
  3. User authorized against maps.me server, if not yet
  4. User is redirected back to %osm%/auth/mapsme/callback?code=...&state=399whatever99
  5. Server sends a request to passport.maps.me with the given code and receives an access token
  6. Server uses that token to query the provider for user id and email
  7. User is redirected to a relevant page, or %osm%/user/new.

Now, if we already have an access token, we would want to skip steps 2-5 and simply give the server the token. It is not bound to an application, so you can make requests even from a different country using the same token. Alas, we have no way of giving the token in that workflow: it has only intermediate steps.

The workflow with a prepared token is:

  1. User goes to %osm%/auth/mapsme_token/callback?access_token=123whatever456
  2. Server uses that token to query the provider for user id and email
  3. User is redirected to a relevant page, or %osm%/user/new.

In theory I could merge these two strategies into one, checking in callback_phase whether the access_token parameter is specified, but it would be harder than just to make two separate strategies.

With a mobile app, when signing in to OSM via a facebook, the following workflow would be used:

  1. Sign in to Facebook or Google using a Native SDK and get an fb_access_token
  2. Use fb_access_token to sign in to passport.maps.me using a series of API calls, and get an access_token
  3. Open a WebView for https://www.openstreetmap.org/auth/mapsme_token/callback?access_token=... and wait for it to return osm_access_token and osm_access_secret.

This way a user won't have to type their password for anything, since WebView doesn't have cookies from a native sdk or from sessions in a web browsing app. And even if that was okay, Google will start blocking OAuth requests in a WebView starting this April.

@tomhughes
Member

So on the basis of that it seems to me that this is missing tests for the token based flow?

@tomhughes
Member

As far as "verified" emails go I understand what you're saying but we really want to be sure we can at least send a working welcome email, which means knowing that the email is working right now.

That's why I am surprised I agreed to make that change as my intention had been to ensure that whatever email we had was working at the time they signed up even if it later stopped working.

@Zverik
Contributor
Zverik commented Feb 9, 2017

So on the basis of that it seems to me that this is missing tests for the token based flow?

Possibly. I'll see how to do them then.

As far as "verified" emails go I understand what you're saying but we really want to be sure we can at least send a working welcome email, which means knowing that the email is working right now.

So why do we have verified email providers then? You can register in Google with your other, non-gmail email, which I did. Google does not send anything to that email, apart from rare google docs notifications. You could have registered in Yahoo Mail 20 years ago and use the account only for authentication. Verification flag for these was introduced in 104727f in 2012. Are you saying we don't trust any third-party server now and the e-mail should be validated in all cases?

@Zverik
Contributor
Zverik commented Feb 9, 2017

Also, what's the point of testing the email address on registering, when it can be easily abandoned 5 minutes after that (re: mailinator)? We don't send any vital information to it. The best way to ensure email validity is to provide a user with a reason they should give a correct email, not by performing checks that can be easily circumvented.

Sending an email to the entered address serves only as a way of validation that the user has not made any typos when entering it. When you get a similarly validated email from a third-party service, there is no need to validate it again, since it was done before.

@tomhughes
Member

As far as I know the google auth provider always gives you back the gmail address which is guaranteed to exist. At least that was my assumption when I made it auto-verify.

The key point as I say isn't so much the test (that's mostly to stop you signing up somebody else) but the ability to send a welcome message, for which we only need to be able to send a message right now.

@Zverik
Contributor
Zverik commented Feb 9, 2017

But we don't send a welcome message, only "please click this link to verify" email. The welcome message is displayed after registering regardless of whether a user clicked a link or registered with a pre-verified email.

@Zverik
Contributor
Zverik commented Feb 9, 2017

As far as I know the google auth provider always gives you back the gmail address which is guaranteed to exist. At least that was my assumption when I made it auto-verify.

Just tried to sign in with Google. It gave my old non-gmail address with which I registered in Google ages ago. It still works, but I doubt google tests its existence after the first time.

And even it's a gmail address that exists, you cannot verify that a user actually uses that email and not some other in other mailing service. I registered in google with my actual address just by chance, I have another google account with a proper gmail address which I check only once every 2-3 years.

@Zverik Zverik Add MAPS.ME authentication
3868260
@Zverik
Contributor
Zverik commented Feb 16, 2017

Added a test for mapsme_token provider. Since it basically mimics the mapsme provider, I decided to add just one.

What doubts do you have that prevent merging this request?

@zerebubuth
Contributor

If we don't currently verify email addresses before activating the account (i.e: allowing edits), then I think we should change the code to do that before adding any new authentication mechanisms. I think it's important that the website supports the community, and that can only happen if user messages and changeset comments actually reach the intended recipient.

As @Zverik points out, we might already be making a false assumption about the value of "guarantees" provided by large, established providers such as Google and Facebook. I think we should check our assumptions thoroughly before going any further.

@tomhughes
Member

We verify for everything except google and facebook.

In the case of google that was because I believed they were only giving us gmail accounts that were implicitly valid by virtue of coming from google. In the case of facebook I think I wasn't really thinking and bought into that claim of facebook having validated them.

Given the evidence presented here I think we should probably stop trusting both.

@tomhughes
Member

Correction, we also trust Yahoo, on the basis as Google, that I assumed they were only giving us Yahoo Mail accounts.

@iandees
Contributor
iandees commented Feb 16, 2017 edited

All of these providers verify that these emails exist: Facebook won't let you sign up without confirming the email you used to sign up with was valid (by clicking on a link sent to you). Google will periodically ask if your backup email is valid, but I don't think there's a single provider out there that actively validates the email after initial signup (by periodically sending an email). That seems like an odd requirement.

@tomhughes
Member

Well that's exactly the point - we want the email to be valid at the point when they signup with us so we need to revalidate.

The logic in the case of Google and Yahoo was that there was no way to sign in with them without the mail being valid but we now know that Google don't always provide the gmail address and I know that I can sign in with Yahoo without having a Yahoo Mail address.

@Zverik
Contributor
Zverik commented Feb 16, 2017 edited

What I am trying to say, is that

  1. We don't need a working e-mail at the sign-up time. Yes we are checking it, by sending a confirmation letter, but after that a few months or even years can pass before a user receives their first e-mail from osm.org.

  2. A belief that e-mail confirmation ensures a e-mail address is working, is not based on anything. It's like these security procedures in airports: they look complex, scary and effective, but when tested, they turn out to be much hassle for regular passengers and transparent for attackers.

For example, I have around 30 accounts on osm.org, most of which I made for teaching or for testing, and while all e-mails from the website are confirmed, I obviously don't check these e-mail accounts. In fact, I've registered another one with mailinator a week ago for testing this reply.

The only reason for confirmation e-mails is to validate the typed address. When we believe that a user wants to receive e-mails and types his address by hand in the relevant field, we send a confirmation e-mail to verify that they haven't made any typos. It is obviously redundant when we get an address from a third-party service, which has verified it the same way.

So even if you remove the line allowing for google, facebook and yahoo users to not use the confirmation link, you won't get better e-mail addresses or protect yourself from mailinator and similar services. Getting a proper e-mail address from a user is not a techical task, but a designer one. To make a user enter their actual working address, you have to show them the value of messages we will send them. The current registration page does not do that (e.g. we don't even assure that we won't send any spam), so we might get some fake e-mails. Again, the reason is not that we don't send confirmation e-mails for some registrations.

@tomhughes
Member

I'm sorry, but you're just wrong.

It is simply not true that "the only reason for confirmation e-mails is to validate the typed address" because our confirmation e-mail is also a welcome email with helpful links.

Even when we don't do confirmation we still send that email, just without the confirmation link, and we want to be sure they are getting it. The only way to be sure (if we don't trust that the email is valid when the auth provider says it is) is to require them to confirm.

@tomhughes
Member

Hmm actually the confirmation email says we will send the welcome email once they've confirmed, but the effect is the same.

That said I can't actually see where we send it!

@tomhughes
Member

Ah actually we stopped doing that in 2690342 and the reference in the confirmation email is actually to us redirecting to the welcome page when signup is complete.

I'm not sure that was a good idea though as I think there are lots of ways to signup that will avoid them ever seeing that welcome page..

@Zverik
Contributor
Zverik commented Feb 16, 2017

...our confirmation e-mail is also a welcome email with helpful links.

Could you please direct me to the code that sends that e-mail? I have never seen it, even when I tested registration from scratch a week ago.

In some countries users even set their welcoming services like this one for Belgium, for Italy or for Russia just to send useful links to new users.

@Zverik
Contributor
Zverik commented Feb 16, 2017

I'm not sure that was a good idea though as I think there are lots of ways to signup that will avoid them ever seeing that welcome page..

Could you name one please?

@tomhughes
Member

Well to start with probably anything that started as a link to a page that needs authentication and hence had a referer parameter attached to specify where to send the user when signup was complete.

@tomhughes
Member

In any case regardless of all that I'm still not really comfortable with the idea that people can signup and be completely uncontactable from the start.

Obviously they can change their email address down the line and it's hard for us to do anything about that, but new mappers are the ones that we are most likely to need to contact anyway.

@Zverik
Contributor
Zverik commented Feb 16, 2017

What I am trying to say, you cannot solve this with technical measures. It is an UI problem.

@Zverik
Contributor
Zverik commented Feb 16, 2017

Well to start with probably anything that started as a link to a page that needs authentication and hence had a referer parameter attached to specify where to send the user when signup was complete.

Just tested it: the referer is lost after redirecting to the terms page.

@BushmanK

The problem of being able to reach any particular user should be seen (and tested) as a system of real cases. From one, where a user honestly wants to receive all possible feedback, to an opposite, where a user does not want to be "bugged" - he just "wants to fix/add something". All these scenarios are real and there is no corresponding UX study (even very simple one) to back it. I'm insisting on UX, not UI as @Zverik suggests because user's intentions and general behavior should be taken into considerations in addition to his reactions on UI. UI might make it easier to do something, but it can never make someone do it if he doesn't want or wants to do it differently.

The current situation, as @tomhughes just said, is based on a whole bunch of assumptions. Obviously, it creates a ridiculous situation with people, trying to reach other people with a welcoming message, changeset comments or any other matters, actually sending it to nowhere.

@simonpoole
Contributor
simonpoole commented Feb 17, 2017 edited

I don't think that the arguments that boil down to "verifying e-mail at sign up isn't perfect so we shouldn't be doing it" hold any water at all. It does give the community a fighting chance to contact new mappers.

Now you could argue that we should allow users to configure additional (social media) communications means, but that is still not going to help with the people that "don't want to be bugged".

I'll have a look at what happens with the editors a zero-hour block (or any matter of fact) and if that can be improved this weekend. See #1443

@BushmanK

That is exactly what I'm pointing at. Currently, purpose and importance of having a valid email of a newly registered user do not seem to be clearly defined. However, it doesn't mean that we don't need it at all or that we need it whatever it takes (like, weekly revalidation).

@Zverik
Contributor
Zverik commented Feb 22, 2017

Okay, I suggest extracting the question about validity of facebook / google emails to a separate issue, since it may require more discussion that is not immediately relevant to this pull request. I'm ready to organize an IRC or Mumble meeting.

In the context of what is already present on the website, adding mapsme-provided emails to me are good, since we get these only from facebook and google. And from OSM, when #1431 is merged. Are there any other outstanding issues?

@zerebubuth
Contributor

In the context of what is already present on the website, adding mapsme-provided emails to me are good, since we get these only from facebook and google.

If the user already has a Facebook or Google account, can they use that to log in instead? I think we already support both of those.

And from OSM, when #1431 is merged. Are there any other outstanding issues?

I would be against merging #1431. And, to be honest, against merging this PR as well. I don't understand the need for this as an alternative to the existing providers, and I worry that this is part of an effort to hide OSM rather than welcome new users into the community.

If people want to contribute fixes without being part of the community, then do we need to think about ways to stage anonymous "suggested edits" for community members to verify and "adopt" as their own?

@Zverik
Contributor
Zverik commented Feb 22, 2017

If the user already has a Facebook or Google account, can they use that to log in instead? I think we already support both of those.

We don't support native sign-in on mobile device — you must remember a big discussion of a year ago, in which I tried to push that into the website. And as I said in this comment, Google will start blocking webview authentication forms in less than two months. MAPS.ME is essentially making a way for mobile apps to authenticate in OSM using native social SDKs.

And please, MAPS.ME doesn't hide OSM. We show a user the complete registration process, every webpage. The name of the project is prominently displayed on each page. We mention OpenStreetMap in all our PR materials. We have links to OpenStreetMap on profile and about pages.

As for the community, I think that is the myth that became obsolete around 2012. We don't have a single community anymore, like e.g. Facebook does not have a community of facebook users. What matters is mappers answering e-mails, and this is a step in that direction.

@zerebubuth
Contributor

Google will start blocking webview authentication forms in less than two months.

The fix that Google suggest is to use "in-app tabs" instead:

Modern “in-app browser tab” patterns available on some operating systems, such as Chrome Custom Tabs on Android and SFSafariViewController on iOS offer further UX improvements for browser-based OAuth flows.

The blog post goes on to say how "in-app tabs" are also an improvement for security. Would Google's suggestion be an easier (and possibly more secure) upgrade path for anyone currently doing OAuth in a WebView?

As for the community, I think that is the myth that became obsolete around 2012. We don't have a single community anymore, like e.g. Facebook does not have a community of facebook users.

I wasn't suggesting that there is a single community. I was suggesting that we might look for alternative routes for people to contribute if they're not interested in becoming part of the wider OSM community or communities.

I think that it's important that we're able to audit the data in OSM, and contact members to ask whether they need help, start discussions about how to map things, or check what their original sources were.

What matters is mappers answering e-mails, and this is a step in that direction.

Please could you explain how this PR helps? At the moment, I can't see how it adds anything we don't already have in our Google and Facebook integrations.

@simonpoole
Contributor
simonpoole commented Feb 22, 2017 edited

I don't see anything in https://www.theregister.co.uk/2016/08/23/google_to_block_web_views_from_using_its_oauth/ indicating that an app accessing the OSM API won't be able to use a webview for the the user to authorize a OAuth request on osm.org.

What they seem to be saying is that you wont be able to run a web app in a webview that uses OAuth. I don't think we really care about that or that it has anything at all to do with the question at hand.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment