Skip to content
This repository was archived by the owner on May 14, 2025. It is now read-only.

[KnownUsernameField] Entries update (main ones)#1034

Merged
cscharf merged 5 commits intomasterfrom
KnownUsernameField--entries-update--main-ones
Aug 7, 2020
Merged

[KnownUsernameField] Entries update (main ones)#1034
cscharf merged 5 commits intomasterfrom
KnownUsernameField--entries-update--main-ones

Conversation

@contribucious
Copy link
Mannequin

@contribucious contribucious mannequin commented Aug 6, 2020

CONTEXT: This is an update of this new system (system allowing "user ID" field detection — i.e. email/username/phone/whatever — without "password" field using the accessibility service).

UPDATED: Entries.

RELATED: #880 (comment)


 

Summary

 
This fixes support for:

  • [see post n°1 below] Google

This adds missing OAuth support for:

  • [see post n°2 below] PayPal

This adds support for:

  • [see post n°3 below] national + desktop Amazon — the latter uses a different value.
  • [see post n°4 below] national eBay
  • [see post n°5 below] Atlassian
  • [see post n°6 below] Bitly — enterprise users.
  • [see post n°7 below] Tumblr
  • [see post n°8 below] Yandex
  • [see post n°9 below] ... + My docomo from NTT DOCOMO — in a separate section, as part of a Top 20 Japan.
     

💡 For all: Both the mobile version and the desktop version of these web sites/applications have been tested and are supported.
 

About OAuth authentication

Read me ...

 

Taken from the source code:

// NOTE: The case of OAuth compatible web sites/applications that also provide
//       a "user ID only" login page in this situation
//       was taken into account in the tests as well.

↪️ See screenshots in the posts below for web sites/applications using a "user ID only" login page also for OAuth authentication.
 

OAuth usage examples ...

OAuth_Google_Yandex_example
➡️ Example of login using OAuth (right), in this case with a lot of choices because on a support forum of a website selling ... an OAuth module for a known CMS and a popular forum system (and using it as a proof of proper functioning on its own website and forum).

OAuth_Atlassian_Bitbucket_example
➡️ Example of login using OAuth, in this case on AppVeyor.

   

About n°6/7/8/9 (coming from the Top 100 WW)

Read me ...  

These were added as part of a standard verification (checked one by one, looking for "user ID only" login pages, in two rounds — round 1: desktop mode, round 2: mobile mode) of a recent Top 100 of the most visited websites in the world(*, see warning before) (source: SimilarWeb 2019) followed by the same process for this Top 50 (source: Alexa Internet 2020).

For the Top 100, about a dozen used a "user ID only" login page, but the majority was well coded, allowing Bitwarden to display a prompt automatically.

Among the remaining contenders:

  • One was not to be added in the main section (customer area of a Japanese mobile phone operator, despite being the first one there in Japan, namely NTT DOCOMO). Added but in a separate section, as part of a Top 20 Japan.
  • Two did not have an "id" attribute regarding their login field (mail.ru both in desktop and mobile version + baidu.com mobile).
  • ... There were ultimately three entries that were addable: Bitly (.com), Tumblr (.com) and Yandex (.com/.ru/various TLDs). :octocat:

top-100-websites-worldwide
 

(*) WARNING > Two entries in this Top 100 list are visibly known to be a source of adware/malware: tsyndicate (position 86) and crptgate (position 95). Also note that microsoftonline.com (position 52) is indeed a domain name belonging to Microsoft, despite what the image above indicates.

 
 

ℹ️ Based on my research for this PR, I will take the opportunity to improve a little bit this file as well. 👍

…mblr+Yandex

... also, add missing OAuth support for PayPal. And add "My docomo" from NTT DOCOMO, as part of a Top 20 Japan.
@contribucious
Copy link
Mannequin Author

contribucious mannequin commented Aug 6, 2020

 
Google_Logo


This:

  • fixes support for Google — accounts.google.com.
     

Proof of proper functioning

View me ...  
  • 📱 Mobile: A

  • 💻 Desktop: A

  • 📱 Mobile (using OAuth):

    • A (1st step: we click on "Continue with Google" on a website offering this type of login)
    • B (2nd step: we are redirected ...)
    • C (3rd step: we can log in with our Google account and the Bitwarden prompt appears successfully)
  • 💻 Desktop (using OAuth):

    • A (1st step: we click on "Continue with Google" on a website offering this type of login)
    • B (2nd step: we are redirected ...)
    • C (3rd step: we can log in with our Google account and the Bitwarden prompt appears successfully)

Technical screenshots

View me ...  
  • 📱 Mobile: AB

  • 💻 Desktop: A

Technical details

View me ...  
new KnownUsernameField("accounts.google.com", new (string, string)[] { ("identifier", "identifierId"), ("ServiceLogin", "Email") }),

This:

  • keeps the existing ("ServiceLogin", "Email"):

Google_ServiceLogin_v1
↪️ Still used today (desktop/mobile) in some cases. OAuth compatible. (Official image from the Bitwarden Firefox extension page.)
 

  • adds ("identifier", "identifierId"):

Google_ServiceLogin_v2
↪️ V2. Used in most cases. OAuth compatible.
 
 
This additional entry covers:

Standard auth.

  • /ServiceLogin/identifier?[…] (used when existing Google account(s) is/are listed / just after a logout)
  • /signin/v2/identifier?[…] (used when no existing Google account(s) is/are listed)

OAuth

  • /signin/oauth/identifier?[…] (used when a Google account is used to connect to a third party website such as eBay, Reddit, etc.)
  • /o/oauth2/v2/auth/identifier?[…] (same note as the entry above — variant)

@contribucious
Copy link
Mannequin Author

contribucious mannequin commented Aug 6, 2020

 
PayPal_Logo


This:

  • adds missing OAuth support for PayPalpaypal.com.
     

Proof of proper functioning

View me ...

 

💡 About the example used (Xoom, a PayPal service), see its Wikipedia page.

@contribucious
Copy link
Mannequin Author

contribucious mannequin commented Aug 6, 2020

 
Amazon_Logo


This:

  • adds support for national versions of Amazon (17 entries, not counting the existing .com) and
  • adds support for desktop versions of Amazon (all 18 entries are concerned).
     

Amazon_Country_List

✔️ All entries have been tested.
 

Technical remarks

View me ...    

❓ Why contains:/ap/signin used?

Because I encountered during my tests:

  • /-/XX/ap/signin?[…] ➡️ XX being the non-default language.
  • /ap/signin?[…] ➡️ used almost all the time.
  • /ap/signin/000-0000000-0000000?[…] ➡️ the 0 representing in this case any number. However, currently only appears to be on the mobile version of .cn and on the first visit only, it seems*. Can be reproduced**.
(*) To be completely precise, I once had this also on the desktop version of .co.jp but could not reproduce it. However, I was able to find this also on the mobile version of .com, on an screenshot included in a blog post from Bitwarden (from 2017 however, when Amazon was still using a "user ID + password" login page but this is unrelated, see the case of .cn).
(**) For the .cn, see screenshot A (page) and B (autofill OK). Access to the mobile website without prior trace required. Also confirmed using an incognito window from Google Chrome on my computer in mobile simulation mode — this mode needs to be set before accessing amazon.cn!

❔ But why not contains:/signin or even contains:signin?

Because Amazon doesn't use a separate subdomain for the login, so it would match the page of this object for example among others (found by searching for the keyword signin): https://www.amazon.com/signing-time-first-signs-dvd/dp/b000r3441g
 
 

Proof of proper functioning

View me ...  

amazon.ae: mobiledesktop
amazon.ca: mobiledesktop
amazon.cn: mobiledesktop

amazon.co.jp: mobile A + mobile Bdesktop
↪️ CURRENTLY: uses ap_email also for mobile, has user ID + password fields on the same page on the mobile version.

amazon.co.uk: mobiledesktop

amazon.com: mobiledesktop
💡 mobile support was already present.

amazon.com.au: mobiledesktop
amazon.com.br: mobiledesktop
amazon.com.mx: mobiledesktop
amazon.com.tr: mobiledesktop
amazon.de: mobiledesktop
amazon.es: mobiledesktop
amazon.fr: mobiledesktop
amazon.in: mobiledesktop
amazon.it: mobiledesktop
amazon.nl: mobiledesktop

amazon.sa: mobile A + mobile Bdesktop
↪️ CURRENTLY: has user ID + password fields on the same page on the mobile version.
↪️ INFO: this is the latest one, freshly launched: amazon.sa — see this in the news. 💡

amazon.sg: mobile A + mobile Bdesktop
↪️ CURRENTLY: has user ID + password fields on the same page on the mobile version.

Technical screenshots

View me ...  

amazon.ae: mobile
amazon.ca: mobile
amazon.cn: mobile

amazon.co.jp: mobile A + mobile B
↪️ CURRENTLY: uses ap_email also for mobile, has user ID + password fields on the same page on the mobile version.

amazon.co.uk: mobile

amazon.com: mobile
💡 mobile support was already present.

amazon.com.au: mobile
amazon.com.br: mobile
amazon.com.mx: mobile
amazon.com.tr: mobile
amazon.de: mobile
amazon.es: mobile
amazon.fr: mobile
amazon.in: mobile
amazon.it: mobile
amazon.nl: mobile

amazon.sa: mobile A + mobile B
↪️ CURRENTLY: has user ID + password fields on the same page on the mobile version.

amazon.sg: mobile A + mobile B
↪️ CURRENTLY: has user ID + password fields on the same page on the mobile version.

Regarding desktop

See here for .com; and here for .fr as an example of a national variant — all identical, verification done.

💻 About Desktop mode

View me ...  

Used in particular on tablets, sometimes automatically forced via an option available on certain browsers. So, useful to have. Besides, on my 10.5" tablet (2560x1600 but websites displayed in 1280x800) using Google Chrome, Amazon provides me with its Desktop version automatically (see here and here when the "Desktop site" option of Google Chrome is unchecked, and compare with here and here when "Desktop site" is checked — it's identical).

📘 Technical information that nobody cares

View me ...  

➡️ It is particularly essential here to put the mobile value (ap_email_login) before the desktop value (ap_email) because it turns out that ap_email is also used on the mobile version page but for another use*.

(*) In an account creation subsection located in first position**, displayed instantly when a radio button is tapped — see A (before tapped) and B (after tapped).
(**) Just for information, note that a useless prompt will always be available in this subsection by the way, probably as the first input[type="email"] field found.

@contribucious
Copy link
Mannequin Author

contribucious mannequin commented Aug 6, 2020

 
eBay_Logo


This:

  • adds support for national versions of eBay (20 entries, not counting the existing .com).
    ↪️ Each language variant has been included, regarding the login part (only Belgium and Canada have this particularity).
     

eBay_Country_List
↪️ 50 entries displayed (including "United States") but less than half of that number has been added; this is mostly because many western websites use signin.ebay.com and some eastern websites use signin.ebay.com.hk — see the last section of this post for more details on this.

✔️ All entries have been tested.
 
 

💡 For information, no difference between the mobile and desktop version, regarding the login part (I mean from a technical point of view, i.e. it's always userid).
 
 

Technical remarks

View me ...  

Why case-insensitive used — i.e. iendswith/icontains?

Because eBay returns for these parts of address a 200 OK whatever the case (instead of redirecting to the correct case).

 

Proof of proper functioning

View me ...  

signin.befr.ebay.be: mobiledesktop (all hidden: mobile alt & desktop alt)
signin.benl.ebay.be: mobiledesktop
signin.cafr.ebay.ca: mobiledesktop
signin.ebay.at: mobiledesktop
signin.ebay.be: mobiledesktop
signin.ebay.ca: mobiledesktop
signin.ebay.ch: mobiledesktop
signin.ebay.co.uk: mobiledesktop

signin.ebay.com: mobiledesktop
💡 .com support was already present.

signin.ebay.com.au: mobiledesktop
signin.ebay.com.hk: mobiledesktop
signin.ebay.com.my: mobiledesktop
signin.ebay.com.sg: mobiledesktop
signin.ebay.de: mobiledesktop
signin.ebay.es: mobiledesktop
signin.ebay.fr: mobiledesktop
signin.ebay.ie: mobiledesktop
signin.ebay.it: mobiledesktop
signin.ebay.nl: mobiledesktop
signin.ebay.ph: mobiledesktop
signin.ebay.pl: mobiledesktop

Regarding direct access (by going to https://signin.ebay.tld/)

View me ...

See below for .com

  • 📱 Mobile: AB

  • 💻 Desktop: AB

See below for .be (as an example of a national variant — all identical, verification done)

  • 📱 Mobile: AB

  • 💻 Desktop: AB

Technical screenshots

View me ...

See below for .com

  • 📱 Mobile: AB

  • 💻 Desktop: AB

See below for .fr (as an example of a national variant — all identical, verification done)

  • 📱 Mobile: AB

  • 💻 Desktop: AB

📘 Technical information that nobody cares

View me ...  

Have been removed from the list ...

 

www.gittigidiyor.com
global.gmarket.co.kr/Home/Main

➡️ Entries for Turkey and Korea respectively.
↪️ Removed because not really eBay as such / websites too different from traditional eBay (and have the two fields — user ID + password — on the same page anyway).
 
 

www.ebay.co.th (redirects to sellercenter.ebay.co.th)

➡️ Entry for Thailand.
↪️ Removed because this website is only a sales guide for Thai sellers to sell on ebay.com. And the .com uses signin.ebay.com.

💡 Informative page dedicated to buyers (translated)
 
 

www.ebay.co.jp

➡️ Entry for Japan.
↪️ Removed because this website is a sales guide + a management portal for Japanese sellers to sell more easily on ebay.com. And this portal (https://eportal.ebay.co.jp/) uses signin.ebay.com for its access.

💡 About this portal intended for sellers (translated)
💡 Informative page dedicated to buyers (translated)
 
 

www.ebay.cn
www.ebay.com.tw

➡️ Entries for China and Taiwan respectively.
↪️ Removed because both use signin.ebay.com.hk — or ... a variant from ebay.com.hk which always has the two fields "user ID + password" — (but maybe only when accessed from outside these countries though?).

💡 For info, in this case too, both are intended for sellers.
     Buyers (including those within these countries) need to use www.ebay.com (or another one).

 
 

ar.ebay.com
bo.ebay.com
br.ebay.com
by.ebay.com
cl.ebay.com
co.ebay.com
cr.ebay.com
do.ebay.com
ec.ebay.com
gt.ebay.com
hn.ebay.com
il.ebay.com
kz.ebay.com
mx.ebay.com
ni.ebay.com
pa.ebay.com
pe.ebay.com
pr.ebay.com
pt.ebay.com
py.ebay.com
ru.ebay.com
sv.ebay.com
uy.ebay.com

+

www.ebay.in (which redirects to in.ebay.com)
www.ebay.se (which redirects to www.ebay.com)

➡️ Entries for various countries.
↪️ Removed because all use signin.ebay.com (when tested/accessed from Belgium at least).
 
 

💡 Note that, regarding the mobile version, you can have during your tests on the various TLDs once a "user ID + password" page, another time a "user ID only" page. Served randomly.

 


UPDATE — 2020/08/14

Spotted a special case, that of eBay India.
↪️ See PR #1041.

@contribucious
Copy link
Mannequin Author

contribucious mannequin commented Aug 6, 2020

 
Atlassian_Logo


This:

Proof of proper functioning

View me ...  
  • 📱 Mobile: Generic loginBitbucket login

  • 💻 Desktop: Generic loginBitbucket login

  • 📱 Mobile (using OAuth):

    • A (1st step: we click on "Continue with Bitbucket" — but can be any web application from Atlassian — on a website offering this type of login)
      (we are then redirected to id.atlassian.com ...)
    • B (2nd step: we can log in with our Atlassian account and the Bitwarden prompt appears successfully)
  • 💻 Desktop (using OAuth):

    • A (1st step: we click on "Continue with Bitbucket" — but can be any web application from Atlassian — on a website offering this type of login)
      (we are then redirected to id.atlassian.com ...)
    • B (2nd step: we can log in with our Atlassian account and the Bitwarden prompt appears successfully)

Technical screenshots

View me ...  

@contribucious
Copy link
Mannequin Author

contribucious mannequin commented Aug 6, 2020

 
Bitly_Logo


This:

  • adds support for Bitly (enterprise users)bitly.com.
     

Proof of proper functioning

View me ...  
  • 📱 Mobile: A (the login page for enterprise users)B (... which can be accessed from this link: "Log in with SSO")

  • 💻 Desktop: A (the login page for enterprise users)B (... which can be accessed from this link: "Log in with SSO")

Technical screenshots

View me ...  
  • 📱 Mobile: A

  • 💻 Desktop: A

@contribucious
Copy link
Mannequin Author

contribucious mannequin commented Aug 6, 2020

 
Tumblr_Logo


This:

  • adds support for Tumblrtumblr.com.
     

Proof of proper functioning

View me ...  
  • 📱 Mobile: A

  • 💻 Desktop: A

Technical screenshots

View me ...  
  • 📱 Mobile: A

  • 💻 Desktop: A

 

💡 Thanks to the way Tumblr is coded, Bitwarden will even be able to automatically fill in the (hidden) password field in the process (in one go), password field that will be displayed in the next step! The login action is therefore even faster on this website!

@contribucious
Copy link
Mannequin Author

contribucious mannequin commented Aug 6, 2020

 
Yandex_Logo


This:

✔️ All entries have been tested.
 

Proof of proper functioning

View me ...  
  • 📱 Mobile: dot com / dot ru (as an example of a national TLD — all identical, verification done)

  • 💻 Desktop: dot com / dot ru (as an example of a national TLD — all identical, verification done)

  • 📱 Mobile (using OAuth):

    • A (1st step: we click on "Continue with Yandex" on a website offering this type of login)
      (we are then redirected to passport.yandex.{by, com, com.tr, kz, ru, ua, uz} ...)
    • B (2nd step: we can log in with our Yandex account and the Bitwarden prompt appears successfully)
  • 💻 Desktop (using OAuth):

    • A (1st step: we click on "Continue with Yandex" on a website offering this type of login)
      (we are then redirected to passport.yandex.{by, com, com.tr, kz, ru, ua, uz} ...)
    • B (2nd step: we can log in with our Yandex account and the Bitwarden prompt appears successfully)

Technical screenshots

View me ...  
  • 📱 Mobile: dot com / dot ru (as an example of a national TLD — all identical, verification done)

  • 💻 Desktop: dot com / dot ru (as an example of a national TLD — all identical, verification done)

 
Source for the list of TLDs used by Yandex: https://yandex.com/support/passport/troubleshooting/forgot-password.html (ya.ru doesn't have a passport. subdomain).

 
 


UPDATE — 2020/08/13

I recently contacted Yandex support by email to ensure the completeness of this list of TLDs used by Yandex. It is in fact much longer.
↪️ Therefore, PR to come. 👍

UPDATE — 2020/08/19

Pull request made: #1044.

UPDATE — 2020/08/21

Pull request now merged. 👍

@contribucious
Copy link
Mannequin Author

contribucious mannequin commented Aug 6, 2020

 
My_docomo_Logo
 
(in a separate section, as part of a Top 20 Japan)


This:

  • adds support for My docomocfg.smt.docomo.ne.jp.
     

Technical remarks

View me ...  

/auth/ has been used because access to the login page is technically possible via:

Proof of proper functioning

View me ...  

Technical screenshots

View me ...  
  • 📱 Mobile:
    anidlogin_mltdom (default page): A / B
    idauth (page displayed after a login failure): A / B

  • 💻 Desktop:
    anidlogin_mltdom (default page): A / B
    idauth (page displayed after a login failure): A / B

@contribucious
Copy link
Mannequin Author

contribucious mannequin commented Aug 6, 2020

@kspearrin Hello Kyle. In view of my next PR, in case this information is not considered confidential, would it be possible for me to know the 5 (at the very least, the 3) countries where the Bitwarden Android app is the most installed/used? This, in order to enrich this new section C (Top 20 for selected countries) of the KnownUsernameField system, to improve the user experience for a large number of people.

For information, on average, less than 3 entries will be added per country with a Top 20 system (because only web sites/applications using a "user ID only" login page are to be added).

Thank you in advance! 👍

@contribucious
Copy link
Mannequin Author

contribucious mannequin commented Aug 6, 2020

P.S. Sorry for the number of consecutive posts, but this is for greater clarity. 👍

@kspearrin
Copy link
Mannequin

kspearrin mannequin commented Aug 6, 2020

@contribucious United States, France, Germany are the top 3

@contribucious
Copy link
Mannequin Author

contribucious mannequin commented Aug 6, 2020

@kspearrin OK, thank you very much!

➡️ Note: This will be the subject of a separate PR though.

@cscharf
Copy link
Mannequin

cscharf mannequin commented Aug 6, 2020

Thank you for the explicit detail @contribucious , will be reviewing this shortly!

@contribucious
Copy link
Mannequin Author

contribucious mannequin commented Aug 6, 2020

@cscharf No problem! And thank you!

P.S. The series of post updates earlier concerned the addition of a space line just after "View me ..." (when the latter is clicked/opened) in almost each post, for better readability. Be sure to get the refreshed version considering the length of reading! ☺️

cscharf
cscharf mannequin approved these changes Aug 7, 2020
Copy link
Mannequin

@cscharf cscharf mannequin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent, thank you. And, btw, "Technical information that nobody cares", I certainly do and was great background and much appreciated detail.

@cscharf cscharf mannequin merged commit 39de2c1 into master Aug 7, 2020
@contribucious
Copy link
Mannequin Author

contribucious mannequin commented Aug 7, 2020

☺️

Thanks to you for all the follow-up! 👍

@contribucious contribucious mannequin deleted the KnownUsernameField--entries-update--main-ones branch August 7, 2020 15:44
@contribucious
Copy link
Mannequin Author

contribucious mannequin commented Aug 13, 2020

POSTS UPDATE

♦️  2020/08/08 — eBay

@cscharf Speaking of which, I've just updated this section of the eBay post. More precise and detailed. ☺️

♦️  2020/08/13 — eBay

Small addition about eBay China (www.ebay.cn) and eBay Taiwan (www.ebay.com.tw), in the eBay post (last section).

♦️  2020/08/13 — Yandex

Information and details concerning a PR to come ... See the footer of the Yandex post.

♦️  2020/08/14 — eBay

  1. Small addition again about eBay China (www.ebay.cn) and eBay Taiwan (www.ebay.com.tw), in the eBay post (last section).

  2. Small modification concerning this new addition.

  3. ❗ Spotted a special case, that of eBay India. PR link added to the footer of the eBay post.

♦️  2020/08/19 — Yandex

Pull request made. Link added to the footer of the Yandex post.

♦️  2020/08/21 — Yandex

Pull request now merged. Link added to the footer of the Yandex post.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants