Google Sign-In issues (fix and workaround available!)

[The-Compiler]

Given that information about this is a bit scattered at the moment, I'm opening (and pinning) this issue in the hope that it's easier to find that way. Here's what's going on:

Explanation

• Mid 2019, Google started blocking various smaller browsers from logging in to a Google account. This initially only affected few people, but seems to be rolled out to many Google accounts now (but not all of them, my own accounts are still unaffected...)
• Google claims that you're using "a browser [...] that doesn't allow us to keep your account secure". There's not much further rationale available for that claim - all I'm aware of is a blogpost titled "Better protection against Man in the Middle phishing attacks" where this originally was announced and a Google Help Center entry which also mentions that this change is targetting automation testing frameworks and browsers embedded in a different application. The change seems to affect much more than that - also see a Bleepingcomputer article and related Reddit discussions (/r/kde, /r/privacy and others).
• It looks like Google tries to block the QtWebEngine library which could indeed be used to build applications using a Google account by letting the user sign in and then "stealing" the cookie instead of e.g. using OAuth to log in and some API. It looks like blocking various browsers (including qutebrowser) just is collateral damage.

Fix and workaround

• Fortunately, it was discovered that setting the user agent to Firefox bypasses whatever weird check Google is doing.
• With qutebrowser v1.9.0 (2019-01-08), a content.site_specific_quirks option was added (see Site-specific quirks overview #4810 and Refactor user agent handling and introduce site-specific quirks #5157 for details). With it enabled (which is the default), qutebrowser sends a Firefox user agent for the Google Accounts page (and other problematic pages like WhatsApp), thus fixing the issue.
• As a workaround, you can get the equivalent of that fix by running :set -u https://accounts.google.com/* content.headers.user_agent 'Mozilla/5.0 (X11; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0' (needs v1.2.0 or newer) (see below for an updated workaround)

Jan 2021 Update

According to some people, this might be coming back - but not all Google Accounts seem to be affected, or at least not to the same degree. If you're seeing this with your account, please answer here!

Aug 2021 Update

This came back for most (all?) accounts now, with the Edge user agent qutebrowser uses as a workaround in v1.14.0 (db13e52). Thankfully, the Firefox one still works.

• The v2.3.1 release contains an updated workaround and should work out of the box
• For versions between v1.10.0 (inclusive) and v2.3.1 (exclusive), run :set -u https://accounts.google.com/* content.headers.user_agent "Mozilla/5.0 ({os_info}; rv:90.0) Gecko/20100101 Firefox/90.0" for an equivalent workaround.
• For versions between v1.2.0 (inclusive) and v1.10.0 (exclusive), use X11; Linux x86_64 in place of {os_info}.
• For versions older than v1.2.0, no workaround is available due to missing per-domain settings. Setting the user agent to Firefox globally is strongly discouraged as it breaks a variety of other pages.

[The-Compiler]

Also, let me add my personal opinion on this, from an earlier mailinglist mail:

I wouldn't be surprised if Google just blocks pretty much everything other than Chrome and maybe Firefox. It's ridiculous, but it's yet another sign showing that maybe it isn't a good thing to make yourself dependent on Google - you're absolutely at their mercy, and they don't exactly have a good track record.

I've once tried helping someone who only got an internal server error when trying to open an important attachment of an older mail in GMail. We contacted Google Support, and as you'd predict, nothing happened.

Given that Google really doesn't care about its users, your only option is either switching browsers, or moving away from Google. I did the latter years ago[*]. Seeing what's going on here, this absolutely was the right decision, but I'm aware this isn't something everyone can realistically do...

[*] At least for stuff like Calendar and Mail. I still use Youtube (mostly via mpv/youtube-dl), Google Play and sometimes Maps (mostly switched to Openstreetmap/OSMand).

[JonnyHaystack]

Sorry for being possibly a bit off-topic, but out of interest what replacements do you use for GMail and Google Calendar?

[The-Compiler]

• GMail: uberspace - other providers I've seen people use are (in no particular order) Posteo, Protonmail and Tutanota.
• Google Calendar: I host a Nextcloud instance on uberspace (guide) and use DAVx⁵ to sync it with Android.

[JonnyHaystack]

Thanks! I'll check out all of those

[The-Compiler]

For anyone watching this issue: I just released qutebrowser v1.9.0 with the fix.

[jb55]

Sorry for being possibly a bit off-topic, but out of interest what replacements do you use for GMail and Google Calendar?

I run radicale and sync with DAVx⁵ as well. radicale works great! I haven't had any issues with it.

[The-Compiler]

I don't want to turn this into a conversation about Google, but either way: I just read Why I quit using Google – Kyle Piira which is a nice (even if horrifying) post which lists some more alternatives.

[ashwinmahantha]

I am a new user of Qutebrowser. I installed Qute on Ubuntu 18.04 using apt-get.
Version: qutebrowser v1.1.1

1. By when can I get a fix to this issue via apt?
2. :set -u https://accounts.google.com/*... Where do I enter this command? Entering it after hitting ':' in Qutebrowser gives error: Unrecognized arguments -u Mozilla/5.0 (X11; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0

[The-Compiler]

@ashwinmahantha

1. Once Ubuntu 20.04 is out in April and you do a system upgrade to that.
2. Support for per-domain settings (which this uses) was added in qutebrowser v1.2.0, so this won't work on v1.1.1 (which is from January 2020).

[facebookegypt]

I have the same problem when I try to test the console project using webbrower control from Visual Studio [Not the default windows browser]

[The-Compiler]

FWIW looks like there's a chance this will break again when Google changes their behavior on January 4th. Right now our workaround still works even with the mentioned header, but with Google, you never know what'll change.

Their "browser guidlines" are hypocritical to say the least:

The browser must not provide automation features. This includes scripts that automate keystrokes or clicks

cough ChromeDriver cough

[chaorace]

I encountered this issue today on the google accounts sign-in, despite having content.site_specific_quirks set to true. Strangely, the headers fix still worked when I tried that instead. I'm on a recent git commit, so maybe it's a regression affecting the site specific quirks feature?

qutebrowser v1.14.1
Git commit: 21b20116f on master (2021-01-19 19:14:55 +0100)
Backend: QtWebEngine (Chromium 83.0.4103.122)
Qt: 5.15.2

[The-Compiler]

@chaorace Oh no... 😟

Can you reproduce this with qutebrowser --temp-basedir accounts.google.com as well?

I have two Google accounts (@gmail.com and @the-compiler.org via Google Workspace), and currently see this:

• GMail, quirks enabled: I get a captcha, but can then log in normally
• GMail, quirks disabled: "This browser or app may not be secure" after entering username
• Workspace, quirks enabled: Captcha
• Workspace, quirks disabled: Captcha (!)

Can you elaborate what you mean with "the headers fix still worked when I tried that instead" exactly?

[chaorace]

Regarding the "headers fix", I'm referring to this snippet you included in the issue description:
:set -u https://accounts.google.com/* content.headers.user_agent 'Mozilla/5.0 (X11; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0'

I experienced the issue on a fresh system install (as in a brand new disk, not just qutebrowser). I can't seem to replicate it on my primary disk, now that I'm attempting to do that. That could be related, more likely a red herring imo.

In any case, I'll be hopping back onto my fresh install later today and will see if I can get the issue to reproduce once more from there.

[The-Compiler]

On the system where it breaks with the default setup, can you try if this is broken as well:

:set -u https://accounts.google.com/* content.headers.user_agent "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36 Edg/80.0.3987.163"

If it is, but the Firefox UA you used above helps, that'd be an easy fix. If that's the case, can you try with an updated Firefox UA too please?

:set -u https://accounts.google.com/* content.headers.user_agent "Mozilla/5.0 ({os_info}; rv:84.0) Gecko/20100101 Firefox/84.0"

(leave the {os_info} in there, qutebrowser should replace it with the correct content)

My expectation is that the Edge UA won't help (because that's what the site-specific quirk is doing), but the new Firefox one would still help.

[The-Compiler]

@chaorace Any update on this? If the workaround needs adjustments, I'd really like to do them before the next release (which might happen soon-ish).

[chaorace]

Sorry for the delayed response. I just can't seem to reproduce the issue anymore. I even tried navigating back to the exact same URL in my history and that worked just fine. (this is all whilst using temp-basedir, of course).

I can only guess it's some crazy Google heuristic or (more likely) some kind of flub on my end. All's well that ends well, I suppose?

[The-Compiler]

That's just how things are with Google... I remember back when this started (late 2019/early 2020) it was quite flaky for a while as well. Let's hope it keeps working now! If that's not the case, please let me know though.

[micycle8778]

I was having the same issue, but using the command you recommended worked.

[michaelskyba]

Might be redundant at this point to mention, but I, too, can reproduce this.

With (only) --temp-base-dir, it fails, but using :set -u https://accounts.google.com/* content.headers.user_agent 'Mozilla/5.0 (X11; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0' fixes it.

[saemideluxe]

I had the same problem just starting today. Using :set -u https://accounts.google.com/* content.headers.user_agent "Mozilla/5.0 ({os_info}; rv:84.0) Gecko/20100101 Firefox/84.0" solved the problem (even in the same session).

[The-Compiler]

I could reproduce with one of my Google accounts (though not another...). I updated the workaround to use an updated Firefox UA and released v2.3.1 with it.

[ghost]

Hey, I'm glad to see you fighting the good fight here, but this will break, and soon. It only works for Firefox 90 because it's so new. Google gathers fingerprint data from each release of (what they consider to be) a major browser and clusters the results. Once they have enough results to partition the real Firefox 90 from qutebrowser, this will break. That will happen very very soon.

Google published a paper explaining exactly how they do this (or rather, how they did this five years ago; what they do now is at least this good and almost certainly better):

https://research.google/pubs/pub45581.pdf

Unfortunately what you're doing here is only going to work for a few weeks after each release of Edge or Firefox. I came across comments in Mozilla's bugzilla where the Firefox devs noticed this weird adaptive behavior and figured it out (sorry, don't have the bug link anymore). Apparently not even they are looped in on all the details.

Sorry to bear bad news. It's pretty evil what Google is doing here, basically blocking anybody from competing with their browser (except for a token nonprofit that is totally dependent on Google funding).

[The-Compiler]

@a-m-joseph Evidence seems to suggest otherwise, given that:

• Specifying an older Firefox version works just as well
• We've been using this workaround successfully (originally with Firefox rather than Edge) since January 2020

[ghost]

I do not know if it helps only saemideluxe's solution worked on me not 90 nor other peoples

[toofar]

I wonder if we still need the workaround for this? We currently have a quirk which is active by default which sets a user agent of firefox 90. I wonder if we should do one of:

1. update that UA to a later firefox one (eg rv:109.0 ... Firefox/118)
2. remove the quirk

If anyone wants to test (2) you should be able to add "ua-google" to the content.site_specific_quirks.skip setting. And removing any other domain specific user agent override you have for https://accounts.google.com/*.
I tested logging into gmail on my work account and it worked fine, but I don't think I could reproduce the issue in the first place anyhow.