New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

trouble with github.com #518

Closed
kencu opened this Issue Aug 22, 2018 · 34 comments

Comments

Projects
None yet
5 participants
@kencu
Copy link

kencu commented Aug 22, 2018

Sadly, github seems to be giving me trouble with TFF-FPR8 (as well as Firefox versions 46 and 52) as of this morning.

The "recent activity" ball spins endlessly, recent commits can't be shown, and the commit buttons don't seem to function correctly.

The banner bar shows a new "incompatible with Firefox version" bar.

This one is painful :>

@kencu

This comment has been minimized.

Copy link
Author

kencu commented Aug 23, 2018

Goodish news -- changing the user age to 60ESR(Intel) works to fix it!

@classilla

This comment has been minimized.

Copy link
Owner

classilla commented Aug 23, 2018

It also works if set to 52ESR, which is probably more appropriate.

There are two possible solutions:

  • Use the same workaround for Imgur for Github (no user agent, unless the user explicitly specifies one). I don't know if this will work.
  • Change the base UA to 52. I'm a little resistant to doing this because we don't support everything in 52, though we support a fair bit. For example, no CSS grid.
@kencu

This comment has been minimized.

Copy link
Author

kencu commented Aug 23, 2018

I should also say that with this github change, TFF on Intel is now my only fully functional browser on 10.6.8. So good for us all for making it work!

@kencu

This comment has been minimized.

Copy link
Author

kencu commented Aug 23, 2018

@classilla

This comment has been minimized.

Copy link
Owner

classilla commented Aug 24, 2018

Currently typing this in a build that suppresses user agent on any github.com page using the workaround from #422. So far seems to work fine, so I'll commit this for next beta.

classilla added a commit that referenced this issue Aug 24, 2018

@rmottola

This comment has been minimized.

Copy link
Contributor

rmottola commented Aug 25, 2018

@kencu you summon me :)

GitHub is a "pain" and now that it is Microsoft I suppose it will be even harder on non-mainstream browsers.

I'm a bit back with TFF - if you remember I still have issues on 10.5, hoped some input for you. I wanted also work with you in GCC.

As you write, it might become one of the main browsers! We are near it

I did try GitHub on FF 49 "official" and apart from getting the warning about unsupported site, it works though.. so maybe there is a specific issue of TFF?

(I will make an update to our intel repository in the meanwhile).

@kencu

This comment has been minimized.

Copy link
Author

kencu commented Aug 25, 2018

did try GitHub on FF 49 "official" and apart from getting the warning about unsupported site, it works though.. so maybe there is a specific issue of TFF?

The way it looked to me, GitHub changed something to make older FF versions not work, and then changed it back again a day later -- FF 45.9.0ESR works again I notice. I think perhaps they were testing it?

Anyway, when this happens, you can pretty much assume the change is coming.

@classilla

This comment has been minimized.

Copy link
Owner

classilla commented Aug 25, 2018

I didn't really notice, though I do most of my work from the command line. But if you see those problems again, please report anything interesting the Browser Console. Maybe it's just some DOM support that needs being added.

Meanwhile, no problems with FPR9b3 typing this.

@classilla

This comment has been minimized.

Copy link
Owner

classilla commented Aug 27, 2018

FPR9b3 is out and seems to work fine. Was able to update the wiki and handle issues without difficulty, so marking closed. If you see errors in the Browser Console, open new tickets for them; hopefully it's just a matter of adding needed DOM support.

@classilla classilla closed this Aug 27, 2018

@ccorn

This comment has been minimized.

Copy link

ccorn commented Sep 5, 2018

Reporting this here, because I suspect it is related:

Upgrading to FPR9 has broken Github Login for me until I specified the UA (as 52ESR Intel).

The phenomenon is that (with TenFourFox default UA, presumably this means no UA at all for github.com) Github delivers a "500" error page after the login attempt, without actually logging me in.

Since that is a server error, TFF should not need to care. However, it seems that Github tries to do something fragile with the UA information after login, and since this thread is about tuning the UA specially for Github, I suppose that you might be interested.

FPR8 lets me login, but has the issues with some Github buttons and Github menus mentioned in this thread unless the UA is changed.

If it helps, I use private browsing mode, uBlock Origin and NoScript with github.com whitelisted.
3rd-party cookies are rejected, other cookies are deleted when closing TFF. The same configuration works with FPR8.

Interestingly, FPR9 lets me login to Github with a changed UA, and Github does not break if I change the UA back to the default (or to, say, Classilla) after having logged in.

@ccorn

This comment has been minimized.

Copy link

ccorn commented Sep 5, 2018

More precisely, Login to Github

  • yields "error 500" with a freshly restarted TFF (everything cleared) and default UA,
  • works even with UA = Classilla
  • still works after signout (without quitting TFF), even with default UA.

I hope this enables you to reproduce the problem.

@classilla

This comment has been minimized.

Copy link
Owner

classilla commented Sep 5, 2018

@kencu or @rmottola , do you get anything like that? It worked fine on FPR9b3 and I didn't change that code for final. I am on a business trip, so I can't test.

However, if it really does cause a 500 error on Github's side to suppress the user-agent header, that's definitely a Github bug. Hopefully someone reads their logs. I would prefer to send no user agent at all than a dummy one because then we have to keep updating the dummy.

@ccorn

This comment has been minimized.

Copy link

ccorn commented Sep 6, 2018

I agree that Github's susceptibility to User-agent header contents, both for login and UI button/menu functionality, ought to be fixed entirely on Github's side. Ideally, TFF should have no special Github codepaths at all, at least none that need to speculate about erroneous Github behavior.
If this bug gets confirmed, I have no satisfying proposal to offer. The best way forward seems to nudge Github about their error 500, and in the wake of that, remind them that user-agent headers are generally not helpful for feature detection purposes and perhaps cause more bug tickets than ignoring the user-agent header would.

@kencu

This comment has been minimized.

Copy link
Author

kencu commented Sep 6, 2018

Github turned back on whatever it was they turned on and turned off again when I first reported this. My "official" Firefox 46 no longer works on GitHub again.

TenFourFox, however, is working great! I have it set to "use the default user agent string", and I'm running (on 10.6.8 Intel) a build that matches the commit dated 20180824, which is this one c4c83f3281f6ed2e1c259f591f63c82f218471597.

@ccorn

This comment has been minimized.

Copy link

ccorn commented Sep 6, 2018

On my G5, TenFourFoxG5-FPR9b3.app behaves the same as TenFourFoxG5-FPR9.app, getting me Github's fancy "Error 500" response page when trying to log in with default UA.

Since getting a clean browser state seems to be difficult, here are steps to reproduce with curl:

read -p "Github username: " USERNAME
read -s -p "Github password: " PASSWORD
echo
curl -L -s -c github-cookies-pre-login.txt https://github.com/login >login.html
AUTHENTICITY_TOKEN_DEF=$(sed -n -E \
  's,^.* name="(authenticity_token)" +value="([^"]*)".*$,\1=\2,p' login.html)
curl -L -s -b github-cookies-pre-login.txt \
  -c github-cookies-post-login.txt \
  --data-urlencode "utf8=✓" \
  --data-urlencode "$AUTHENTICITY_TOKEN_DEF" \
  --data-urlencode "login=$USERNAME" \
  --data-urlencode "password=$PASSWORD" \
  --data-urlencode "commit=commit" \
  -e "https://github.com/login;auto" \
  -H User-Agent: \
  https://github.com/session >post-login.html
open post-login.html

(Edited to automate the token extraction, properly urlencode parameters, add referrer, and to avoid cleartext credentials in the .bash_history.)

Updated Notes:

  • If you omit the -H User-Agent: (that is, you let curl provide its User-Agent header), the login should succed: post-login.html shows you as logged in.

  • With the -H User-Agent: (that is, without User-Agent header) or with -A '' (that is, empty User-Agent) you get instead a HTML page titled Server Error with a large pic showing the 500 code, followed by the following text:

    Looks like something went wrong!

  • Sometimes, Github titles its response Oh no and texts

    What?! Your browser did something unexpected.

    That may be due to an anti-bot measure.

@rmottola

This comment has been minimized.

Copy link
Contributor

rmottola commented Sep 8, 2018

Sorry for the absence - I was out of country and away from my Macs.
I an updating Macports and will try a current TenFourFox/Intel build as soon as possible on 10.6 (on 10.5 my builds are unusable).
A of today, I tried logging in and accessing basic functionality with latest official FF 45.9 on github and although I get an "outdated" warning, login works most seems working, although not everything (e.g. "Fetching contributors" looks to take forever.. and I get "Cannot retrieve the latest commit at this time")

Using TenFourFox/PPC FPR8 "release" I can login, but there also "fetching contributors" takes forever and get the same "latest commit" issue. If i fake the UserAgent to FF 52ESR Intel, it works! this means that GitHub does some dirty tricks which, at least for these things are not needed.

@rmottola

This comment has been minimized.

Copy link
Contributor

rmottola commented Sep 8, 2018

FPR9/PPC seems to work out of the box for me, I just upgraded on PPC. Does it send the fake UA? How can I know?

@ccorn

This comment has been minimized.

Copy link

ccorn commented Sep 8, 2018

@rmottola: Firefox recognizes some environment variables for debugging and logging purposes. In the terminal:

export NSPR_LOG_MODULES=nsHttp:3,nsHostResolver:3
export NSPR_LOG_FILE=$HOME/Desktop/TFF.log
open -a /Applications/TenFourFoxG5.app

As you can see, the log details are configurable. To reduce log file size, I recommend that you plan your actions in the to-be-logged session, use a blank start page, and quit immediately when done.

E.g. my login attempt after filling out the login page logged like this:

-1597736928[2016a0]: http request [
-1597736928[2016a0]:   POST /session HTTP/1.1
-1597736928[2016a0]:   Host: github.com
-1597736928[2016a0]:   Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
-1597736928[2016a0]:   Accept-Language: en-US,en;q=0.5
-1597736928[2016a0]:   Accept-Encoding: gzip, deflate, br
-1597736928[2016a0]:   DNT: 1
-1597736928[2016a0]:   Referer: https://github.com/login
-1597736928[2016a0]:   Cookie: has_recent_activity=1; logged_in=no; _gh_sess=[redacted]; _octo=[redacted]
-1597736928[2016a0]:   Connection: keep-alive
-1597736928[2016a0]: ]
-266842112[2338c0]: http response [
-266842112[2338c0]:   HTTP/1.1 500 Internal Server Error
-266842112[2338c0]:   Server: GitHub.com
-266842112[2338c0]:   Date: Sat, 08 Sep 2018 14:39:51 GMT
-266842112[2338c0]:   Content-Type: text/html; charset=utf-8
-266842112[2338c0]:   Transfer-Encoding: chunked
-266842112[2338c0]:   Status: 500 Internal Server Error

and more stuff.
Note that the request has no User-Agent. (Try with other sites to see the User-Agent header appearing.)

@classilla

This comment has been minimized.

Copy link
Owner

classilla commented Sep 8, 2018

@ccorn

This comment has been minimized.

Copy link

ccorn commented Sep 8, 2018

Here is a way that should allow you to reproduce the issue independently from your browser settings:

  1. Quit TenFourFox.
  2. Move/rename the folder ~/Library/Application Support/Firefox
  3. Start TenFourFox and enter https://github.com/login in the URL bar.
  4. Try to login. This should get you the Server Error page with the 500 depiction.
  5. Quit TenFourFox.
  6. Trash the folder ~/Library/Application Support/Firefox
  7. Restore ~/Library/Application Support/Firefox by moving/renaming back the folder created in step 2.
@classilla

This comment has been minimized.

Copy link
Owner

classilla commented Sep 9, 2018

I can't reproduce this, even with a totally clean profile. I'm able to log right in and type this reply.

@ccorn

This comment has been minimized.

Copy link

ccorn commented Sep 10, 2018

That's weird. I still get the 500 internal server error as described.
Anyway, if most users are not affected, this means that there is no need for action here, except for Github and me. I can live with a non-default UA for the time being.

@classilla

This comment has been minimized.

Copy link
Owner

classilla commented Sep 10, 2018

I'll reopen to see if we get any other reports.

@classilla classilla reopened this Sep 10, 2018

@ccorn

This comment has been minimized.

Copy link

ccorn commented Sep 10, 2018

Looking at my NSPR log (above) again, I find something puzzling: A POST request should have a body containing the form data; therefore the headers should include Content-Type (presumably something like application/x-www-form-urlencoded) and Content-Length. None of those have been logged, but this random example shows that nsHttp:3 logs them if they exist. So... is my request body empty? (Cannot be the case: Bad credentials get rejected without server error.)
Would you like to run a NSPR-logged session yourself for comparison? (If you do, remember to redact the session cookie contents before posting.)
Update: Even successful logins (with User-Agent) do not have Content-Type among their request headers. I suppose that's OK then.

@NapalmSauce

This comment has been minimized.

Copy link
Contributor

NapalmSauce commented Sep 10, 2018

@ccorn you're not the only one experiencing this. My NSPR log (POST /session request + response) is exactly the same. I don't have anything outstanding with my profile or build and trying with a fresh profile yields the same result.

If only a small portion of users experience this and others don't, could it be related to geographical areas(though the problem definitely seems server-related)?

@NapalmSauce

This comment has been minimized.

Copy link
Contributor

NapalmSauce commented Sep 10, 2018

I'm using a fpr9 final build ATM.

@ccorn

This comment has been minimized.

Copy link

ccorn commented Sep 11, 2018

@NapalmSauce: I am both glad and sorry to find someone else experiencing the same Github login issue with FPR9!

If only a small portion of users experience this and others don't, could it be related to geographical areas(though the problem definitely seems server-related)?

That is a possibility. Github should be able to diagnose the cause. I reported the issue to Github on Sep 06 (augmented with the URL of this thread) and received a response one day later that they have escalated the report for further investigation. It would be best if they fix that server error soon.

@classilla

This comment has been minimized.

Copy link
Owner

classilla commented Sep 11, 2018

It could be one server not updated with others, or something. At least they've taken the issue under advisement. @ccorn , is there a public issue just to see what they do?

@ccorn

This comment has been minimized.

Copy link

ccorn commented Sep 11, 2018

is there a public issue just to see what they do?

I suppose that there is none (yet), because the reply did not give a URL to a public issue thread, as any Github user would probably do if such a thing existed.
(I used the Contact support web form instead of trying the community forums. And it seems that that is the recommended way to proceed.)

@ccorn

This comment has been minimized.

Copy link

ccorn commented Sep 13, 2018

Github support has asked for details, mentioning that they do not require the User-Agent header anywhere when logging in.
I have then described the issue as best as I could, condensing the reproducibility info gathered here so far, providing a curl-based script (running which still shows the issue) and the NSPR log excerpt, and linking to this thread again.

@ccorn

This comment has been minimized.

Copy link

ccorn commented Sep 14, 2018

I just received a mail that Github staff have deployed a fix that should have resolved the issue. And indeed I can now login with the default TenFourFox User-Agent setting.
@NapalmSauce: Does that work for you as well?

@NapalmSauce

This comment has been minimized.

Copy link
Contributor

NapalmSauce commented Sep 14, 2018

@ccorn Yes, works for me as well.

@classilla

This comment has been minimized.

Copy link
Owner

classilla commented Sep 14, 2018

@classilla

This comment has been minimized.

Copy link
Owner

classilla commented Sep 15, 2018

Closing as repaired server-side.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment