403 client error on api.github.com

[rabernat]

I am trying to set up doctr on a new repo following the instructions in the readme. I have the latest version of doctr (1.7.3).

After I enter my authentication code, I get a 403 Client Error.

$ doctr configure
Welcome to Doctr.

We need to ask you a few questions to get you on your way to automatically
deploying from Travis CI to GitHub pages.

What is your GitHub username? rabernat
Enter the GitHub password for rabernat: 
A two-factor authentication code is required: sms
Authentication code: ******
Traceback (most recent call last):
  File "/Users/rpa/miniconda3/envs/geo_scipy/bin/doctr", line 6, in <module>
    sys.exit(doctr.__main__.main())
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/doctr/__main__.py", line 517, in main
    return process_args(get_parser(config=config))
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/doctr/__main__.py", line 249, in process_args
    return args.func(args, parser)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/doctr/__main__.py", line 383, in configure
    login_kwargs = GitHub_login()
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/doctr/local.py", line 141, in GitHub_login
    return GitHub_login(username=username, password=password, OTP=OTP, headers=headers)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/doctr/local.py", line 145, in GitHub_login
    r.raise_for_status()
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/requests/models.py", line 935, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://api.github.com/

I am totally stuck on this issue! Any help would be greatly appreciated.

[rabernat]

cc @tjcrone

[asmeurer]

I've been getting this error a lot too in my testing of #310. I don't know what causes it. It seems to go away after a while. Can you try again later and let me know if it works.

[rabernat]

Thanks for the quick reply! So you're saying possibly an issue with github's API service?

I have retried several times over the past 8 minutes with no change. I will wait longer and then retry.

It only happens when I enter a valid SMS code.

[asmeurer]

Yeah, I use the authenticator app. I don't know if that's related. For me it happens after testing it for a while so maybe the API is trying to anti-spam me. But that doesn't explain why it would happen to you, unless you are also using the API for other things, or have run doctr configure a few times already before this.

[asmeurer]

Apparently rate limit messages raise 403 https://developer.github.com/v3/#rate-limiting. I'm not sure if the message in the response is being included in the exception from requests. So this could be the issue.

[kyleniemeyer]

This is also happening to me on my first doctr configure attempt (plus subsequent ones), also using the authenticator app.

[kyleniemeyer]

Update: it worked eventually. I do think it's somehow a rate limiting issue, because I tried a different API call and got such a message. However, I don't understand how I exceeded the limit, since I got the error on my very first config attempt.

[asmeurer]

The API limits are pretty low for unauthenticated calls. But I wouldn't think an authentication request itself would count as unauthenticated. I'll try to take a closer look and see if I can figure out what is going on.

[moorepants]

Here is what I did (on brand new repo):

1. doctr configure
2. entered username
3. entered password
4. when it asked for the repo, i tried to run but it couldn't find it on travis (i hadn't set it up)
5. left in this state, went to travis for like 10 minutes trying to enable that repo, finally did
6. went back to terminal and pressed enter for the repo question, got 403 error
7. subsequent runs of doctr configure got 403s at the password question

Maybe this will reproduce it?

[asmeurer]

I can reproduce getting 403 after waiting a while. This is the response {'documentation_url': 'https://developer.github.com/v3/auth#working-with-two-factor-authentication', 'message': 'Must specify two-factor authentication OTP code.'}. I think it just means the authentication timed out.

But I can't currently reproduce the 403 on every login attempt thereafter.

[rabernat]

Here is what I did (on brand new repo):

doctr configure
entered username
entered password
when it asked for the repo, i tried to run but it couldn't find it on travis (i hadn't set it up)
left in this state, went to travis for like 10 minutes trying to enable that repo, finally did
went back to terminal and pressed enter for the repo question, got 403 error
subsequent runs of doctr configure got 403s at the password question

This is almost exactly the same as what I just tried (again). The difference is that, after finally enabling on travis, I got a slightly different error after the "What repo do you want to build the docs for" question:

Traceback (most recent call last):
  File "/Users/rpa/miniconda3/envs/geo_scipy/bin/doctr", line 6, in <module>
    sys.exit(doctr.__main__.main())
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/doctr/__main__.py", line 517, in main
    return process_args(get_parser(config=config))
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/doctr/__main__.py", line 249, in process_args
    return args.func(args, parser)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/doctr/__main__.py", line 399, in configure
    is_private = check_repo_exists(build_repo, service='github', **login_kwargs)
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/doctr/local.py", line 268, in check_repo_exists
    r.raise_for_status()
  File "/Users/rpa/miniconda3/envs/geo_scipy/lib/python3.6/site-packages/requests/models.py", line 939, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://api.github.com/repos/rabernat/research_computing_2018

Then after that, on all subsequent attempts, I got the 403 error.

This is a frustrating situation. I was relying on doctr to deploy a website for a course that is starting in one week. (It worked like a charm last year!) Any suggestions for a workaround would be highly appreciated.

[rabernat]

Update: this morning it finally worked.

[asmeurer]

OK, I was able to reproduce this again, by doing exactly what you did (created a new repo, tried doctr before enabling it, then enabling it). With some debug printing of the response json and headers, I can confirm that the problem is indeed a rate limit. GitHub allows 5000 authenticated requests per hour, so I'm not sure how it is being hit. My guess is that when you press the "sync account" button on Travis this sucks up all your requests. The requests reset every hour, so the best workaround for now if you get this is to just wait and try again. I'll work on printing better messaging about this from doctr's side.

401 just means your OTP timed out. The doctr code assumes the session is short enough that it can get the OTP once at the beginning and use it throughout. I'm not too worried about it if someone leaves doctr configure unfinished for a few minutes and it fails and they have to run it again (though we could make the error messaging better).

[asmeurer]

Regarding why it happens when OTPs are used, I guess it's because we do a "fake" post when OTPs are used to force SMS codes to be sent (#203).

[asmeurer]

OK, I'm now sure that the Travis "sync account" button is the source of the trouble here. In particular, if you are a member of the conda-forge organization, then Travis has to look at over 5000 repositories (the rate limit is 5000 requests per user per hour, shared across all oauth applications).

There's not much we can do about this, aside from improving the messaging. Unfortunately, I suspect Travis also monitors the rate limit and restarts the sync. So it is possible that even after the reset time, the limit will be hit again. I don't have access to 10000 repos, so I can't say for sure (all I could find was this old issue).

Also, it looks like the request before the OTP code does count as an unauthenticated request as far as rate limits are concerned, at least according to the headers, but it doesn't actually prevent the final authentication with OTP if the remaining unauthenticated requests is 0.

I've made a PR with improved messaging at #320.

[moorepants]

Wow! Good sleuthing. Very interesting consequence to the size of conda forge.

[rabernat]

OK, I'm now sure that the Travis "sync account" button is the source of the trouble here. In particular, if you are a member of the conda-forge organization, then Travis has to look at over 5000 repositories (the rate limit is 5000 requests per user per hour, shared across all oauth applications).

😱 yes this sounds right! Every time I have tried this, it has been right after doing travis sync account. And yes, I am a conda-forge member.

cc @ocefpaf, who may be interested in this conda-forge related issue.

[asmeurer]

In my own case, this happened to me after setting up https://github.com/drdoctr/travis-ci-com-testing on Travis to test #309.

@rabernat @moorepants if you can comment on the messaging in #320 that would be great.

[kyleniemeyer]

I am also a member of conda-forge, so this all tracks for me too...