Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Contributors list is empty - error? #112

Closed
jywarren opened this issue Jan 2, 2019 · 21 comments · Fixed by #125
Closed

Contributors list is empty - error? #112

jywarren opened this issue Jan 2, 2019 · 21 comments · Fixed by #125
Labels
bug help wanted

Comments

@jywarren
Copy link
Member

jywarren commented Jan 2, 2019

Hmm, at https://code.publiclab.org#r=all, i'm seeing nobody in the contributors list; same in incognito mode. I haven't loaded it for days, so I don't think it's queue exhaustion. Anyone have ideas? @Rishabh570 maybe?

Thanks!

screenshot 2019-01-02 at 5 35 57 pm
@Rishabh570
Copy link
Collaborator

Hmm, I've also tried it on code.publiclab.org and it turns out that it's hitting the quota limit and hence not being able to show every piece of info that is required...
For some quick maths,

  • Recent contributors part eats up 20 requests (10 for monthly and 10 for weekly) and its optimization is WIP, we'll able to reduce it to 10 requests in total...
  • All contributors list (as shown on code.publiclab.org#r=all) eats up 20 requests
  • Repo contributors list (which currently shows only for a single repo) eats up only 1 request
  • Finally, fetching the list of top repos takes 1 request.

So, according to that, we're well under 60, I wonder how it's hitting the limit...
@jywarren I doubt it could be due to the recent PR merged which included changes in dist/ file...I'll revert to previous commits and see if this problem persists...

@Rishabh570
Copy link
Collaborator

@jywarren I tested after going back in commits (recent ones) and it's working properly at that point...I believe 2 of the recent commits (which include changes in dist/) are the cause, let me just be sure enough then I'll update you!

@jywarren
Copy link
Member Author

jywarren commented Jan 3, 2019 via email

@subhahu123
Copy link
Contributor

screenshot from 2019-01-06 23-41-46

github api server is showing 403 error
@jywarren @Rishabh570

@subhahu123
Copy link
Contributor

it's showing that

{
"message": "API rate limit exceeded for 139.167.207.65. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)",
"documentation_url": "https://developer.github.com/v3/#rate-limiting"
}
screenshot from 2019-01-06 23-44-46

@subhahu123
Copy link
Contributor

subhahu123 commented Jan 6, 2019 •
edited
Loading

i don't think that this is because of last commits
it is about api rate limit exceding

@Rishabh570
Copy link
Collaborator

Hi @jywarren, I just tested code.publiclab.org by first flushing all the localStorage and it worked just as expected...here is the GIF,

ezgif com-optimize 2

I wonder what's the cause...what do you think might it be?

@jywarren
Copy link
Member Author

jywarren commented Jan 7, 2019

Hmm. Here's what i see in logs:

https://gist.github.com/jywarren/d1e74c754a52cc51db709f7c4d0a13f7

It still looks like the 403 exhausted queue. How long should it take for my queue to refill?

/#r=all:1 Uncaught (in promise) StatusCodeError {name: "StatusCodeError", statusCode: 403, message: "403 - [object Object]", error: {…}, options: {…}, …}

@jywarren
Copy link
Member Author

jywarren commented Jan 7, 2019

Anyways glad most people aren't seeing this!

@Rishabh570
Copy link
Collaborator

After every 60min queue gets refilled...
Can you please try once again when your quota get refilled (takes 1hr to refill), just remove everything from your localStorage and then visit the site (make sure to visit the URL after removing the localStorage)...if it still throws 403, something wrong is in the code then...

@jywarren
Copy link
Member Author

jywarren commented Jan 7, 2019 via email

@jywarren
Copy link
Member Author

jywarren commented Jan 7, 2019

hmm.... even on a separate computer which hasn't been used for several days:

image

After clearing local storage:

image

@Rishabh570
Copy link
Collaborator

@jywarren Hmm, I see...I've also tested it on another computer and it's still showing these 403s...I saw how all this is all happening (plots2 sends 4 requests and fetching all the contributors takes ~40 requests not 20 😕, according to network tab)...
Actually, we're on the track of making the least possible requests, my MVC work removes the redundant separate calls to Github for all contributors and a particular repo contributors...recent contributors part is remaining to be optimized.

I guess to resolve this problem I should make my changes live somewhere so that people can test it before actually merging it because it includes changes in almost every part of the project...what do you think, should I make my MVC work live somewhere first (in order to check it) once complete it?

@Rishabh570
Copy link
Collaborator

@jywarren One more thing, can we just remove these two commits from main, this and this as these contain changes in dist/?

As an aside, gh-pages would be suitable for testing my changes I guess..?

@Rishabh570
Copy link
Collaborator

@jywarren I've found why we're hitting the limits of API almost everytime 😄
Actually, we're making 2 requests per repository...first to collect total number of pages that the request gives us back and then iterating on those pages to collect the data of contributors...so we're making ~40 requests for 20 repos and 20 is consumed by the data of recent contributors and some extra requests for fetching repos and issues. Hence, we're going above 60.

I'll be refactoring the code now...Mainly the focus will be,

  • Reducing the requests per repo so that we make ~20 requests for 20 repos (we might sacrifice preciseness of contributors count, the biggest difference will occur in plots2 contributors' data because its response contains 4 pages of data & we'll be getting data from only 1 page if we want to reduce the requests count...it would be a tradeoff)...[suggestion needed]

  • Reducing the number of requests we make for showing recent contributors. Currently, code makes 10 requests for fetching contributors last week and 10 for contributors last month...I'll try to do all this with 10 requests for contributors last month and filtering out contributors last week out of them.

And, after above things get done...there'll be 1 remaining thing to which is, showing recent contribs and normal contribs according to repository filter provided in the URL...but this will be covered in my "MVC" PR...

@jywarren
Copy link
Member Author

jywarren commented Jan 10, 2019 via email

@Rishabh570
Copy link
Collaborator

Actually, I'm also thinking this way :)
I guess only plots2 contains 4 pages of data (because it has approx 334 contribs) and other top 19 repos need only 1 page (because they have less than 100 contribs)...

I'll take a look at the contribs count once again for each repo to be sure...I guess we'll able to allow few repos to make more than 1 requests...

@Rishabh570
Copy link
Collaborator

Hi @jywarren, can you please take a look at https://rishabh570.github.io/community-toolbox/
I've done some modifications and pushed them to my gh-pages branch and they're live at above URL...can you please visit this and check if everything is working fine or not (hopefully 403 shouldn't be there now)...

@jywarren
Copy link
Member Author

jywarren commented Jan 11, 2019 via email

@Rishabh570
Copy link
Collaborator

@jywarren Yes, quota should be full...

@Rishabh570
Copy link
Collaborator

Rishabh570 commented Jan 12, 2019 •
edited
Loading

@jywarren Is it working fine? I would be opening a PR for it if its working okay... 🤞
(I've not faced any problems yet...)

@Rishabh570 Rishabh570 mentioned this issue Jan 14, 2019
Merged
5 tasks
@jywarren jywarren mentioned this issue Oct 19, 2019
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug help wanted
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix site issues [403 errors] Rishabh570/community-toolbox
3 participants
@jywarren @Rishabh570 @subhahu123