What is the current rate limiting rates for Legitify

[raoganeshr]

Running the GitHub Action more than once ends up triggering the rate limiting error. Can you detail out what the limits are and when are they reset?

[noamd-legit]

@raoganeshr
legitify is bound to the specific platform rate limit boundaries, for GitHub it is 5000 requests per hour (github rate limits).
If your organization is not very big, running legitify twice shouldn't trigger rate limit so it might be the case that your token is used elsewhere.

We added better rate limit handling in v0.2.2, could you check it out?

[raoganeshr]

I tried with 0.2.2. This time I got 67 repos. So is there a way to reduce the number of calls made by using bulk calls or adding gaps been calls? Seems the tool is too fast to get rate limited (secondary rate limited in my case).

2023/01/25 13:53:02 2023/01/25 13:53:02 error getting repository actions settings for repo: GET https://api.github.com/repos/ORG/REPO/actions/permissions/workflow: 403 You have exceeded a secondary rate limit. Please wait a few minutes before you try again.

[noamd-legit]

Thanks for testing it again. We are looking into it. Could you please share how many repositories your organization contains?

[raoganeshr]

Around 250

[gal-legit]

hey @raoganeshr,
I tried to patch legitify to avoid any failure of secondary rate limit.
Can you please checkout gofri/secondary_rate_limit_round_2, run make build and see if that solves the problem?
Alternatively, I prepared a couple of archives for you to run it quickly (I simply built it locally for mac and linux):
linux.tar.gz
mac.tar.gz
sha256sum for these two:

038a25d27392599cb652491e2e18808556d9fb51336b39f5885df2e32a7839fd  linux.tar.gz
c72da5410deb2c2a82c0ebc25579a481561bbb9ebfebb47424815726fa07c171  mac.tar.gz

[raoganeshr]

Hi Gal,

This patch did help. However I believe I am still hitting the rate limit. Are we waiting for a minute just for the first time? As per https://docs.github.com/en/rest/overview/resources-in-the-rest-api?apiVersion=2022-11-28#secondary-rate-limits we need to

If the retry-after response header is present, you should not retry your request until after that many seconds has elapsed. Otherwise, you should not retry your request until the time, in UTC epoch seconds, specified by the x-ratelimit-reset header.

So instead of waiting for 1 minute or one second, our wait should be time mentioned in the header + 1 sec. You may want a overall timeout just incase this wait and execute takes too much time or you can recommend the end user to have a action timeout.

Logs from the last run using the debug build:

Gathering collection metadata...
Progress bar is disabled because stderr is not a terminal. Starting collection...
Secondary rate limit reached! Sleeping for 59.9999992 seconds
Secondary rate limit reached! Sleeping for 59.997637013 seconds
Secondary rate limit reached! Sleeping for 59.994215231 seconds
Secondary rate limit reached! Sleeping for 0.005144367 seconds
Secondary rate limit reached! Sleeping for 0.005122767 seconds
Secondary rate limit reached! Sleeping for 0.005118767 seconds
Secondary rate limit reached! Sleeping for 0.005116067 seconds
Secondary rate limit reached! Sleeping for 0.005112667 seconds
Secondary rate limit reached! Sleeping for 0.005109767 seconds
Secondary rate limit reached! Sleeping for 0.005107267 seconds
Secondary rate limit reached! Sleeping for 0.005104667 seconds
Secondary rate limit reached! Sleeping for 0.005101367 seconds
Secondary rate limit reached! Sleeping for 0.005098867 seconds
Secondary rate limit reached! Sleeping for 0.005096067 seconds
Secondary rate limit reached! Sleeping for 0.005093067 seconds
Secondary rate limit reached! Sleeping for 0.005090367 seconds
Secondary rate limit reached! Sleeping for 0.005088167 seconds
Secondary rate limit reached! Sleeping for 0.005085667 seconds
Secondary rate limit reached! Sleeping for 0.005083367 seconds
Secondary rate limit reached! Sleeping for 0.005080367 seconds
Secondary rate limit reached! Sleeping for 0.005077567 seconds
Secondary rate limit reached! Sleeping for 0.005075367 seconds
Secondary rate limit reached! Sleeping for 0.005072567 seconds
Secondary rate limit reached! Sleeping for 0.005069767 seconds
Secondary rate limit reached! Sleeping for 0.005067367 seconds
Secondary rate limit reached! Sleeping for 0.005065067 seconds
Secondary rate limit reached! Sleeping for 0.005062667 seconds
Secondary rate limit reached! Sleeping for 0.005060467 seconds
Secondary rate limit reached! Sleeping for 0.005058167 seconds
Secondary rate limit reached! Sleeping for 0.005055967 seconds
Secondary rate limit reached! Sleeping for 0.005053467 seconds
Secondary rate limit reached! Sleeping for 0.005051167 seconds
Secondary rate limit reached! Sleeping for 0.005048767 seconds
Secondary rate limit reached! Sleeping for 0.005046467 seconds
Secondary rate limit reached! Sleeping for 0.005044267 seconds
Secondary rate limit reached! Sleeping for 0.005040068 seconds
Secondary rate limit reached! Sleeping for 0.005042768 seconds
Secondary rate limit reached! Sleeping for 0.005033668 seconds
Secondary rate limit reached! Sleeping for 0.005037668 seconds
Secondary rate limit reached! Sleeping for 0.005030968 seconds
Secondary rate limit reached! Sleeping for 0.005028268 seconds
Secondary rate limit reached! Sleeping for 0.005036068 seconds
Secondary rate limit reached! Sleeping for 0.005025068 seconds
Secondary rate limit reached! Sleeping for 0.005034668 seconds
Secondary rate limit reached! Sleeping for 0.005022068 seconds
Secondary rate limit reached! Sleeping for 0.005019768 seconds
Secondary rate limit reached! Sleeping for 0.005019068 seconds
Secondary rate limit reached! Sleeping for 0.00035629 seconds
Secondary rate limit reached! Sleeping for 0.001253784 seconds
Secondary rate limit reached! Sleeping for 0.002353076 seconds
Secondary rate limit reached! Sleeping for 0.005888055 seconds
Secondary rate limit reached! Sleeping for 0.005111855 seconds
Secondary rate limit reached! Sleeping for 1.279e-06 seconds
Secondary rate limit reached! Sleeping for 0.000416277 seconds
Secondary rate limit reached! Sleeping for 0.00532045 seconds
Secondary rate limit reached! Sleeping for 0.00440405 seconds
Secondary rate limit reached! Sleeping for 0.002923255 seconds
Secondary rate limit reached! Sleeping for 0.002510755 seconds
Secondary rate limit reached! Sleeping for 0.005847334 seconds
Secondary rate limit reached! Sleeping for 0.001553453 seconds
Secondary rate limit reached! Sleeping for 0.000981655 seconds
Secondary rate limit reached! Sleeping for 0.003516441 seconds
Secondary rate limit reached! Sleeping for 0.001458849 seconds
Secondary rate limit reached! Sleeping for 0.001634846 seconds
Secondary rate limit reached! Sleeping for 8.0854e-05 seconds
Secondary rate limit reached! Sleeping for 0.001341946 seconds
Secondary rate limit reached! Sleeping for 0.502072258 seconds
Secondary rate limit reached! Sleeping for 0.38885709 seconds
Secondary rate limit reached! Sleeping for 0.347352384 seconds
Secondary rate limit reached! Sleeping for 0.318712619 seconds
Secondary rate limit reached! Sleeping for 0.317702323 seconds
Secondary rate limit reached! Sleeping for 0.300522304 seconds
Secondary rate limit reached! Sleeping for 0.296113225 seconds
Secondary rate limit reached! Sleeping for 0.018108393 seconds
Finished collecting repository
Some errors raised during the execution. Check errors/error.log for more details

[gal-legit]

hey @raoganeshr,
First, thanks for helping us to test this feature.
I just implemented a secondary rate limit injection mechanism to help us test, so things should be smoother going forward.

This patch did help.

That's great!
I patched it to improve the behavior, so you shouldn't see all these extra prints etc.

However I believe I am still hitting the rate limit. Are we waiting for a minute just for the first time? As per https://docs.github.com/en/rest/overview/resources-in-the-rest-api?apiVersion=2022-11-28#secondary-rate-limits we need to
...
So instead of waiting for 1 minute or one second, our wait should be time mentioned in the header + 1 sec. You may want a overall timeout just incase this wait and execute takes too much time or you can recommend the end user to have a action timeout.
Thanks for the comment.

We actually use the retry-after header to only wait for the secondary rate limit (because waiting an hour for the primary rate limit doesn't make sense for us).
If multiple secondary rate limits are reached during the same run, legitify waits for each of them.
If you think that it'd be valuable to allow users to limit the maximum total sleeping time, please open an issue for enhancement (or a pull request once this one is merged).

---

I pushed an improved version to the branch, please feel free to check it out.
Note that I moved these prints to the log instead of the screen (because of the progressbar issues)

[raoganeshr]

I think we can manage the overall execution time using workflow timeouts, so it is not a deal breaker. I will try the changes once you merge it to the main branch. Thanks for your quick response.

[gal-legit]

hey again @raoganeshr,
I pushed some more improvements, I think it should be ready for merge now.

linux.tar.gz
mac.tar.gz

b3b45b3d42dfc4e2c96af6dcc67d74e916719fccef69ce06f30fbb820a8e7d15  linux.tar.gz
cabf91f140dc4fac4b3cfbade87d106ce2218550a5cd292b7de1b7dfdcc3e033  mac.tar.gz