Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HttpError: You have exceeded a secondary rate limit. #2204

Open
floydspace opened this issue Oct 25, 2021 · 42 comments
Open

HttpError: You have exceeded a secondary rate limit. #2204

floydspace opened this issue Oct 25, 2021 · 42 comments

Comments

@floydspace
Copy link

Current behavior

I got the error when semantic-release interacts with GitHub to write success messages

[6:24:06 AM] [semantic-release] › ✖  Failed step "success" of plugin "@semantic-release/github"
[6:24:06 AM] [semantic-release] › ✖  An error occurred while running semantic-release: RequestError [HttpError]: You have exceeded a secondary rate limit. Please wait a few minutes before you try again.
    at /home/runner/work/serverless-esbuild/serverless-esbuild/node_modules/@octokit/request/dist-node/index.js:86:21
    at processTicksAndRejections (internal/process/task_queues.js:97:5) {
  status: 403,
  response: {
    url: 'https://api.github.com/search/issues?q=repo%3Afloydspace%2Fserverless-esbuild+type%3Apr+is%3Amerged+8e105a6ff5174d23adb1eb0e42606c63e1224d24',
    status: 403,
    headers: {
      'access-control-allow-origin': '*',
      'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, Deprecation, Sunset',
      connection: 'close',
      'content-encoding': 'gzip',
      'content-security-policy': "default-src 'none'",
      'content-type': 'application/json; charset=utf-8',
      date: 'Mon, 25 Oct 2021 06:24:06 GMT',
      'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
      server: 'GitHub.com',
      'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
      'transfer-encoding': 'chunked',
      vary: 'Accept-Encoding, Accept, X-Requested-With',
      'x-content-type-options': 'nosniff',
      'x-frame-options': 'deny',
      'x-github-media-type': 'github.v3; format=json',
      'x-github-request-id': '0402:301C:BA78B:7F32F3:61764D85',
      'x-ratelimit-limit': '30',
      'x-ratelimit-remaining': '29',
      'x-ratelimit-reset': '1635143106',
      'x-ratelimit-resource': 'search',
      'x-ratelimit-used': '1',
      'x-xss-protection': '0'
    },
    data: {
      documentation_url: 'https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits',
      message: 'You have exceeded a secondary rate limit. Please wait a few minutes before you try again.'
    }
  },
  request: {
    method: 'GET',
    url: 'https://api.github.com/search/issues?q=repo%3Afloydspace%2Fserverless-esbuild+type%3Apr+is%3Amerged+8e105a6ff5174d23adb1eb0e42606c63e1224d24',
    headers: {
      accept: 'application/vnd.github.v3+json',
      'user-agent': 'octokit-rest.js/18.12.0 octokit-core.js/3.5.1 Node.js/12.22.7 (linux; x64)',
      authorization: 'token [REDACTED]'
    },
    request: { agent: undefined, hook: [Function: bound bound register] }
  },
  pluginName: '@semantic-release/github'
}
AggregateError: 
    HttpError: You have exceeded a secondary rate limit. Please wait a few minutes before you try again.
        at /home/runner/work/serverless-esbuild/serverless-esbuild/node_modules/@octokit/request/dist-node/index.js:86:21
    at /home/runner/work/serverless-esbuild/serverless-esbuild/node_modules/semantic-release/lib/plugins/pipeline.js:54:11
    at async Object.pluginsConf.<computed> [as success] (/home/runner/work/serverless-esbuild/serverless-esbuild/node_modules/semantic-release/lib/plugins/index.js:80:11)
    at async run (/home/runner/work/serverless-esbuild/serverless-esbuild/node_modules/semantic-release/index.js:201:3)
    at async module.exports (/home/runner/work/serverless-esbuild/serverless-esbuild/node_modules/semantic-release/index.js:260:22)
    at async module.exports (/home/runner/work/serverless-esbuild/serverless-esbuild/node_modules/semantic-release/cli.js:55:5)

Expected behavior

The pipeline completes without errors, though I have nothing to suggest how to fix it.

Environment

@prantlf
Copy link

prantlf commented Jan 10, 2022

This is a duplicate of #843. I suspect that that issue was not fixed properly. I got this error today too just for a small push.

I am not sure if semantic-release waits 1 s between two calls, or just 720 ms.

You can wait for a couple of minutes and then retry the job. I did it and it helped.

@gr2m
Copy link
Member

gr2m commented Jan 10, 2022

I think the throttling is currently broken because GitHub's response changed its wording a few months ago.

This will be resolved once we resolve semantic-release/github#299. Pull request welcome!

@elizabethsjudd
Copy link

Curious if this is on the roadmap. This is impacting our releases since we use semantic release.

@AntonyF-Andreani
Copy link

Same case here!

achingbrain added a commit to achingbrain/github that referenced this issue Apr 8, 2022
Refactors the client to use [@octokit/plugin-retry](https://www.npmjs.com/package/@octokit/plugin-retry)
and [@octokit/plugin-throttling](https://www.npmjs.com/package/@octokit/plugin-throttling)
as GitHub occasionally changes it's API and these plugins can abstract
those changes away from this module.

- Removes `lib/definitions/rate-limit.js`
- Adds `lib/definitions/retry.js` and `lib/definitions/throttle.js` to handle retry/throttle settings
- Updates tests to be more like GitHub (returing the correct rate limit response headers, etc)

Fixes semantic-release#299 and semantic-release/semantic-release#2204
See also semantic-release#480 and semantic-release#378
achingbrain added a commit to ipfs/aegir that referenced this issue Apr 8, 2022
@semantic-release/github doesn't respect GitHub's rate limiting - it
did once but I think GH changed their response type and now it doesn't
which means publishing large modules like `@libp2p/interfaces` is a dice
roll as to whether it will succeed or not.

Here we switch to a temporary fork that uses the `@octokit` plugins
for throttling and retrying so it should be more reliable.

Refs: semantic-release/semantic-release#2204
achingbrain added a commit to ipfs/aegir that referenced this issue Apr 8, 2022
@semantic-release/github doesn't respect GitHub's rate limiting - it
did once but it looks like GH changed their response type and now it doesn't
which means publishing large monorepos like `@libp2p/interfaces` is a dice
roll as to whether it will succeed or not.

Here we switch to a temporary fork that uses the `@octokit` plugins
for throttling and retrying so it should be more reliable.

Refs: semantic-release/semantic-release#2204
@jakeowenp
Copy link

We've had this issue twice within our team this morning in different repos

Both times it received the error You have exceeded a secondary rate limit. Please wait a few minutes before you try again when trying to hit the url https://api.github.com/search/issues

@Benjadahl
Copy link

We're seeing this issue quite often as well. Is there any known work arounds?

@xinatcg
Copy link

xinatcg commented Aug 11, 2022

i think if the error cannot prevent, we should have an option to help retry. otherwise, the CI will be totally broken by this error and cannot restart, because the version already created, then the trigger action is missed.

and for me, the error happen tyr to search issue:

[https://api.github.com/search/issues?](https://api.github.com/search/issues?

so is it possible to reduce the Github API invoke by disabling some features?

@anderssonjohan
Copy link

What I don't get from this error is that the numbers seems wrong. It always says limit 30 and remaining 29, while in any other example we see limit 1000, remaining 0 (and "used" is often 1005 or similar) when getting the error.

A small semantic-release demo repo where I tried to reproduce the error:
https://github.com/anderssonjohan/semantic-release-github-actions-example/

Example run showing the very small rate limit for search:
https://github.com/anderssonjohan/semantic-release-github-actions-example/runs/7784143308?check_suite_focus=true

...
"search": {
      "limit": 30,
      "used": 2,
      "remaining": 28,
      "reset": 1660210515
    },
...

With the builds of the example repo taking only 1 min, I can only get the "used" counter up to 2. Next run it's back to 30. So it more seems to be 30 requests/min.

However, when you get the error from GitHub, the remaining + used counters still seem wrong:

      'x-ratelimit-limit': '30',
      'x-ratelimit-remaining': '29',
      'x-ratelimit-resource': 'search',
      'x-ratelimit-used': '1',
      'x-xss-protection': '0'

Reading the semantic-release/github code it seems it can produce a lot of search requests if there are many commits pushed at the same time?

@viceice
Copy link

viceice commented Aug 18, 2022

😕

An error occurred while running semantic-release: RequestError [HttpError]: You have exceeded a secondary rate limit. Please wait a few minutes before you try again.
    at /home/runner/work/buildpack/buildpack/node_modules/@octokit/request/dist-node/index.js:88:21
    at processTicksAndRejections (node:internal/process/task_queues:96:5) {
  status: 403,
  response: {
    url: 'https://api.github.com/search/issues?q=in%3Atitle+repo%3Acontainerbase%2Fbuildpack+type%3Aissue+state%3Aopen+The%20automated%20release%20is%20failing%20%F0%9F%9A%A8',
    status: 403,
    headers: {
      'access-control-allow-origin': '*',
      'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset',
      connection: 'close',
      'content-encoding': 'gzip',
      'content-security-policy': "default-src 'none'",
      'content-type': 'application/json; charset=utf-8',
      date: 'Thu, 18 Aug 2022 06:38:52 GMT',
      'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
      server: 'GitHub.com',
      'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
      'transfer-encoding': 'chunked',
      vary: 'Accept-Encoding, Accept, X-Requested-With',
      'x-content-type-options': 'nosniff',
      'x-frame-options': 'deny',
      'x-github-media-type': 'github.v3; format=json',
      'x-github-request-id': '0781:4BC1:3B86C5:A50266:62FDDE7C',
      'x-ratelimit-limit': '30',
      'x-ratelimit-remaining': '29',
      'x-ratelimit-reset': '1660804792',
      'x-ratelimit-resource': 'search',
      'x-ratelimit-used': '1',
      'x-xss-protection': '0'
    },
    data: {
      documentation_url: 'https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits',
      message: 'You have exceeded a secondary rate limit. Please wait a few minutes before you try again.'
    }
  },
  request: {
    method: 'GET',
    url: 'https://api.github.com/search/issues?q=in%3Atitle+repo%3Acontainerbase%2Fbuildpack+type%3Aissue+state%3Aopen+The%20automated%20release%20is%20failing%20%F0%9F%9A%A8',
    headers: {
      accept: 'application/vnd.github.v3+json',
      'user-agent': 'octokit-rest.js/19.0.3 octokit-core.js/4.0.4 Node.js/16.16.0 (linux; x64)',
      authorization: 'token [REDACTED]'
    },
    request: { agent: undefined, hook: [Function: bound bound register] }
  },
  pluginName: '@semantic-release/github'
}

https://github.com/containerbase/buildpack/runs/7893136166?check_suite_focus=true

@yarkoyarok
Copy link

We had the same, publish to npm registry not happened because of this error and on restart of action It was not publishing already, I guess because finding own commits after the commit, which triggered release.

It was telling on restart that:

The local branch main is behind the remote one, therefore a new version won't be published.

@yarkoyarok
Copy link

In our case we had such headers too

      'x-ratelimit-limit': '30',
      'x-ratelimit-remaining': '29',
      'x-ratelimit-reset': '1660805994',
      'x-ratelimit-resource': 'search',
      'x-ratelimit-used': '1',

@anderssonjohan
Copy link

We had the same, publish to npm registry not happened because of this error and on restart of action It was not publishing already, I guess because finding own commits after the commit, which triggered release.

It was telling on restart that:

The local branch main is behind the remote one, therefore a new version won't be published.

Re-running jobs at GitHub Actions for example will check out the same commit again so it will always end up in that error since there will be a commit made by semantic-release (version number bump and possibly update to a changelog file).
My only tip for re-triggering the release in that case is unfortunately to do a git commit --allow-empty -m "fix: trigger release" and push.

@yuliankarapetkov
Copy link

yuliankarapetkov commented Sep 19, 2022

Any update on this? 👀

The issue was reported in version 17.4.7. I'm using 19.0.3 and the issue is still there.

@yuliankarapetkov
Copy link

Our CI workflows are now failing every time due to this issue.

Shelex pushed a commit to Shelex/cypress-allure-plugin that referenced this issue Sep 30, 2022
@foxted
Copy link

foxted commented Oct 17, 2022

Same problem here. However, looks like the release is created, only the "success" step is failing, which makes Github Action report a failure.

@AdamBebko
Copy link

Same issue here. Release gets created in GitHub, but doesn't trigger the followup custom actions on success.

@rikkit
Copy link

rikkit commented Nov 3, 2022

I have started to get this too, after having used semantic release for a few weeks now. Looks like the PR to follow is semantic-release/github#487

@mathcodes
Copy link

Just started to get it here too...

coderbyheart added a commit to NordicSemiconductor/cloud-aws-firmware-ci-feature-runner-action that referenced this issue Apr 13, 2023
coderbyheart added a commit to NordicSemiconductor/e118-iin-list-js that referenced this issue Apr 13, 2023
coderbyheart added a commit to NordicSemiconductor/cloud-azure-firmware-ci-feature-runner-action that referenced this issue Apr 13, 2023
coderbyheart added a commit to NordicSemiconductor/cloud-e2e-bdd-test-runner-example-js that referenced this issue Apr 13, 2023
coderbyheart added a commit to NordicSemiconductor/asset-tracker-cloud-memfault-aws-js that referenced this issue Apr 13, 2023
coderbyheart added a commit to NordicSemiconductor/cloud-azure-firmware-ci-runner-js that referenced this issue Apr 13, 2023
@foxhound87
Copy link

I'm getting this error randomly on my project
An error occurred while running semantic-release: RequestError [HttpError]: You have exceeded a secondary rate limit. Please wait a few minutes before you try again.

@beiertu-mms
Copy link

I'm getting this error randomly on my project An error occurred while running semantic-release: RequestError [HttpError]: You have exceeded a secondary rate limit. Please wait a few minutes before you try again.

@foxhound87 this is currently the work-around.

@foxhound87
Copy link

@beiertu-mms thank you. where should add that config?
my actual workflow config is like this:
https://github.com/foxhound87/mobx-react-form/blob/master/.github/workflows/ci.yml

@beiertu-mms
Copy link

@beiertu-mms thank you. where should add that config? my actual workflow config is like this: foxhound87/mobx-react-form@master/.github/workflows/ci.yml

@foxhound87 it depends on how you configure it.
We use a .releaserc with the following content (I can't share a link to the file because it's an internal repo :) )

{
  "branches": [
    "master"
  ],
  "plugins": [
    "@semantic-release/commit-analyzer",
    [
      "@semantic-release/github",
      {
        "successComment": false,
        "failTitle": false
      }
    ],
    "@semantic-release/release-notes-generator"
  ]
}

github-actions bot pushed a commit to shuttleday/shuttleday that referenced this issue Apr 27, 2023
## [1.1.1](v1.1.0...v1.1.1) (2023-04-27)

### Bug Fixes

* **ci:** prevent semantic-release hitting secondary rate limit ([639802d](639802d)), closes [/github.com/semantic-release/semantic-release/issues/2204#issuecomment-1486299917](https://github.com//github.com/semantic-release/semantic-release/issues/2204/issues/issuecomment-1486299917)

### Performance Improvements

* run processUploadedFiles at middleware level ([f0dd353](f0dd353))
lapa182 pushed a commit to duffelhq/duffel-api-javascript that referenced this issue May 4, 2023
This is a work-around to stop the rate limit issue when publishing. You can check the issue here semantic-release/semantic-release#2204
lapa182 pushed a commit to duffelhq/duffel-api-javascript that referenced this issue May 4, 2023
)

* docs(readme): update README to include the minimum node.js version

* ci(release): stop adding comments to issues

This is a work-around to stop the rate limit issue when publishing. You can check the issue here semantic-release/semantic-release#2204
willsoto added a commit to willsoto/nestjs-objection that referenced this issue Sep 29, 2023
See semantic-release/semantic-release#2204

Signed-off-by: Will Soto <willsoto@users.noreply.github.com>
lydavid pushed a commit to lydavid/MusicSearch that referenced this issue Nov 26, 2023
coderbyheart added a commit to NordicSemiconductor/asset-tracker-cloud-firmware-aws that referenced this issue Feb 19, 2024
tbinna added a commit to toolsplus/nx-forge that referenced this issue Apr 22, 2024
disable semantic release success comments to fix Github rate limits as suggested here: semantic-release/semantic-release#2204 (comment)
@pcnate
Copy link

pcnate commented Jun 7, 2024

For those of you who still experience this problem. I solved this in the meantime by disabling the features in the @semantic-release/github plugin that uses the https://api.github.com/search endpoint, which is the one that causes the secondary rate limit exceeds.

Configure @semantic-release/github as follows:

[
  "@semantic-release/github",
  {
    "successComment": false,
    "failTitle": false
  }
]

What would be great is push these at a different server like one that I control so that I can still make the requests to post these comments but do so with a queue and delays. I probably don't need it to be posted exactly when the release is being made but some time afterwards

raae added a commit to cloudinary-community/gatsby-transformer-cloudinary that referenced this issue Jun 7, 2024
@IvanJosipovic
Copy link

Hitting this bug regularly, we shouldn't have to disable features of this tool for it to work consistently :(

@dword-design
Copy link

Same here

@gr2m
Copy link
Member

gr2m commented Jul 1, 2024

It's being addressed: semantic-release/github#857

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests