Release-drafter gets HTTP 500 error when trying to fetch more than 1000 releases

[seancaulfield]

Going to (try to) report this to Github as well, but posting a bug report here first so others hopefully aren't left wondering if they were crazy.

The Issue

Sometime between 2022-02-23 17:37:24 GMT and 2022-02-23 21:33:53 GMT, the Github API started returning 500 errors when you try to retrieve past 1000 releases. We discovered this because the release-drafter Github action suddenly stopped working in that time frame.

Here is a Github Action log snippet from before, when release-drafter found our 1227 releases just fine:

Wed, 23 Feb 2022 17:37:10 GMT Run release-drafter/release-drafter@v5
Wed, 23 Feb 2022 17:37:24 GMT pepsico-ecommerce/airflow_dags: Found 1227 releases
Wed, 23 Feb 2022 17:37:24 GMT { name: 'event', id: '1888766142' }
Wed, 23 Feb 2022 17:37:24 GMT pepsico-ecommerce/airflow_dags: Draft release: 3.0.718
Wed, 23 Feb 2022 17:37:24 GMT { name: 'event', id: '1888766142' }
Wed, 23 Feb 2022 17:37:24 GMT pepsico-ecommerce/airflow_dags: Last release: 3.0.717
Wed, 23 Feb 2022 17:37:24 GMT { name: 'event', id: '1888766142' }
Wed, 23 Feb 2022 17:37:24 GMT pepsico-ecommerce/airflow_dags: Fetching all commits for reference refs/heads/main since 2022-02-23T12:35:36Z
Wed, 23 Feb 2022 17:37:24 GMT { name: 'event', id: '1888766142' }
Wed, 23 Feb 2022 17:37:24 GMT pepsico-ecommerce/airflow_dags: Updating existing release
Wed, 23 Feb 2022 17:37:24 GMT { name: 'event', id: '1888766142' }

The next time someone pushed a commit was a few hours later, and every run of release-drafter since has failed with this 500 error:

Wed, 23 Feb 2022 21:33:27 GMT Run release-drafter/release-drafter@v5
Wed, 23 Feb 2022 21:33:53 GMT Error: invalid json response body at https://api.github.com/repositories/191452916/releases?per_page=100&page=11 reason: Unexpected end of JSON input

Testing the URL with a personal Github token runs into the same issue: as soon as I try to fetch further back than 1000 releases, I get a 500 error.

Note that the Link header is still suggesting that these pages are valid to retrieve:

The issue seems to be around 1000 releases and not page numbers, as changing the per_page parameter changes where the 500 error pops up.

Environment

I've tried updating to the latest version of release-drafter (v5.18.1), but still get the same behavior, which is not entirely surprising if this is an API issue.

We are using release-drafter via Github Actions. Our action parameters are extremely basic:

  update_release_draft:
    runs-on: ubuntu-latest
    needs: [bumpversion]
    steps:
      - uses: release-drafter/release-drafter@v5.18.1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

As is our release-drafter.yml (lightly edited):

filter-by-commitish: true
commitish: main
name-template: 'Airflow2 release: 3.0.726'
tag-template: '3.0.726'
change-template: '- $TITLE (#$NUMBER, @$AUTHOR)'
template: |
  ## Airflow v2, release: 3.0.726
  ## Changes
  $CHANGES
  ## Contributors
  $CONTRIBUTORS

Note: I am NOT with the official Apache Airflow project. My team are just users of it. This is also happening with a private repository, or I'd provide direct links to the code.

Possible Workaround?

As a workaround for this, is there any way to limit the number of releases that release-drafter tries to fetch? (I realize y'all are not in charge of the Github REST API and don't expect miracles here.)

[jetersen]

Would love to see a PR for fixing it. We could set an limit on something reasonable like 500 or just max it to 1000.

It seems doable:

release-drafter/lib/releases.js

Lines 25 to 31 in bd3f0d7

	let releases = await context.octokit.paginate(
	context.octokit.repos.listReleases.endpoint.merge(
	context.repo({
	per_page: 100,
	})
	)
	)

See the use of done in the example here: https://michaelheap.com/octokit-pagination/

[jetersen]

Sometime between 2022-02-23 17:37:24 GMT and 2022-02-23 21:33:53 GMT, the Github API started returning 500 errors when you try to retrieve past 1000 releases. We discovered this because the release-drafter Github action suddenly stopped working in that time frame.

Seems to be around the same time as #1081

[eddmann]

Hey, we experienced this issue with the 500 HTTP response, as a quick workaround for now we have applied this.

mybuilder@de80a23

[jetersen]

@eddmann feel free to open a PR :)

[eddmann]

Sure thing @jetersen #1085 :)

[juanecabellob]

this is also happening to us. @jetersen when are you planning to release the patch?

[jetersen]

You can point to master to the time being.

[eddmann]

This looks to have been addressed upstream within the GitHub API, so this fix can be removed 👍🏻