GitHub API rate limit exceeded when yarn policies set-version ^1

[manrueda]

Bug description

When running yarn policies set-version ^1 yarn tries to use GitHub API to access the releases (calls to https://api.github.com/repos/yarnpkg/yarn/releases) and because all unauthenticated requests have a rate limit based in IP. We are in a corporate network where almost all traffic goes out using the same public IP so that limit gets hit almost immediately.

Command

yarn policies set-version ^1

What is the current behavior?
GitHub API rate limit gets hit.

What is the expected behavior?
The CLI should be able to get the release versions successfully.

Steps to Reproduce
Any repository will do, just need to be in a network that goes out using the same public IP to hit the rate limit.

1. Run yarn policies set-version ^1

Environment

• Node Version: 12.14.1
• Yarn v1 Version: 1.21.1
• OS and version: OSX 10.15.2

[olingern]

You can use a Github personal access token with no permissions granted.

Create here:
https://github.com/settings/tokens

Add to your env. Example:
export GITHUB_TOKEN=token_created

[manrueda]

I work in a very board and distributed organization with multiple teams. And, believe it or not, not every developer has a GitHub account.

This makes the tool dependent on having a GitHub account, because of that will make all out projects that use it dependent also.

[olingern]

Not to be too snarky, but we are on Github right now. 🙂

I think making yarn policies set-version work for everyone is a great endeavor, but with limited resources and time -- what's currently here should work for most people.

I'll leave this open and mark as wont-fix for now.

[manrueda]

You and I are on Github right now. I can not say the same about hundreds of developers that work on our code base.

Also to consume, even documentation, an account is not needed. You can just jump to GitHub and inspect code and read documentation without an account. I not talking about the dependency over GitHub but the dependency of a GitHub account.

I understand and respect time/resources constraints, so I will not push on a solution unless I can contribute and provide my time to solve it.

Thanks!

[arcanis]

Hey @manrueda. The idea behind yarn policies set-version is that you check-in the resulting binary to your repository, ensuring that all developers will always use the exact same version. Telling everyone to run it is an anti-pattern (you can just use the global binary in this case¹).

You can read more about this here.

¹ If you're using policies set-version only due to the Yarn 2 post, we're currently rethinking the migration strategy and it's extremely likely that the yarn package on npm will stay on 1.x for a very long time. Still - check-in the Yarn binary, you really realize the benefits once you need to do your first sweep upgrade.

[Daniel15]

Something we could do to avoid the GitHub API rate limits is periodically scrape the version list from the GitHub API and stick it somewhere else (eg. in a file on the Yarn site), then Yarn could look there rather than hitting the API. I could write a script to do that as part of the release tooling, or we could have a GitHub Action to do it (if it's possible to schedule them)

[manrueda]

@arcanis Yeah, I didn't realize that. That will solve my problem.

Also yes, I am doing this to prepare ourself for yarn 2. We have many many many repos that use yarn and we will not be able to move to 2 all at once, so for a long period of time we will have developers working in a computer with repos using v1 and others with v2.