The requested URL returned error: 419

[0xNOY]

yay -S ros-noetic-desktop-full returns the following error.

...
ros-noetic-desktop-full の取得時にエラー: Cloning into 'ros-noetic-desktop-full'...
fatal: unable to access 'https://aur.archlinux.org/ros-noetic-desktop-full.git/': The requested URL returned error: 429
	 context: exit status 128

ros-noetic-compressed-depth-image-transport の取得時にエラー: Cloning into 'ros-noetic-compressed-depth-image-transport'...
error: RPC failed; HTTP 429 curl 22 The requested URL returned error: 429
fatal: expected flush after ref listing
	 context: exit status 128

ros-noetic-theora-image-transport の取得時にエラー: Cloning into 'ros-noetic-theora-image-transport'...
fatal: unable to access 'https://aur.archlinux.org/ros-noetic-theora-image-transport.git/': The requested URL returned error: 429
	 context: exit status 128

ros-noetic-image-transport-plugins の取得時にエラー: Cloning into 'ros-noetic-image-transport-plugins'...
fatal: unable to access 'https://aur.archlinux.org/ros-noetic-image-transport-plugins.git/': The requested URL returned error: 429
	 context: exit status 128

ros-noetic-image-common の取得時にエラー: Cloning into 'ros-noetic-image-common'...
fatal: unable to access 'https://aur.archlinux.org/ros-noetic-image-common.git/': The requested URL returned error: 429
	 context: exit status 128

ros-noetic-image-publisher の取得時にエラー: Cloning into 'ros-noetic-image-publisher'...
error: RPC failed; HTTP 429 curl 22 The requested URL returned error: 429
fatal: expected flush after ref listing
	 context: exit status 128

ros-noetic-position-controllers の取得時にエラー: Cloning into 'ros-noetic-position-controllers'...
fatal: unable to access 'https://aur.archlinux.org/ros-noetic-position-controllers.git/': The requested URL returned error: 429
	 context: exit status 128

ros-noetic-joint-limits-interface の取得時にエラー: Cloning into 'ros-noetic-joint-limits-interface'...
error: RPC failed; HTTP 429 curl 22 The requested URL returned error: 429
fatal: expected flush after ref listing
	 context: exit status 128
...

How can I change the execution speed of the git clone command?
Thank you.

[erus]

It looks like the issue here is not the speed at which the commands are issued, but the number of them. See https://www.reddit.com/r/archlinux/comments/w56khw/how_does_the_ratelimit_or_error_429_in_the_aur/

[gnaggnoyil]

I'm having the same issue now, where I have many packages need to upgrade and yay returns this error.

[AladW]

Yay guzzles AUR resources. The issue is two-fold: rate-limiting on the RPC and rate-limiting on git operations.

For the former, yay's provides heuristic causes an excessive amount of requests to resolve dependencies, and for the latter parallel clones cause the burst limit on git operations to be hit very quickly.

The action plan I propose is far beyond the scope of this issue, so here's the quick summary:

RPC

• Disable provides heuristic by default
• When enabled, only use search+info for missing targets in the dependency chain, not all targets.
• Avoid splitting on - when doing provides searches, this can cause the number of results to blow up
• Use the metadata archives with a reasonable TTL. This also can handle things like .so depends.
• Allow users to point yay to a different RPC backend, e.g. https://github.com/moson-mo/goaurrpc

These are multiple alternatives, not things that have to be implemented in order. The first bullet is the quickest fix either way.

Git

• Disable parallel git requests by default
• Use the read-only github mirror for cloning repositories, cf. lib: add aur-fetch--mirror aurutils/aurutils#1017

I should add that using the github mirror vastly outperforms regular git-clones, parallel or not. Updates for hundreds of packages can be retrieved in seconds instead of minutes.

[Jguer]

Hey @AladW, I'll push an update to yay with provides disabled by default and a migration that sets existing installs/configs to false as most were probably left on by default and we have no way of differentiating default from custom settings for now.

This should alleviate the issue for now and hopefully avoid taxing the AUR's resources.

Once that's done I'll verify how the metadata archives could replace some queries (and probably have a local cache)

Allow users to point yay to a different RPC backend, e.g. https://github.com/moson-mo/goaurrpc

If I understand the point, users should already be able to use the aururl setting to change the RPC server

[Jguer]

As of yay v11.3.0 , provides has been disabled by default. I'd move on to #1781 for continued provides future work

[moson-mo]

If I understand the point, users should already be able to use the aururl setting to change the RPC server

aururl is also used to link to other resources from aurweb, not only /rpc.... To make that work there needs to be an additional url setting like aurrpcurl or so.

btw. I'd be happy work on this and create a merge request if that makes sense @Jguer

edit
Proposal

[Jguer]

Hey @moson-mo , we can go with that if you're willing to put in effort 👍 thanks