Debian CloudFoundry key not accessible by typical wget / curl on Ubuntu

[SeanKilleen]

Please fill out the issue checklist below and provide ALL the requested information.

• I reviewed open and closed github issues that may be related to my problem.
• I tried updating to the latest version of the CF CLI to see if it fixed my problem.
• I attempted to run the command with CF_TRACE=1 to help debug the issue.
• I am reporting a bug that others will be able to reproduce.

Describe the bug and the command you saw an issue with

I am attempting to use the CF CLI on Ubuntu (in a GitHub action via ubuntu-latest runner, and reproduced locally on Ubuntu 22.04 as well)

What happened

• I attempted to use the installation instructions from the CF-CLI but could not get it to work.
• I kept seeing that no PGP information was found.
• When I broke down the command, I decided to try the wget command first. It was empty. I then tried with curl, and I saw an access denied error message. That's odd...because I had just gone to that URL in my browser a moment ago.
  • URL in question: https://packages.cloudfoundry.org/debian/cli.cloudfoundry.org.key
• I was able to fix by supplying a user agent with curl's -A option.

Expected behavior

CF8-CLI to install correctly

Exact Steps To Reproduce

On Ubuntu 22.04

curl https://packages.cloudfoundry.org/debian/cli.cloudfoundry.org.key | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/cloudfoundrycli.gpg
echo "deb https://packages.cloudfoundry.org/debian stable main" | sudo tee /etc/apt/sources.list.d/cloudfoundry-cli.list
sudo apt-get update
sudo apt-get install cf8-cli

What does work (workaround):

curl -A "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:15.0) Gecko/20100101 Firefox/15.0.1" https://packages.cloudfoundry.org/debian/cli.cloudfoundry.org.key | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/cloudfoundrycli.gpg
echo "deb https://packages.cloudfoundry.org/debian stable main" | sudo tee /etc/apt/sources.list.d/cloudfoundry-cli.list
sudo apt-get update
sudo apt-get install cf8-cli

It appears that some sort of security tooling or configuration is blocking access to the public key data via typical wget / curl commands, which have a legitimate use case.

[sammetb]

Getting the same access denied

`

<TITLE>Access Denied</TITLE>

Access Denied

You don't have permission to access "http://packages.cloudfoundry.org/stable?" on this server.

Reference #18.26701102.1679991638.9dbfd60

`

This was working well until yesterday late afternoon UTC+1

[pfrenssen]

I can confirm. This started failing today. Akamai is returning a 403 when retrieving the GPG key using curl or wget.

[vlaurin]

This access denied is also returned when trying to download compressed binaries with:

curl -L "https://packages.cloudfoundry.org/stable?release=linux64-binary&version=v7&source=github"

as per https://github.com/cloudfoundry/cli/blob/main/doc/installation-instructions/installation-instructions-v7.md#installers-and-compressed-binaries

[vlaurin]

It looks like the 403 error is because of the curl user agent. Doing the same request without the user agent (option -A "") works okay for me.

curl -A "" -L "https://packages.cloudfoundry.org/stable?release=linux64-binary&version=v7&source=github"

Further to the above:

• curl -A "cur" works
• curl -A "curl" fails with 403
  So it really looks like it's the curl user agent that's rejected.

[andreasvoigt]

It is also possible to use a custom user agent like for a standard Chrome browser to be able to retrieve the pgp key.
curl -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36" https://packages.cloudfoundry.org/debian/cli.cloudfoundry.org.key

[SeanKilleen]

Thanks for all the validation here! I was creating a build process using the CF cli for the first time yesterday. It worked in my first workflow but failed in my second and I couldn't understand what I had done wrong. Bad luck/timing it seems! 😆

[sammetb]

Thanks for all the validation here! I was creating a build process using the CF cli for the first time yesterday. It worked in my first workflow but failed in my second and I couldn't understand what I had done wrong. Bad luck/timing it seems! 😆

Haha, yes. Using it also in a couple of pipelines for my customers and they all mentioned the issue.
I've provided a fix on a alternative location from a manually download of the cli files.
This will be used if the endpoint would return a statuscode != 200

[a-b]

We're looking into this issue. CC @ccjaimes

[a-b]

This issue should be resolved by now. The primary reason is a new CDN security rule to fight bots. We will take additional steps to improve monitoring to respond faster to such incidents.