compile-sketches reporting 403 when trying to download arduino-cli

[nomakewan]

Describe the problem

When trying to run the compile-sketches action, all actions fail and the traceback indicates that this is due to a 403 when attempting to access the URL for the Arduino CLI.

Traceback (most recent call last):
  File "/home/runner/work/_actions/arduino/compile-sketches/main/compilesketches/compilesketches.py", line 1717, in <module>
    main()  # pragma: no cover
    ^^^^^^
  File "/home/runner/work/_actions/arduino/compile-sketches/main/compilesketches/compilesketches.py", line 63, in main
    compile_sketches.compile_sketches()
  File "/home/runner/work/_actions/arduino/compile-sketches/main/compilesketches/compilesketches.py", line 216, in compile_sketches
    self.install_arduino_cli()
  File "/home/runner/work/_actions/arduino/compile-sketches/main/compilesketches/compilesketches.py", line 253, in install_arduino_cli
    self.install_from_download(
  File "/home/runner/work/_actions/arduino/compile-sketches/main/compilesketches/compilesketches.py", line 290, in install_from_download
    with contextlib.closing(thing=urllib.request.urlopen(url=url)) as file_pointer:
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/urllib/request.py", line 525, in open
    response = meth(req, response)
               ^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/urllib/request.py", line 634, in http_response
    response = self.parent.error(
               ^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/urllib/request.py", line 563, in error
    return self._call_chain(*args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

To reproduce

Attempt to run the compile-sketches action.

Expected behavior

Sketches compile.

'arduino/compile-sketches' version

Same error both with v1 and main

Additional context

No response

Issue checklist

• I searched for previous reports in the issue tracker
• I verified the problem still occurs when using the latest version
• My report contains all necessary details

[aliphys]

As a result of this issue, the compile-sketches.yml workflow is failing across multiple owners. E.g.

• cristidragomir97/Arduino_Portenta_C33_LowPower
• arduino-libraries / Arduino_PortentaMachineControl
• aliphys/ArduinoBasicCompileSketch

The compile-sketches.yml workflow relies on the downloads.arduino.cc domain for obtaining the CLI, as defined in the install_arduino_cli method here.

    def install_arduino_cli(self):
        """Install Arduino CLI."""
        self.verbose_print("Installing Arduino CLI version", self.cli_version)
        arduino_cli_archive_download_url_prefix = "https://downloads.arduino.cc/arduino-cli/"
        arduino_cli_archive_file_name = "arduino-cli_" + self.cli_version + "_Linux_64bit.tar.gz"

The urllib.error.HTTPError: HTTP Error 403: Forbidden error suggests that https://downloads.arduino.cc/arduino-cli/arduino-cli_latest_Linux_64bit.tar.gz is inaccessible to the GitHub Runner.

[Foroxon]

Here is the fix: Foroxon@2c502ef

[nomakewan]

Here is the fix: Foroxon@2c502ef

Awesome! Mind slapping that into a PR so it can be merged with main?

[Foroxon]

#228

[per1234]

Thanks for your report @nomakewan!

As already indicated by @Foroxon, this was related to the default User-Agent header produced by urllib.

A change to the arduino.cc web infrastructure yesterday caused requests with that header to be blocked.

The Arduino web team have since restored compatibility with the User-Agent header produced by urllib so the fault will no longer occur even without applying @Foroxon's patch.