deb822_repository fails to fetch signed_by key: HTTP Error 403: Forbidden

[sebdanielsson]

Summary

When I try to add the cloudflared repo with the new deb822_repository module I get the following response:

fatal: [xxxxxx]: FAILED! => {"changed": false, "msg": "Could not fetch signed_by key: HTTP Error 403: Forbidden"}

I've tested the module with the Tailscale repo and that worked. Then I tested if there was something up with the cloudflare repo but fetching the gpg key with wget and curl -fsSL https://pkg.cloudflare.com/cloudflare-main.gpg | sudo tee /usr/share/keyrings/cloudflare-main.gpg >/dev/null work as expected. Must be some kind of mismatch between this module and the cloudflared repository but I'm not really skilled enough to troubleshoot this further.

Issue Type

Bug Report

Component Name

deb822_repository

Ansible Version

$ ansible --version
ansible [core 2.15.0]
  config file = None
  configured module search path = ['/Users/sebastian/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/homebrew/lib/python3.11/site-packages/ansible
  ansible collection location = /Users/sebastian/.ansible/collections:/usr/share/ansible/collections
  executable location = /opt/homebrew/bin/ansible
  python version = 3.11.3 (main, Apr  7 2023, 20:13:31) [Clang 14.0.0 (clang-1400.0.29.202)] (/opt/homebrew/opt/python@3.11/bin/python3.11)
  jinja version = 3.1.2
  libyaml = False

Configuration

# if using a version older than ansible-core 2.12 you should omit the '-t all'
$ ansible-config dump --only-changed -t all

OS / Environment

macOS Ventura 13.3.1 (a)

Steps to Reproduce

This code doesn't work

---
- name: xxxxxx Proxmox
  hosts: xxxxxx

  tasks:
    - name: Add cloudflared repository
      become: true
      ansible.builtin.deb822_repository:
        name: cloudflared
        types: deb
        uris: https://pkg.cloudflare.com/cloudflared
        suites: "{{ ansible_distribution_release }}"
        components: main
        signed_by: https://pkg.cloudflare.com/cloudflare-main.gpg
        state: present

This code is working

---
- name: xxxxxx Proxmox
  hosts: xxxxxx

  tasks:
    - name: Add tailscale repository
      become: true
      ansible.builtin.deb822_repository:
        name: tailscale
        types: deb
        uris: https://pkgs.tailscale.com/stable/debian
        suites: "{{ ansible_distribution_release }}"
        components: main
        signed_by: https://pkgs.tailscale.com/stable/debian/bullseye.noarmor.gpg
        state: absent

Expected Results

I expected the repository to be added with the specified key.

Actual Results

ansible-playbook -i ./inventory.yml ./apt_beta.yml

PLAY [xxxxxx Proxmox] ****************************************************************

TASK [Gathering Facts] ******************************************************************
ok: [xxxxxx]

TASK [Add cloudflared repository] ***********************************************
fatal: [xxxxxx]: FAILED! => {"changed": false, "msg": "Could not fetch signed_by key: HTTP Error 403: Forbidden"}

PLAY RECAP ******************************************************************************
xxxxxx                  : ok=1    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[ansibot]

Files identified in the description:

• lib/ansible/modules/deb822_repository.py

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

[sivel]

The issue would seem to be that pkg.cloudflare.com is blocking HTTP requests with specific user agents, such as the default python user agent.

curl -A 'Python-urllib/3.11' https://pkg.cloudflare.com/cloudflare-main.gpg

deb822_repository uses open_url which does not set a default user agent, whereas fetch_url does. We likely just need to set the user agent to the same value of ansible-httpget.

diff --git a/lib/ansible/modules/deb822_repository.py b/lib/ansible/modules/deb822_repository.py
index 0c706ce06e..9f90b88e06 100644
--- a/lib/ansible/modules/deb822_repository.py
+++ b/lib/ansible/modules/deb822_repository.py
@@ -325,7 +325,7 @@ def write_signed_by_key(module, v, slug):
     parts = generic_urlparse(urlparse(v))
     if parts.scheme:
         try:
-            r = open_url(v)
+            r = open_url(v, http_agent='ansible-httpget')
         except Exception as exc:
             raise_from(RuntimeError(to_native(exc)), exc)
         else:

[sivel]

I've added easyfix to this, since I've basically provided a fix. If anyone is interested in working on this feel free to put together a full PR with hopefully tests if feasible to test this, and a changelog entry.

[Akasurde]

resolved_by_pr #80876