dnf module ignores repo files inside installroot path

[mskarbek]

Summary

When using installroot option, dnf module ignores repo files inside installroot path.

The playbook creates an empty directory (actually it is buildah mount point, but that should be irrelevant to this issue) and then creates a minimal dnf configuration adding dnf.conf and repo file inside that empty directory.

dnf command will work just fine:

# dnf -c /var/lib/containers/storage/zfs/graph/07ff29954653e299e16949763371a53c633b96a8825023597135cc68c0a0253f/etc/dnf/dnf.conf --installroot /var/lib/containers/storage/zfs/graph/07ff29954653e299e16949763371a53c633b96a8825023597135cc68c0a0253f/ --disablerepo="*" --enablerepo="baseos" --disableplugin="subscription-manager" --releasever=9 install glibc-minimal-langpack coreutils-single
Last metadata expiration check: 0:30:18 ago on śro, 26 kwi 2023, 12:40:31.
Dependencies resolved.
==============================================================================================================================================================================
 Package                                           Architecture                      Version                                          Repository                         Size
==============================================================================================================================================================================
Installing:
 coreutils-single                                  x86_64                            8.32-32.el9                                      baseos                            601 k
 glibc-minimal-langpack                            x86_64                            2.34-40.el9_1.1                                  baseos                             28 k
Installing dependencies:
 almalinux-gpg-keys                                x86_64                            9.1-1.9.el9                                      baseos                            9.6 k
 almalinux-release                                 x86_64                            9.1-1.9.el9                                      baseos                             20 k
 almalinux-repos                                   x86_64                            9.1-1.9.el9                                      baseos                            9.7 k
 basesystem                                        noarch                            11-13.el9                                        baseos                            7.0 k
 bash                                              x86_64                            5.1.8-6.el9_1                                    baseos                            1.7 M
 filesystem                                        x86_64                            3.16-2.el9                                       baseos                            1.1 M
 glibc                                             x86_64                            2.34-40.el9_1.1                                  baseos                            1.9 M
 glibc-common                                      x86_64                            2.34-40.el9_1.1                                  baseos                            300 k
 libacl                                            x86_64                            2.3.1-3.el9                                      baseos                             24 k
 libattr                                           x86_64                            2.5.1-3.el9                                      baseos                             18 k
 libcap                                            x86_64                            2.48-8.el9                                       baseos                             68 k
 libgcc                                            x86_64                            11.3.1-2.1.el9.alma                              baseos                            115 k
 libselinux                                        x86_64                            3.4-3.el9                                        baseos                             85 k
 libsepol                                          x86_64                            3.4-1.1.el9                                      baseos                            315 k
 ncurses-base                                      noarch                            6.2-8.20210508.el9                               baseos                             61 k
 ncurses-libs                                      x86_64                            6.2-8.20210508.el9                               baseos                            323 k
 pcre2                                             x86_64                            10.40-2.el9                                      baseos                            236 k
 pcre2-syntax                                      noarch                            10.40-2.el9                                      baseos                            143 k
 setup                                             noarch                            2.13.7-7.el9                                     baseos                            140 k
 tzdata                                            noarch                            2023c-1.el9                                      baseos                            432 k

Transaction Summary
==============================================================================================================================================================================
Install  22 Packages

Total download size: 7.5 M
Installed size: 22 M
Is this ok [y/N]: 

dnf module fails because it tries to use repo files in host /etc/yum.repos.d not in the container path. When I copy the repo file to the host path, dnf module works fine.

Issue Type

Bug Report

Component Name

dnf

Ansible Version

# ansible --version

ansible [core 2.13.3]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.9/site-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /bin/ansible
  python version = 3.9.14 (main, Nov  7 2022, 00:00:00) [GCC 11.3.1 20220421 (Red Hat 11.3.1-2)]
  jinja version = 3.1.2
  libyaml = True

Configuration

# if using a version older than ansible-core 2.12 you should omit the '-t all'
$ ansible-config dump --only-changed -t all

OS / Environment

RHEL 9.1

Steps to Reproduce

    - ansible.builtin.file:
        dest: "{{ container_path }}{{ item }}"
        state: "touch"
        owner: "root"
        group: "root"
        mode: "0644"
      loop:
        - "/etc/dnf/dnf.conf"
        - "/etc/yum.repos.d/almalinux-baseos.repo"
    - ansible.builtin.template:
        src: "dnf.conf.j2"
        dest: "{{ container_path }}/etc/dnf/dnf.conf"
        owner: "root"
        group: "root"
        mode: "0644"
    - ansible.builtin.copy:
        src: "repos/almalinux-baseos.repo"
        dest: "{{ container_path }}/etc/yum.repos.d/almalinux-baseos.repo"
        owner: "root"
        group: "root"
        mode: "0644"
    - ansible.builtin.dnf:
        name:
          - "glibc-minimal-langpack"
          - "coreutils-single"
        conf_file: "{{ container_path }}/etc/dnf/dnf.conf"
        disablerepo: "*"
        enablerepo: "baseos"
        disable_plugin:
          - "subscription-manager"
        install_weak_deps: false
        releasever: 9
        state: "latest"
        disable_gpg_check: true
        installroot: "{{ container_path }}"

Expected Results

dnf module properly installs packages in installroot path using repo files inside installroot.

Actual Results

fatal: [localhost]: FAILED! => {                                                                                                                                              
    "changed": false,                                                                                                                                                         
    "failures": [                                                                                                                                                             
        "No package glibc-minimal-langpack available.",                                                                                                                       
        "No package coreutils-single available."                                                                                                                              
    ],                                                                                                                                                                        
    "invocation": {                                                                                                                                                           
        "module_args": {                                                                                                                                                      
            "allow_downgrade": false,                                                                                                                                         
            "allowerasing": false,                                                                                                                                            
            "autoremove": false,                                                                                                                                              
            "bugfix": false,                                                                                                                                                  
            "cacheonly": false,                                                                                                                                               
            "conf_file": "/var/lib/containers/storage/zfs/graph/cb1058f461f79c362c61ed60738366f0e7af91476d5cc61b06eeb41a5b097498/etc/dnf/dnf.conf",                           
            "disable_excludes": null,                                                                                                                                         
            "disable_gpg_check": true,                                                                                                                                        
            "disable_plugin": [
                "subscription-manager"
            ],
            "disablerepo": [
                "*"
            ],
            "download_dir": null,
            "download_only": false,
            "enable_plugin": [],
            "enablerepo": [
                "baseos"
            ],
            "exclude": [],
            "install_repoquery": true,
            "install_weak_deps": false,
            "installroot": "/var/lib/containers/storage/zfs/graph/cb1058f461f79c362c61ed60738366f0e7af91476d5cc61b06eeb41a5b097498",
            "list": null,
            "lock_timeout": 30,
            "name": [
                "glibc-minimal-langpack",
                "coreutils-single"
            ],
            "nobest": false,
            "releasever": "9",
            "security": false,
            "skip_broken": false,
            "sslverify": true,
            "state": "latest",
            "update_cache": false,
            "update_only": false,
            "validate_certs": true
        }
    },
    "msg": "Failed to install some of the specified packages",
    "rc": 1,
    "results": []
}

### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct

[ansibot]

Files identified in the description:

• lib/ansible/modules/dnf.py

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

[bcoca]

note: installroot might not be a current feature of the module, this is probably a RFE

[mskarbek]

@bcoca installroot is a current feature: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/dnf_module.html#parameter-installroot

[mskarbek]

Use case example: https://github.com/mskarbek/containers/blob/main/micro/tasks/rhel.yaml

[mkrizek]

This is what I did in an attempt to reproduce the issue:

% podman run --name container --rm -it --entrypoint bash almalinux:9

inside the container:

mkdir -p /tmp/installroot/etc/yum.repos.d/
cp /etc/yum.repos.d/almalinux-baseos.repo /tmp/installroot/etc/yum.repos.d/

Running the following playbook:

- hosts: container
  gather_facts: false
  tasks:                                                                                           
    - dnf:
        name:
          - "glibc-minimal-langpack"
          - "coreutils-single"
        disablerepo: "*"
        enablerepo: "baseos"
        disable_plugin:
          - "subscription-manager"
        install_weak_deps: false
        releasever: 9
        state: "latest"
        disable_gpg_check: true
        installroot: /tmp/installroot

produces the following output:

changed: [container] => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": true,
    "invocation": {
        "module_args": {
            "allow_downgrade": false,
            "allowerasing": false,
            "autoremove": false,
            "bugfix": false,
            "cacheonly": false,
            "conf_file": null,
            "disable_excludes": null,
            "disable_gpg_check": true,
            "disable_plugin": [
                "subscription-manager"
            ],
            "disablerepo": [
                "*"
            ],
            "download_dir": null,
            "download_only": false,
            "enable_plugin": [],
            "enablerepo": [
                "baseos"
            ],
            "exclude": [],
            "install_repoquery": true,
            "install_weak_deps": false,
            "installroot": "/tmp/installroot",
            "list": null,
            "lock_timeout": 30,
            "name": [
                "glibc-minimal-langpack",
                "coreutils-single"
            ],
            "nobest": false,
            "releasever": "9",
            "security": false,
            "skip_broken": false,
            "sslverify": true,
            "state": "latest",
            "update_cache": false,
            "update_only": false,
            "use_backend": "auto",
            "validate_certs": true
        }
    },
    "msg": "",
    "rc": 0,
    "results": [
        "Installed: glibc-minimal-langpack-2.34-60.el9.x86_64",
        "Installed: tzdata-2023c-1.el9.noarch",
        "Installed: almalinux-gpg-keys-9.2-1.el9.x86_64",
        "Installed: almalinux-release-9.2-1.el9.x86_64",
        "Installed: almalinux-repos-9.2-1.el9.x86_64",
        "Installed: libacl-2.3.1-3.el9.x86_64",
        "Installed: pcre2-10.40-2.el9.x86_64",
        "Installed: pcre2-syntax-10.40-2.el9.noarch",
        "Installed: libattr-2.5.1-3.el9.x86_64",
        "Installed: filesystem-3.16-2.el9.x86_64",
        "Installed: basesystem-11-13.el9.noarch",
        "Installed: bash-5.1.8-6.el9_1.x86_64",
        "Installed: libcap-2.48-8.el9.x86_64",
        "Installed: setup-2.13.7-9.el9.noarch",
        "Installed: glibc-2.34-60.el9.x86_64",
        "Installed: glibc-common-2.34-60.el9.x86_64",
        "Installed: coreutils-single-8.32-34.el9.x86_64",
        "Installed: libselinux-3.5-1.el9.x86_64",
        "Installed: ncurses-base-6.2-8.20210508.el9.noarch",
        "Installed: ncurses-libs-6.2-8.20210508.el9.x86_64",
        "Installed: libsepol-3.5-1.el9.x86_64",
        "Installed: libgcc-11.3.1-4.3.el9.alma.x86_64"
    ]
}

So I was not able to replicate the issue. To move this issue forward we would need a minimal self-contained reproducer to replicate the issue in order to debug it.

needs_info

[mskarbek]

@mkrizek you will replicate this issue only when the repo file is in installroot path but NOT in /etc/yum.repos.d.
After cp /etc/yum.repos.d/almalinux-baseos.repo /tmp/installroot/etc/yum.repos.d/ remove almalinux-baseos.repo from /etc/yum.repos.d.

[mkrizek]

Then a simple reproducer would be:

# podman run --name container --rm -it --entrypoint bash almalinux:9
- hosts: container
  gather_facts: false
  vars:
    installroot_path: /tmp/installroot
  tasks:                                                                                                                                              
    - shell: "mkdir -p {{ installroot_path}}/etc/yum.repos.d/ && mv /etc/yum.repos.d/almalinux-baseos.repo {{ installroot_path }}/etc/yum.repos.d/"

    - dnf:
        name: coreutils-single
        releasever: 9
        installroot: "{{ installroot_path }}"

This would require more testing but this appears to be a fix:

diff --git a/lib/ansible/modules/dnf.py b/lib/ansible/modules/dnf.py
index 3bfca3cd715..02e114ba2f7 100644
--- a/lib/ansible/modules/dnf.py
+++ b/lib/ansible/modules/dnf.py
@@ -652,6 +652,8 @@ class DnfModule(YumDnf):
         # Load substitutions from the filesystem
         conf.substitutions.update_from_etc(installroot)
 
+        conf._search_inside_installroot('reposdir')
+
         # Handle different DNF versions immutable mutable datatypes and
         # dnf v1/v2/v3
         #

Note that the _search_inside_installroot method potentially prepends an installroot path to all repository directories as a side-effect. The method appears to be a private helper, but the public API method conf.prepend_installroot does not seem to handle multiple directories being a value of reposdir so we would have to replicate the behavior of _search_inside_installroot.