Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dnf5: set releasever in the base object's vars #1379

Closed
wants to merge 1 commit into from

Conversation

AdamWill
Copy link
Contributor

With Fedora 40 now branched, openQA installer build tests started failing with an error:

RuntimeError: cannot open file: (2) - No such file or directory [/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$releasever-x86_64]

it seems like setting releasever via set_substitutions just does not fully work as expected, and libdnf goes looking for a file with a literal $releasever in it based on the gpgkey line in the repo file. Though it seems like the releasever substitution must work to some extent because we at least manage to resolve and download packages before the failure.

This seems to fix it, anyhow, and is based on code from libdnf5 itself in dnf5/main.cpp RootCommand::set_argument_parser() .

With Fedora 40 now branched, openQA installer build tests started
failing with an error:

  RuntimeError: cannot open file: (2) - No such file or directory [/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$releasever-x86_64]

it seems like setting releasever via set_substitutions just does
not fully work as expected, and libdnf goes looking for a file
with a literal `$releasever` in it based on the gpgkey line in
the repo file. Though it seems like the releasever substitution
must work to *some* extent because we at least manage to resolve
and download packages before the failure.

This seems to fix it, anyhow, and is based on code from libdnf5
itself in dnf5/main.cpp RootCommand::set_argument_parser() .

Signed-off-by: Adam Williamson <awilliam@redhat.com>
@AdamWill
Copy link
Contributor Author

I really don't know all the details behind what's going on here, or if this is the correct long-term fix. But it at least fixes it for now and I need to get the openQA tests working again, so I'm going to do a downstream build with this patch, and I submitted this PR so we have everything nicely referenced.

I kinda suspect this may ultimately be a libdnf bug? It seems like setting this via set_substitutions really ought to work, though of course without proper API documentation it's hard to be sure.

@AdamWill
Copy link
Contributor Author

AdamWill commented Feb 14, 2024

Oh, forgot to note - this happened only after Branching because Rawhide repo config files don't use $releasever. Releases before F40 aren't affected because they use older lorax versions which don't use libdnf5. The official compose wasn't affected either, I think because in the official composes we don't use the regular repo definitions at all, we use completely custom repo definitions that point to a temporary location that's part of the compose process, and which probably also don't use $releasever.

@coveralls
Copy link

Pull Request Test Coverage Report for Build 7898100212

Details

  • 0 of 1 (100.0%) changed or added relevant line in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.01%) to 43.554%

Totals Coverage Status
Change from base Build 7808681794: 0.01%
Covered Lines: 1615
Relevant Lines: 3501

💛 - Coveralls

@AdamWill
Copy link
Contributor Author

Full lorax.log for a failure, for the record:

2024-02-14 01:05:28,337 DEBUG lorax: Namespace(product='Fedora', version='40', release='40', source=[], repos=['/etc/yum.repos.d/fedora.repo', '/etc/yum.repos.d/workarounds.repo', '/etc/yum.repos.d/advisory.repo'], mirrorlist=[], variant='', bugurl='your distribution provided bug reporting tool', isfinal=False, config='/etc/lorax/lorax.conf', proxy=None, installpkgs=[], excludepkgs=[], buildarch=None, volid=None, domacboot=True, doupgrade=True, logfile='/root/imgbuild/lorax.log', tmp='/var/tmp/lorax', cachedir=None, workdir=None, force=False, add_templates=[], add_template_vars=[], add_arch_templates=[], add_arch_template_vars=[], verify=True, sharedir=None, enablerepos=[], disablerepos=[], rootfs_size=3, noverifyssl=False, dnfplugins=[], squashfs_only=True, skip_branding=False, dracut_conf=None, dracut_args=None, outputdir='/root/imgbuild/results')
2024-02-14 01:05:28,337 INFO pylorax: selinux is enabled and in Permissive mode
2024-02-14 01:05:28,348 INFO pylorax: Using platform:f40 for module_platform_id
2024-02-14 01:05:28,402 INFO pylorax: Using repos: workarounds, fedora, advisory
2024-02-14 01:05:28,404 INFO pylorax: Fetching metadata...
2024-02-14 01:06:18,416 DEBUG pylorax: version is 40.5-1
2024-02-14 01:06:18,416 INFO pylorax: selinux is enabled and in Permissive mode
2024-02-14 01:06:18,420 DEBUG pylorax: using work directory /var/tmp/lorax/lorax.d0a9dil7
2024-02-14 01:06:18,420 DEBUG pylorax: using log directory /root/imgbuild
2024-02-14 01:06:18,420 DEBUG pylorax: using output directory /root/imgbuild/results
2024-02-14 01:06:18,420 INFO pylorax: checking for root privileges
2024-02-14 01:06:18,421 INFO pylorax: checking dnf base object
2024-02-14 01:06:18,422 DEBUG pylorax: using install root: /var/tmp/lorax/lorax.d0a9dil7/installtree
2024-02-14 01:06:18,454 INFO pylorax: setting up build architecture
2024-02-14 01:06:18,456 DEBUG pylorax: self.arch.buildarch = x86_64
2024-02-14 01:06:18,456 DEBUG pylorax: self.arch.basearch = x86_64
2024-02-14 01:06:18,456 DEBUG pylorax: self.arch.libdir = lib64
2024-02-14 01:06:18,456 INFO pylorax: setting up build parameters
2024-02-14 01:06:18,457 DEBUG pylorax: product data: {'name': 'Fedora', 'version': '40', 'release': '40', 'variant': '', 'bugurl': 'your distribution provided bug reporting tool', 'isfinal': False}
2024-02-14 01:06:18,457 INFO pylorax: Using templatedir /usr/share/lorax/templates.d/99-generic
2024-02-14 01:06:41,372 DEBUG pylorax.treebuilder: system-release packages: fedora-release,fedora-release-budgie,fedora-release-cinnamon,fedora-release-cloud,fedora-release-compneuro,fedora-release-container,fedora-release-coreos,fedora-release-designsuite,fedora-release-i3,fedora-release-iot,fedora-release-kde,fedora-release-kinoite,fedora-release-lxqt,fedora-release-matecompiz,fedora-release-mobility,fedora-release-onyx,fedora-release-sericea,fedora-release-server,fedora-release-silverblue,fedora-release-snappy,fedora-release-soas,fedora-release-sway,fedora-release-toolbx,fedora-release-workstation,fedora-release-xfce
2024-02-14 01:06:41,373 INFO pylorax.treebuilder: got release: fedora-release
2024-02-14 01:06:41,377 INFO pylorax: installing runtime packages
2024-02-14 01:06:41,412 DEBUG pylorax.ltmpl: executing runtime-install.tmpl with variables={'arch': {'buildarch': 'x86_64', 'basearch': 'x86_64', 'libdir': 'lib64', 'bcj': 'x86'}, 'product': {'name': 'fedora', 'version': '40', 'release': '40', 'variant': '', 'bugurl': 'your distribution provided bug reporting tool', 'isfinal': False}, 'dbo': <libdnf5.base.Base; proxy of <Swig Object of type 'libdnf5::Base *' at 0x7fec8438ff90> >, 'root': '/var/tmp/lorax/lorax.d0a9dil7/installtree', 'basearch': 'x86_64', 'libdir': 'lib64', 'branding': {'release': 'fedora-release', 'logos': 'fedora-logos'}, 'exists': <function LoraxTemplateRunner.__init__.<locals>.<lambda> at 0x7fec843b9c60>, 'glob': <function LoraxTemplateRunner.__init__.<locals>.<lambda> at 0x7fec843b9da0>}
2024-02-14 01:06:41,443 INFO pylorax.ltmpl: running runtime-install.tmpl
2024-02-14 01:06:41,446 DEBUG pylorax.ltmpl: template line 1: installpkg anaconda anaconda-widgets
2024-02-14 01:06:41,446 DEBUG pylorax.ltmpl: template line 2: installpkg anaconda-install-img-deps>=40.15
2024-02-14 01:06:41,454 INFO pylorax.ltmpl: installpkg: anaconda-install-img-deps>=40.15 expands to anaconda-install-img-deps-40.21-2.fc40.x86_64
2024-02-14 01:06:41,455 DEBUG pylorax.ltmpl: template line 3: installpkg dnf
2024-02-14 01:06:41,455 DEBUG pylorax.ltmpl: template line 4: installpkg rpm-ostree ostree
2024-02-14 01:06:41,456 DEBUG pylorax.ltmpl: template line 5: installpkg pigz
2024-02-14 01:06:41,456 DEBUG pylorax.ltmpl: template line 6: installpkg kernel kernel-modules kernel-modules-extra
2024-02-14 01:06:41,456 DEBUG pylorax.ltmpl: template line 7: installpkg grubby
2024-02-14 01:06:41,457 DEBUG pylorax.ltmpl: template line 8: installpkg --optional *-firmware --except alsa* --except midisport-firmware --except crystalhd-firmware --except ivtv-firmware --except cx18-firmware --except iscan-firmware --except uhd-firmware --except lulzbot-marlin-firmware --except gnome-firmware --except sigrok-firmware --except liquidio-firmware --except netronome-firmware --except mrvlprestera-firmware --except mlxsw_spectrum-firmware --except hackrf-firmware --except python-virt-firmware --except python3-virt-firmware --except iwl3945-firmware --except iwl4965-firmware --except iwl100-firmware --except iwl105-firmware --except iwl135-firmware --except iwl1000-firmware --except iwl2000-firmware --except iwl2030-firmware --except iwl5000-firmware --except iwl5150-firmware --except iwl6000-firmware --except iwl6000g2a-firmware --except iwl6000g2b-firmware --except iwl6050-firmware --except iwl3160-firmware --except iwl7260-firmware --except iwlax2xx-firmware --except libertas-sd8686-firmware --except libertas-sd8787-firmware --except libertas-usb8388-firmware --except libertas-usb8388-olpc-firmware --except qcom-firmware
2024-02-14 01:06:41,481 INFO pylorax.ltmpl: installpkg: *-firmware expands to amd-gpu-firmware-20240115-2.fc40.noarch,amd-ucode-firmware-20240115-2.fc40.noarch,atheros-firmware-20240115-2.fc40.noarch,atmel-firmware-1.3-32.fc40.noarch,brcmfmac-firmware-20240115-2.fc40.noarch,cirrus-audio-firmware-20240115-2.fc40.noarch,dvb-firmware-20240115-2.fc40.noarch,intel-audio-firmware-20240115-2.fc40.noarch,intel-gpu-firmware-20240115-2.fc40.noarch,intel-vsc-firmware-20240115-2.fc40.noarch,iwlegacy-firmware-20240115-2.fc40.noarch,iwlwifi-dvm-firmware-20240115-2.fc40.noarch,iwlwifi-mvm-firmware-20240115-2.fc40.noarch,libertas-firmware-20240115-2.fc40.noarch,linux-firmware-20240115-2.fc40.noarch,mt7xxx-firmware-20240115-2.fc40.noarch,nvidia-gpu-firmware-20240115-2.fc40.noarch,nxpwireless-firmware-20240115-2.fc40.noarch,realtek-firmware-20240115-2.fc40.noarch,tiwilink-firmware-20240115-2.fc40.noarch,zd1211-firmware-1.5-15.fc40.noarch
2024-02-14 01:06:41,487 DEBUG pylorax.ltmpl: template line 9: installpkg b43-openfwwf
2024-02-14 01:06:41,487 DEBUG pylorax.ltmpl: template line 10: installpkg glibc-all-langpacks
2024-02-14 01:06:41,488 DEBUG pylorax.ltmpl: template line 11: installpkg grub2-tools-efi>=1:2.06-67
2024-02-14 01:06:41,488 INFO pylorax.ltmpl: installpkg: grub2-tools-efi>=1:2.06-67 expands to grub2-tools-efi-1:2.06-118.fc40.x86_64
2024-02-14 01:06:41,488 DEBUG pylorax.ltmpl: template line 12: installpkg efibootmgr
2024-02-14 01:06:41,489 DEBUG pylorax.ltmpl: template line 13: installpkg shim-x64
2024-02-14 01:06:41,489 DEBUG pylorax.ltmpl: template line 14: installpkg grub2-efi-x64-cdboot>=1:2.06-67
2024-02-14 01:06:41,489 INFO pylorax.ltmpl: installpkg: grub2-efi-x64-cdboot>=1:2.06-67 expands to grub2-efi-x64-cdboot-1:2.06-118.fc40.x86_64
2024-02-14 01:06:41,491 DEBUG pylorax.ltmpl: template line 15: installpkg shim-ia32
2024-02-14 01:06:41,491 DEBUG pylorax.ltmpl: template line 16: installpkg grub2-efi-ia32-cdboot>=1:2.06-67
2024-02-14 01:06:41,491 INFO pylorax.ltmpl: installpkg: grub2-efi-ia32-cdboot>=1:2.06-67 expands to grub2-efi-ia32-cdboot-1:2.06-118.fc40.x86_64
2024-02-14 01:06:41,492 DEBUG pylorax.ltmpl: template line 17: installpkg biosdevname
2024-02-14 01:06:41,492 DEBUG pylorax.ltmpl: template line 18: installpkg grub2-tools>=1:2.06-67 grub2-tools-minimal>=1:2.06-67
2024-02-14 01:06:41,492 INFO pylorax.ltmpl: installpkg: grub2-tools>=1:2.06-67 expands to grub2-tools-1:2.06-118.fc40.x86_64
2024-02-14 01:06:41,493 INFO pylorax.ltmpl: installpkg: grub2-tools-minimal>=1:2.06-67 expands to grub2-tools-minimal-1:2.06-118.fc40.x86_64
2024-02-14 01:06:41,494 DEBUG pylorax.ltmpl: template line 19: installpkg grub2-tools-extra>=1:2.06-67
2024-02-14 01:06:41,494 INFO pylorax.ltmpl: installpkg: grub2-tools-extra>=1:2.06-67 expands to grub2-tools-extra-1:2.06-118.fc40.x86_64
2024-02-14 01:06:41,495 DEBUG pylorax.ltmpl: template line 20: installpkg grub2-pc-modules>=1:2.06-67
2024-02-14 01:06:41,495 INFO pylorax.ltmpl: installpkg: grub2-pc-modules>=1:2.06-67 expands to grub2-pc-modules-1:2.06-118.fc40.noarch
2024-02-14 01:06:41,496 DEBUG pylorax.ltmpl: template line 21: installpkg plymouth
2024-02-14 01:06:41,496 DEBUG pylorax.ltmpl: template line 22: installpkg anaconda-dracut dracut-network dracut-config-generic
2024-02-14 01:06:41,496 DEBUG pylorax.ltmpl: template line 23: installpkg initscripts
2024-02-14 01:06:41,497 DEBUG pylorax.ltmpl: template line 24: installpkg cryptsetup
2024-02-14 01:06:41,497 DEBUG pylorax.ltmpl: template line 25: installpkg rpcbind
2024-02-14 01:06:41,497 DEBUG pylorax.ltmpl: template line 26: installpkg kbd kbd-misc
2024-02-14 01:06:41,497 DEBUG pylorax.ltmpl: template line 27: installpkg tar xz curl bzip2
2024-02-14 01:06:41,498 DEBUG pylorax.ltmpl: template line 28: installpkg rsyslog
2024-02-14 01:06:41,498 DEBUG pylorax.ltmpl: template line 29: installpkg device-mapper-persistent-data
2024-02-14 01:06:41,498 DEBUG pylorax.ltmpl: template line 30: installpkg xfsdump
2024-02-14 01:06:41,499 DEBUG pylorax.ltmpl: template line 31: installpkg volume_key
2024-02-14 01:06:41,499 DEBUG pylorax.ltmpl: template line 32: installpkg nss-tools
2024-02-14 01:06:41,499 DEBUG pylorax.ltmpl: template line 33: installpkg selinux-policy-targeted audit
2024-02-14 01:06:41,500 DEBUG pylorax.ltmpl: template line 34: installpkg ethtool openssh-server nfs-utils openssh-clients
2024-02-14 01:06:41,500 DEBUG pylorax.ltmpl: template line 35: installpkg tigervnc-server-minimal
2024-02-14 01:06:41,500 DEBUG pylorax.ltmpl: template line 36: installpkg tigervnc-server-module
2024-02-14 01:06:41,501 DEBUG pylorax.ltmpl: template line 37: installpkg net-tools
2024-02-14 01:06:41,501 DEBUG pylorax.ltmpl: template line 38: installpkg bridge-utils
2024-02-14 01:06:41,501 DEBUG pylorax.ltmpl: template line 39: installpkg nmap-ncat
2024-02-14 01:06:41,501 DEBUG pylorax.ltmpl: template line 40: installpkg pciutils usbutils ipmitool
2024-02-14 01:06:41,502 DEBUG pylorax.ltmpl: template line 41: installpkg mt-st smartmontools
2024-02-14 01:06:41,502 DEBUG pylorax.ltmpl: template line 42: installpkg hdparm
2024-02-14 01:06:41,502 DEBUG pylorax.ltmpl: template line 43: installpkg rdma-core
2024-02-14 01:06:41,503 DEBUG pylorax.ltmpl: template line 44: installpkg rng-tools
2024-02-14 01:06:41,503 DEBUG pylorax.ltmpl: template line 45: installpkg dmidecode
2024-02-14 01:06:41,503 DEBUG pylorax.ltmpl: template line 46: installpkg nvme-cli
2024-02-14 01:06:41,503 DEBUG pylorax.ltmpl: template line 47: installpkg default-fonts-core-sans
2024-02-14 01:06:41,503 DEBUG pylorax.ltmpl: template line 48: installpkg default-fonts-other-sans
2024-02-14 01:06:41,504 DEBUG pylorax.ltmpl: template line 49: installpkg google-noto-sans-cjk-fonts
2024-02-14 01:06:41,504 DEBUG pylorax.ltmpl: template line 50: installpkg gdb-gdbserver
2024-02-14 01:06:41,504 DEBUG pylorax.ltmpl: template line 51: installpkg libreport-plugin-bugzilla libreport-plugin-reportuploader
2024-02-14 01:06:41,504 DEBUG pylorax.ltmpl: template line 52: installpkg fpaste
2024-02-14 01:06:41,504 DEBUG pylorax.ltmpl: template line 53: installpkg python3-pyatspi
2024-02-14 01:06:41,505 DEBUG pylorax.ltmpl: template line 54: installpkg nano nano-default-editor
2024-02-14 01:06:41,505 DEBUG pylorax.ltmpl: template line 55: installpkg vim-minimal strace lsof dump xz less
2024-02-14 01:06:41,506 DEBUG pylorax.ltmpl: template line 56: installpkg wget2-wget rsync bind-utils ftp mtr vconfig
2024-02-14 01:06:41,507 DEBUG pylorax.ltmpl: template line 57: installpkg spice-vdagent
2024-02-14 01:06:41,507 DEBUG pylorax.ltmpl: template line 58: installpkg gdisk hexedit sg3_utils
2024-02-14 01:06:41,507 DEBUG pylorax.ltmpl: template line 59: run_pkg_transaction
2024-02-14 01:06:41,507 INFO pylorax.ltmpl: Checking dependencies
2024-02-14 01:07:33,946 INFO pylorax.ltmpl: 818 packages selected
2024-02-14 01:07:33,996 INFO pylorax.ltmpl: Downloading packages
2024-02-14 01:07:47,857 INFO pylorax.ltmpl: Preparing transaction from installation source
2024-02-14 01:07:47,862 WARNING pylorax.imgutils: Making missing mount directory: /proc
2024-02-14 01:07:47,865 WARNING pylorax.imgutils: Making missing mount directory: /dev
2024-02-14 01:07:47,941 ERROR pylorax.ltmpl: The transaction process has ended abruptly: cannot open file: (2) - No such file or directory [/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$releasever-x86_64]
2024-02-14 01:07:47,999 ERROR pylorax.ltmpl: template command error in runtime-install.tmpl:
2024-02-14 01:07:48,003 ERROR pylorax.ltmpl:   run_pkg_transaction
2024-02-14 01:07:48,009 ERROR pylorax.ltmpl:   RuntimeError: cannot open file: (2) - No such file or directory [/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$releasever-x86_64]
2024-02-14 01:07:48,012 DEBUG pylorax.ltmpl:   Traceback (most recent call last):
2024-02-14 01:07:48,012 DEBUG pylorax.ltmpl:     File "/usr/lib/python3.12/site-packages/pylorax/ltmpl.py", line 826, in run_pkg_transaction
2024-02-14 01:07:48,013 DEBUG pylorax.ltmpl:       result = self.transaction.run()
2024-02-14 01:07:48,013 DEBUG pylorax.ltmpl:                ^^^^^^^^^^^^^^^^^^^^^^
2024-02-14 01:07:48,013 DEBUG pylorax.ltmpl:     File "/usr/lib64/python3.12/site-packages/libdnf5/base.py", line 725, in run
2024-02-14 01:07:48,013 DEBUG pylorax.ltmpl:       return _base.Transaction_run(self)
2024-02-14 01:07:48,014 DEBUG pylorax.ltmpl:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-14 01:07:48,014 DEBUG pylorax.ltmpl:   RuntimeError: cannot open file: (2) - No such file or directory [/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$releasever-x86_64]
2024-02-14 01:07:48,057 INFO lorax: Cleaning up tempdir - /var/tmp/lorax/lorax.d0a9dil7

@bcl
Copy link
Contributor

bcl commented Feb 16, 2024

Fixed in #1380

@bcl bcl closed this Feb 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants