What happened?
We manage our NVIDIA driver versions on Ubuntu 24.04 with salt and pin some NVIDIA driver related packages, for example libnvidia-cfg1-570-server, or libnvidia-container1 (quite a few other as well). We hold these packages in apt to prevent accidental updates.
Our salt state does more or less this, just for more packages:
nvidia_driver_bundle:
pkg.installed:
- pkgs:
- "libnvidia-container1:amd64"
- "libnvidia-cfg1-570-server"
- "libnvidia-nscq-570"
- update_holds: True
nvidia_hold_driver:
pkg.held:
- pkgs:
- "libnvidia-container1:amd64"
- "libnvidia-cfg1-570-server"
- "libnvidia-nscq-570"
- require:
- pkg: nvidia_driver_bundle
However, salt complains when applying this state with:
ID: nvidia_driver_bundle
Function: pkg.installed
Result: False
Comment: Problem encountered installing package(s). Additional info follows:
errors:
- Running as unit: run-r0bbc4ec2649a4d86872ea20b6d897502.scope; invocation ID: e337b69ea5f341c585af53786f3e9502
E: Held packages were changed and -y was used without --allow-change-held-packages.
Started: 13:26:35.502637
Duration: 2186.388 ms
Changes:
----------
ID: nvidia_hold_driver
Function: pkg.held
Result: False
Comment: One or more requisite failed: default.nvidia.driver.nvidia_driver_bundle
Started: 13:26:37.690044
Duration: 0.002 ms
Changes:
In the salt log we can see (showing all packages we install/hold):
Building dependency tree...
Reading state information...
Suggested packages:
nvidia-driver-570-server
The following held packages will be changed:
libnvidia-cfg1-570-server libnvidia-compute-570-server
libnvidia-decode-570-server libnvidia-encode-570-server
libnvidia-extra-570-server
The following packages will be upgraded:
libnvidia-cfg1-570-server libnvidia-compute-570-server
libnvidia-decode-570-server libnvidia-encode-570-server
libnvidia-extra-570-server nvidia-compute-utils-570-server
nvidia-dkms-570-server-open nvidia-firmware-570-server-570.211.01
nvidia-headless-570-server-open nvidia-headless-no-dkms-570-server-open
nvidia-kernel-common-570-server nvidia-kernel-source-570-server-open
nvidia-utils-570-server
The packages still held here are all multi-arch packages.
dpkg shows:
dpkg --get-selections | grep -E 'libnvidia-(cfg1|compute|decode|encode|extra)-570-server|nvidia-kernel-common-570-server|libnvidia-nscq-570|libnvidia-container1'
libnvidia-cfg1-570-server:amd64 hold
libnvidia-compute-570-server:amd64 hold
libnvidia-container1:amd64 hold
libnvidia-decode-570-server:amd64 hold
libnvidia-encode-570-server:amd64 hold
libnvidia-extra-570-server:amd64 hold
libnvidia-nscq-570 hold
nvidia-kernel-common-570-server hold
which I also think already shows the issue: Some of the libnvidia packages are shown with their arch identifier as suffix of the name. If I read the current code correctly, apt compares the raw dpgs name (e.g. libnvidia-cfg1-570-server:amd64) with the normalized package name (libnvidia-cfg1-570-server), so it isn't identified as package to unhold.
Adding a manual unhold state before the nvidia_driver_bundle state "fixes" it, since it correctly compares the names for APT.
I am already working on a PR, it's currently building locally.
Should you need any additional information, please let me know :)
Type of salt install
Official deb
Major version
3006.x
What supported OS are you seeing the problem on? Can select multiple. (If bug appears on an unsupported OS, please open a GitHub Discussion instead)
ubuntu-24.04
salt --versions-report output
salt --versions-report
Salt Version:
Salt: 3006.13
Python Version:
Python: 3.10.17 (main, Jun 9 2025, 20:41:48) [GCC 11.2.0]
Dependency Versions:
cffi: 1.17.1
cherrypy: unknown
cryptography: 42.0.5
dateutil: 2.8.1
docker-py: Not Installed
gitdb: 4.0.12
gitpython: 3.1.44
Jinja2: 3.1.6
libgit2: 1.9.0
looseversion: 1.0.2
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.2
msgpack-pure: Not Installed
mysql-python: Not Installed
packaging: 22.0
pycparser: 2.21
pycrypto: Not Installed
pycryptodome: 3.19.1
pygit2: 1.18.0
python-gnupg: 0.4.8
PyYAML: 6.0.1
PyZMQ: 23.2.0
relenv: 0.19.3
smmap: 5.0.2
timelib: 0.2.4
Tornado: 4.5.3
ZMQ: 4.3.4
System Versions:
dist: ubuntu 24.04.2 noble
locale: utf-8
machine: x86_64
release: 6.8.0-86-generic
system: Linux
version: Ubuntu 24.04.2 noble
What happened?
We manage our NVIDIA driver versions on Ubuntu 24.04 with salt and pin some NVIDIA driver related packages, for example libnvidia-cfg1-570-server, or libnvidia-container1 (quite a few other as well). We hold these packages in apt to prevent accidental updates.
Our salt state does more or less this, just for more packages:
However, salt complains when applying this state with:
In the salt log we can see (showing all packages we install/hold):
The packages still held here are all multi-arch packages.
dpkg shows:
which I also think already shows the issue: Some of the libnvidia packages are shown with their arch identifier as suffix of the name. If I read the current code correctly, apt compares the raw dpgs name (e.g. libnvidia-cfg1-570-server:amd64) with the normalized package name (libnvidia-cfg1-570-server), so it isn't identified as package to unhold.
Adding a manual unhold state before the nvidia_driver_bundle state "fixes" it, since it correctly compares the names for APT.
I am already working on a PR, it's currently building locally.
Should you need any additional information, please let me know :)
Type of salt install
Official deb
Major version
3006.x
What supported OS are you seeing the problem on? Can select multiple. (If bug appears on an unsupported OS, please open a GitHub Discussion instead)
ubuntu-24.04
salt --versions-report output
salt --versions-report Salt Version: Salt: 3006.13 Python Version: Python: 3.10.17 (main, Jun 9 2025, 20:41:48) [GCC 11.2.0] Dependency Versions: cffi: 1.17.1 cherrypy: unknown cryptography: 42.0.5 dateutil: 2.8.1 docker-py: Not Installed gitdb: 4.0.12 gitpython: 3.1.44 Jinja2: 3.1.6 libgit2: 1.9.0 looseversion: 1.0.2 M2Crypto: Not Installed Mako: Not Installed msgpack: 1.0.2 msgpack-pure: Not Installed mysql-python: Not Installed packaging: 22.0 pycparser: 2.21 pycrypto: Not Installed pycryptodome: 3.19.1 pygit2: 1.18.0 python-gnupg: 0.4.8 PyYAML: 6.0.1 PyZMQ: 23.2.0 relenv: 0.19.3 smmap: 5.0.2 timelib: 0.2.4 Tornado: 4.5.3 ZMQ: 4.3.4 System Versions: dist: ubuntu 24.04.2 noble locale: utf-8 machine: x86_64 release: 6.8.0-86-generic system: Linux version: Ubuntu 24.04.2 noble