Skip to content

[Bug]: v3006: updating held arch-dependent packages doesn't work with apt with update_holds: True flag #68932

@TobiPeterG

Description

@TobiPeterG

What happened?

We manage our NVIDIA driver versions on Ubuntu 24.04 with salt and pin some NVIDIA driver related packages, for example libnvidia-cfg1-570-server, or libnvidia-container1 (quite a few other as well). We hold these packages in apt to prevent accidental updates.
Our salt state does more or less this, just for more packages:

nvidia_driver_bundle:
  pkg.installed:
    - pkgs:
      - "libnvidia-container1:amd64"
      - "libnvidia-cfg1-570-server"
      - "libnvidia-nscq-570"
    - update_holds: True

nvidia_hold_driver:
  pkg.held:
    - pkgs:
      - "libnvidia-container1:amd64"
      - "libnvidia-cfg1-570-server"
      - "libnvidia-nscq-570"
    - require:
      - pkg: nvidia_driver_bundle

However, salt complains when applying this state with:

          ID: nvidia_driver_bundle
    Function: pkg.installed
      Result: False
     Comment: Problem encountered installing package(s). Additional info follows:
              
              errors:
                  - Running as unit: run-r0bbc4ec2649a4d86872ea20b6d897502.scope; invocation ID: e337b69ea5f341c585af53786f3e9502
                    E: Held packages were changed and -y was used without --allow-change-held-packages.
     Started: 13:26:35.502637
    Duration: 2186.388 ms
     Changes:   
----------
          ID: nvidia_hold_driver
    Function: pkg.held
      Result: False
     Comment: One or more requisite failed: default.nvidia.driver.nvidia_driver_bundle
     Started: 13:26:37.690044
    Duration: 0.002 ms
     Changes:   

In the salt log we can see (showing all packages we install/hold):

Building dependency tree...
Reading state information...
Suggested packages:
  nvidia-driver-570-server
The following held packages will be changed:
  libnvidia-cfg1-570-server libnvidia-compute-570-server
  libnvidia-decode-570-server libnvidia-encode-570-server
  libnvidia-extra-570-server
The following packages will be upgraded:
  libnvidia-cfg1-570-server libnvidia-compute-570-server
  libnvidia-decode-570-server libnvidia-encode-570-server
  libnvidia-extra-570-server nvidia-compute-utils-570-server
  nvidia-dkms-570-server-open nvidia-firmware-570-server-570.211.01
  nvidia-headless-570-server-open nvidia-headless-no-dkms-570-server-open
  nvidia-kernel-common-570-server nvidia-kernel-source-570-server-open
  nvidia-utils-570-server

The packages still held here are all multi-arch packages.

dpkg shows:

dpkg --get-selections | grep -E 'libnvidia-(cfg1|compute|decode|encode|extra)-570-server|nvidia-kernel-common-570-server|libnvidia-nscq-570|libnvidia-container1'
libnvidia-cfg1-570-server:amd64                 hold
libnvidia-compute-570-server:amd64              hold
libnvidia-container1:amd64                      hold
libnvidia-decode-570-server:amd64               hold
libnvidia-encode-570-server:amd64               hold
libnvidia-extra-570-server:amd64                hold
libnvidia-nscq-570                              hold
nvidia-kernel-common-570-server                 hold

which I also think already shows the issue: Some of the libnvidia packages are shown with their arch identifier as suffix of the name. If I read the current code correctly, apt compares the raw dpgs name (e.g. libnvidia-cfg1-570-server:amd64) with the normalized package name (libnvidia-cfg1-570-server), so it isn't identified as package to unhold.

Adding a manual unhold state before the nvidia_driver_bundle state "fixes" it, since it correctly compares the names for APT.

I am already working on a PR, it's currently building locally.

Should you need any additional information, please let me know :)

Type of salt install

Official deb

Major version

3006.x

What supported OS are you seeing the problem on? Can select multiple. (If bug appears on an unsupported OS, please open a GitHub Discussion instead)

ubuntu-24.04

salt --versions-report output

salt --versions-report
Salt Version:
          Salt: 3006.13
 
Python Version:
        Python: 3.10.17 (main, Jun  9 2025, 20:41:48) [GCC 11.2.0]
 
Dependency Versions:
          cffi: 1.17.1
      cherrypy: unknown
  cryptography: 42.0.5
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: 4.0.12
     gitpython: 3.1.44
        Jinja2: 3.1.6
       libgit2: 1.9.0
  looseversion: 1.0.2
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 22.0
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.19.1
        pygit2: 1.18.0
  python-gnupg: 0.4.8
        PyYAML: 6.0.1
         PyZMQ: 23.2.0
        relenv: 0.19.3
         smmap: 5.0.2
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: ubuntu 24.04.2 noble
        locale: utf-8
       machine: x86_64
       release: 6.8.0-86-generic
        system: Linux
       version: Ubuntu 24.04.2 noble

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugbroken, incorrect, or confusing behaviorneeds-triage

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions