Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failed to install packages when cloud-init installs gpg to dearmor key #5223

Closed
AKorezin opened this issue Apr 27, 2024 · 5 comments · Fixed by #5230
Closed

Failed to install packages when cloud-init installs gpg to dearmor key #5223

AKorezin opened this issue Apr 27, 2024 · 5 comments · Fixed by #5230
Assignees
@holmanb
Labels
bug Something isn't working correctly
Milestone
cloud-init-24.2

Comments

@AKorezin
Copy link

Bug report

If gpg is missing and cloud-init tries to install it to dearmor gpg public key, it won't update apt cache, thus breaking package installation.

I noticed this problem while testing cloud images of ubuntu:24.04. It doesn't have gpg by default, so cloud-init installs it. After adding all sources, it tries to call apt-get update, but helpers.py[DEBUG]: update-sources already ran (freq=once-per-instance).

The problem could be reproduced on ubuntu:22.04 by removing gpg using bootcmd.

Steps to reproduce the problem

user-data 
#cloud-config

apt:
  sources:
    docker:
      source: deb [signed-by=$KEY_FILE] https://download.docker.com/linux/ubuntu $RELEASE stable
      key: |
        -----BEGIN PGP PUBLIC KEY BLOCK-----

        mQINBFit2ioBEADhWpZ8/wvZ6hUTiXOwQHXMAlaFHcPH9hAtr4F1y2+OYdbtMuth
        lqqwp028AqyY+PRfVMtSYMbjuQuu5byyKR01BbqYhuS3jtqQmljZ/bJvXqnmiVXh
        38UuLa+z077PxyxQhu5BbqntTPQMfiyqEiU+BKbq2WmANUKQf+1AmZY/IruOXbnq
        L4C1+gJ8vfmXQt99npCaxEjaNRVYfOS8QcixNzHUYnb6emjlANyEVlZzeqo7XKl7
        UrwV5inawTSzWNvtjEjj4nJL8NsLwscpLPQUhTQ+7BbQXAwAmeHCUTQIvvWXqw0N
        cmhh4HgeQscQHYgOJjjDVfoY5MucvglbIgCqfzAHW9jxmRL4qbMZj+b1XoePEtht
        ku4bIQN1X5P07fNWzlgaRL5Z4POXDDZTlIQ/El58j9kp4bnWRCJW0lya+f8ocodo
        vZZ+Doi+fy4D5ZGrL4XEcIQP/Lv5uFyf+kQtl/94VFYVJOleAv8W92KdgDkhTcTD
        G7c0tIkVEKNUq48b3aQ64NOZQW7fVjfoKwEZdOqPE72Pa45jrZzvUFxSpdiNk2tZ
        XYukHjlxxEgBdC/J3cMMNRE1F4NCA3ApfV1Y7/hTeOnmDuDYwr9/obA8t016Yljj
        q5rdkywPf4JF8mXUW5eCN1vAFHxeg9ZWemhBtQmGxXnw9M+z6hWwc6ahmwARAQAB
        tCtEb2NrZXIgUmVsZWFzZSAoQ0UgZGViKSA8ZG9ja2VyQGRvY2tlci5jb20+iQI3
        BBMBCgAhBQJYrefAAhsvBQsJCAcDBRUKCQgLBRYCAwEAAh4BAheAAAoJEI2BgDwO
        v82IsskP/iQZo68flDQmNvn8X5XTd6RRaUH33kXYXquT6NkHJciS7E2gTJmqvMqd
        tI4mNYHCSEYxI5qrcYV5YqX9P6+Ko+vozo4nseUQLPH/ATQ4qL0Zok+1jkag3Lgk
        jonyUf9bwtWxFp05HC3GMHPhhcUSexCxQLQvnFWXD2sWLKivHp2fT8QbRGeZ+d3m
        6fqcd5Fu7pxsqm0EUDK5NL+nPIgYhN+auTrhgzhK1CShfGccM/wfRlei9Utz6p9P
        XRKIlWnXtT4qNGZNTN0tR+NLG/6Bqd8OYBaFAUcue/w1VW6JQ2VGYZHnZu9S8LMc
        FYBa5Ig9PxwGQOgq6RDKDbV+PqTQT5EFMeR1mrjckk4DQJjbxeMZbiNMG5kGECA8
        g383P3elhn03WGbEEa4MNc3Z4+7c236QI3xWJfNPdUbXRaAwhy/6rTSFbzwKB0Jm
        ebwzQfwjQY6f55MiI/RqDCyuPj3r3jyVRkK86pQKBAJwFHyqj9KaKXMZjfVnowLh
        9svIGfNbGHpucATqREvUHuQbNnqkCx8VVhtYkhDb9fEP2xBu5VvHbR+3nfVhMut5
        G34Ct5RS7Jt6LIfFdtcn8CaSas/l1HbiGeRgc70X/9aYx/V/CEJv0lIe8gP6uDoW
        FPIZ7d6vH+Vro6xuWEGiuMaiznap2KhZmpkgfupyFmplh0s6knymuQINBFit2ioB
        EADneL9S9m4vhU3blaRjVUUyJ7b/qTjcSylvCH5XUE6R2k+ckEZjfAMZPLpO+/tF
        M2JIJMD4SifKuS3xck9KtZGCufGmcwiLQRzeHF7vJUKrLD5RTkNi23ydvWZgPjtx
        Q+DTT1Zcn7BrQFY6FgnRoUVIxwtdw1bMY/89rsFgS5wwuMESd3Q2RYgb7EOFOpnu
        w6da7WakWf4IhnF5nsNYGDVaIHzpiqCl+uTbf1epCjrOlIzkZ3Z3Yk5CM/TiFzPk
        z2lLz89cpD8U+NtCsfagWWfjd2U3jDapgH+7nQnCEWpROtzaKHG6lA3pXdix5zG8
        eRc6/0IbUSWvfjKxLLPfNeCS2pCL3IeEI5nothEEYdQH6szpLog79xB9dVnJyKJb
        VfxXnseoYqVrRz2VVbUI5Blwm6B40E3eGVfUQWiux54DspyVMMk41Mx7QJ3iynIa
        1N4ZAqVMAEruyXTRTxc9XW0tYhDMA/1GYvz0EmFpm8LzTHA6sFVtPm/ZlNCX6P1X
        zJwrv7DSQKD6GGlBQUX+OeEJ8tTkkf8QTJSPUdh8P8YxDFS5EOGAvhhpMBYD42kQ
        pqXjEC+XcycTvGI7impgv9PDY1RCC1zkBjKPa120rNhv/hkVk/YhuGoajoHyy4h7
        ZQopdcMtpN2dgmhEegny9JCSwxfQmQ0zK0g7m6SHiKMwjwARAQABiQQ+BBgBCAAJ
        BQJYrdoqAhsCAikJEI2BgDwOv82IwV0gBBkBCAAGBQJYrdoqAAoJEH6gqcPyc/zY
        1WAP/2wJ+R0gE6qsce3rjaIz58PJmc8goKrir5hnElWhPgbq7cYIsW5qiFyLhkdp
        YcMmhD9mRiPpQn6Ya2w3e3B8zfIVKipbMBnke/ytZ9M7qHmDCcjoiSmwEXN3wKYI
        mD9VHONsl/CG1rU9Isw1jtB5g1YxuBA7M/m36XN6x2u+NtNMDB9P56yc4gfsZVES
        KA9v+yY2/l45L8d/WUkUi0YXomn6hyBGI7JrBLq0CX37GEYP6O9rrKipfz73XfO7
        JIGzOKZlljb/D9RX/g7nRbCn+3EtH7xnk+TK/50euEKw8SMUg147sJTcpQmv6UzZ
        cM4JgL0HbHVCojV4C/plELwMddALOFeYQzTif6sMRPf+3DSj8frbInjChC3yOLy0
        6br92KFom17EIj2CAcoeq7UPhi2oouYBwPxh5ytdehJkoo+sN7RIWua6P2WSmon5
        U888cSylXC0+ADFdgLX9K2zrDVYUG1vo8CX0vzxFBaHwN6Px26fhIT1/hYUHQR1z
        VfNDcyQmXqkOnZvvoMfz/Q0s9BhFJ/zU6AgQbIZE/hm1spsfgvtsD1frZfygXJ9f
        irP+MSAI80xHSf91qSRZOj4Pl3ZJNbq4yYxv0b1pkMqeGdjdCYhLU+LZ4wbQmpCk
        SVe2prlLureigXtmZfkqevRz7FrIZiu9ky8wnCAPwC7/zmS18rgP/17bOtL4/iIz
        QhxAAoAMWVrGyJivSkjhSGx1uCojsWfsTAm11P7jsruIL61ZzMUVE2aM3Pmj5G+W
        9AcZ58Em+1WsVnAXdUR//bMmhyr8wL/G1YO1V3JEJTRdxsSxdYa4deGBBY/Adpsw
        24jxhOJR+lsJpqIUeb999+R8euDhRHG9eFO7DRu6weatUJ6suupoDTRWtr/4yGqe
        dKxV3qQhNLSnaAzqW/1nA3iUB4k7kCaKZxhdhDbClf9P37qaRW467BLCVO/coL3y
        Vm50dwdrNtKpMBh3ZpbB1uJvgi9mXtyBOMJ3v8RZeDzFiG8HdCtg9RvIt/AIFoHR
        H3S+U79NT6i0KPzLImDfs8T7RlpyuMc4Ufs8ggyg9v3Ae6cN3eQyxcK3w0cbBwsh
        /nQNfsA6uu+9H7NhbehBMhYnpNZyrHzCmzyXkauwRAqoCbGCNykTRwsur9gS41TQ
        M8ssD1jFheOJf3hODnkKU+HKjvMROl1DK7zdmLdNzA1cvtZH/nCC9KPj1z8QC47S
        xx+dTZSx4ONAhwbS/LN3PoKtn8LPjY9NP9uDWI+TWYquS2U+KHDrBDlsgozDbs/O
        jCxcpDzNmXpWQHEtHU7649OXHP7UeNST1mCUCH5qdank0V1iejF6/CfTFU4MfcrG
        YT90qFF93M3v01BbxP+EIY2/9tiIPbrd
        =0YYh
        -----END PGP PUBLIC KEY BLOCK-----

package_update: true

packages:
  - docker-ce

Environment details

  • Cloud-init version: 24.1.3
  • Operating System Distribution: ubuntu:24.04
  • Cloud provider, platform or installer type: nocloud

cloud-init logs

cloud-init.log
@AKorezin AKorezin added bug Something isn't working correctly new An issue that still needs triage labels Apr 27, 2024
holmanb added a commit to holmanb/cloud-init that referenced this issue Apr 29, 2024
@holmanb
fix(apt): Allow opt-in for calling apt update multiple times
55e205a 
This is required to configure apt when dependency is not installed.

Fixes canonicalGH-5223
@holmanb
Copy link
Member

holmanb commented Apr 29, 2024

@AKorezin Thanks for reporting, and great analysis. I don't see how multiple apt update runs can be avoided here, since this install is explicitly for configuring apt.

@holmanb holmanb mentioned this issue Apr 29, 2024
Merged
2 tasks
@holmanb
Copy link
Member

holmanb commented Apr 29, 2024

@AKorezin I have a branch that might should fix this, if you'd like to test it.

holmanb added a commit to holmanb/cloud-init that referenced this issue Apr 29, 2024
@holmanb
fix(apt): Allow opt-in for calling apt update multiple times
734958e 
This is required to configure apt when dependency is not installed.

Fixes canonicalGH-5223
@AKorezin
Copy link
Author

Hello @holmanb.
Maybe I did something wrong, but here's an error log

2024-04-29 19:38:06,506 - handlers.py[DEBUG]: finish: modules-config/config-apt_configure: FAIL: running config-apt_configure with frequency once-per-instance
2024-04-29 19:38:06,506 - util.py[WARNING]: Running module apt_configure (<module 'cloudinit.config.cc_apt_configure' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_apt_configure.py'>) failed
2024-04-29 19:38:06,506 - util.py[DEBUG]: Running module apt_configure (<module 'cloudinit.config.cc_apt_configure' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_apt_configure.py'>) failed
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/cloudinit/config/modules.py", line 286, in _run_modules
    ran, _r = cc.run(
              ^^^^^^^
  File "/usr/lib/python3/dist-packages/cloudinit/cloud.py", line 71, in run
    return self._runners.run(name, functor, args, freq, clear_on_fail)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 156, in run
    results = functor(**args)
              ^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_apt_configure.py", line 231, in handle
    apply_apt(apt_cfg, cloud, gpg_context)
  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_apt_configure.py", line 262, in apply_apt
    _ensure_dependencies(cfg, matcher, cloud)
  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_apt_configure.py", line 750, in _ensure_dependencies
    cloud.distro.install_packages(
TypeError: Distro.install_packages() got an unexpected keyword argument 'allow_subsequent_updates'

I've built Cloud-init v. 23.4-370-g734958e2-1~bddeb from source in a container using the following script:

apt-get update
apt-get install make devscripts python3-yaml python3-jinja2 debhelper dh-python python3-configobj python3-debconf python3-jsonpatch python3-jsonschema python3-oauthlib python3-serial python3-passlib python3-pytest python3-pytest-cov python3-pytest-mock python3-responses python3-setuptools
git clone https://github.com/canonical/cloud-init
cd cloud-init
git fetch origin pull/5230/head:apt
git switch apt
make deb

@holmanb holmanb removed the new An issue that still needs triage label May 1, 2024
holmanb added a commit to holmanb/cloud-init that referenced this issue May 2, 2024
@holmanb
fix(apt): Allow forcibly updating apt
fb4ba98 
This is required to configure apt when a dependency is not installed.

Co-authored-by: James Falcon <james.falcon@canonical.com>

Fixes canonicalGH-5223
@Guillaumebeuzeboc Guillaumebeuzeboc mentioned this issue May 7, 2024
Merged
@AKorezin
Copy link
Author

AKorezin commented May 8, 2024

27a5398 works for me.

@holmanb
Copy link
Member

holmanb commented May 8, 2024

27a5398 works for me.

Thanks for testing! I need to write up a unittest before it can merge. It might be a week or two before that happens, but it should happen soon.

@Guillaumebeuzeboc Guillaumebeuzeboc mentioned this issue May 16, 2024
Merged
@holmanb holmanb added the 24.2 label May 16, 2024
@blackboxsw blackboxsw added this to the cloud-init-24.2 milestone May 27, 2024
@blackboxsw blackboxsw removed the 24.2 label May 27, 2024
@blackboxsw blackboxsw assigned blackboxsw and holmanb and unassigned blackboxsw Jun 4, 2024
holmanb added a commit that referenced this issue Jun 11, 2024
@holmanb @TheRealFalcon
fix(apt): Enable calling apt update multiple times (#5230)
a01b8d3 
This is required to configure apt when dependency is not installed.

Fixes GH-5223

Co-authored-by: James Falcon <james.falcon@canonical.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@holmanb holmanb

Labels
bug Something isn't working correctly
Projects
None yet
Milestone
cloud-init-24.2
Development

Successfully merging a pull request may close this issue.

fix(apt): Allow opt-in for calling apt update multiple times holmanb/cloud-init
3 participants
@AKorezin @blackboxsw @holmanb