Add nvme support

[holmanb]

Context

Fixes #5246

Also downgrade a warning log

[TheRealFalcon]

I'm still not entirely sure I understand the problem. I created a Noble VM on QEMU using the same version as the bug and used the reproducer as my cloud-config (plus adding my ssh key).

lsblk after cloud-init ran:

ubuntu@originalhostname:~$ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
fd0           2:0    1     4K  0 disk 
sda           8:0    0    10G  0 disk 
├─sda1        8:1    0     9G  0 part /
├─sda14       8:14   0     4M  0 part 
├─sda15       8:15   0   106M  0 part /boot/efi
└─sda16     259:0    0   913M  0 part /boot
sr0          11:0    1  1024M  0 rom  
vda         253:0    0   366K  0 disk 
nvme0n1     259:1    0     4G  0 disk 
├─nvme0n1p1 259:2    0 204.8M  0 part 
└─nvme0n1p2 259:4    0   3.8G  0 part

Logs:

2024-05-09 15:39:21,767 - modules.py[DEBUG]: Running module disk_setup (<module 'cloudinit.config.cc_disk_setup' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_disk_setup.py'>) with frequency once-per-instance
2024-05-09 15:39:21,767 - handlers.py[DEBUG]: start: init-network/config-disk_setup: running config-disk_setup with frequency once-per-instance
2024-05-09 15:39:21,767 - util.py[DEBUG]: Writing to /var/lib/cloud/instances/myid/sem/config_disk_setup - wb: [644] 23 bytes
2024-05-09 15:39:21,767 - helpers.py[DEBUG]: Running config-disk_setup using lock (<FileLock using file '/var/lib/cloud/instances/myid/sem/config_disk_setup'>)
2024-05-09 15:39:21,767 - cc_disk_setup.py[DEBUG]: Partitioning disks: {'/dev/nvme0n1': {'layout': [[5, 82], [95, 83]], 'overwrite': True, 'table_type': 'gpt'}}
2024-05-09 15:39:21,767 - cc_disk_setup.py[DEBUG]: Creating new partition table/disk
2024-05-09 15:39:21,767 - subp.py[DEBUG]: Running command ['udevadm', 'settle'] with allowed return codes [0] (shell=False, capture=True)
2024-05-09 15:39:21,768 - cc_disk_setup.py[DEBUG]: Checking values for /dev/nvme0n1 definition
2024-05-09 15:39:21,768 - cc_disk_setup.py[DEBUG]: Checking against default devices
2024-05-09 15:39:21,769 - cc_disk_setup.py[DEBUG]: Checking if device /dev/nvme0n1 is a valid device
2024-05-09 15:39:21,769 - subp.py[DEBUG]: Running command ['/usr/bin/lsblk', '--pairs', '--output', 'NAME,TYPE,FSTYPE,LABEL', '/dev/nvme0n1', '--nodeps'] with allowed return codes [0] (shell=False, capture=True)
2024-05-09 15:39:21,771 - cc_disk_setup.py[DEBUG]: Checking if device layout matches
2024-05-09 15:39:21,771 - subp.py[DEBUG]: Running command ['/usr/sbin/sgdisk', '-p', '/dev/nvme0n1'] with allowed return codes [0] (shell=False, capture=True)
2024-05-09 15:39:21,780 - cc_disk_setup.py[DEBUG]: called check_partition_gpt_layout(/dev/nvme0n1, [[5, 82], [95, 83]]), returned: []
2024-05-09 15:39:21,780 - cc_disk_setup.py[DEBUG]: Checking if device is safe to partition
2024-05-09 15:39:21,780 - cc_disk_setup.py[DEBUG]: Checking for device size of /dev/nvme0n1
2024-05-09 15:39:21,780 - subp.py[DEBUG]: Running command ['/usr/sbin/blockdev', '--getsize64', '/dev/nvme0n1'] with allowed return codes [0] (shell=False, capture=True)
2024-05-09 15:39:21,781 - subp.py[DEBUG]: Running command ['/usr/sbin/blockdev', '--getss', '/dev/nvme0n1'] with allowed return codes [0] (shell=False, capture=True)
2024-05-09 15:39:21,782 - cc_disk_setup.py[DEBUG]: Calculating partition layout
2024-05-09 15:39:21,782 - cc_disk_setup.py[DEBUG]:    Layout is: [(82, [0, '+419430']), (83, [0, 0])]
2024-05-09 15:39:21,782 - cc_disk_setup.py[DEBUG]: Creating partition table on /dev/nvme0n1
2024-05-09 15:39:21,782 - subp.py[DEBUG]: Running command ['/usr/sbin/sgdisk', '-Z', '/dev/nvme0n1'] with allowed return codes [0] (shell=False, capture=True)
2024-05-09 15:39:22,794 - subp.py[DEBUG]: command ['/usr/sbin/sgdisk', '-Z', '/dev/nvme0n1'] took 1.0s to run
2024-05-09 15:39:22,794 - subp.py[DEBUG]: Running command ['/usr/sbin/sgdisk', '-n', '1:0:+419430', '/dev/nvme0n1'] with allowed return codes [0] (shell=False, capture=True)
2024-05-09 15:39:23,803 - subp.py[DEBUG]: command ['/usr/sbin/sgdisk', '-n', '1:0:+419430', '/dev/nvme0n1'] took 1.0s to run
2024-05-09 15:39:23,803 - subp.py[DEBUG]: Running command ['/usr/sbin/sgdisk', '-t', '1:8200', '/dev/nvme0n1'] with allowed return codes [0] (shell=False, capture=True)
2024-05-09 15:39:24,821 - subp.py[DEBUG]: command ['/usr/sbin/sgdisk', '-t', '1:8200', '/dev/nvme0n1'] took 1.0s to run
2024-05-09 15:39:24,821 - subp.py[DEBUG]: Running command ['/usr/sbin/sgdisk', '-n', '2:0:0', '/dev/nvme0n1'] with allowed return codes [0] (shell=False, capture=True)
2024-05-09 15:39:25,840 - subp.py[DEBUG]: command ['/usr/sbin/sgdisk', '-n', '2:0:0', '/dev/nvme0n1'] took 1.0s to run
2024-05-09 15:39:25,840 - subp.py[DEBUG]: Running command ['/usr/sbin/sgdisk', '-t', '2:8300', '/dev/nvme0n1'] with allowed return codes [0] (shell=False, capture=True)
2024-05-09 15:39:26,863 - subp.py[DEBUG]: command ['/usr/sbin/sgdisk', '-t', '2:8300', '/dev/nvme0n1'] took 1.0s to run
2024-05-09 15:39:26,863 - subp.py[DEBUG]: Running command ['udevadm', 'settle'] with allowed return codes [0] (shell=False, capture=True)
2024-05-09 15:39:26,883 - subp.py[DEBUG]: Running command ['/usr/sbin/partprobe', '/dev/nvme0n1'] with allowed return codes [0] (shell=False, capture=True)
2024-05-09 15:39:26,905 - subp.py[DEBUG]: Running command ['udevadm', 'settle'] with allowed return codes [0] (shell=False, capture=True)
2024-05-09 15:39:26,923 - cc_disk_setup.py[DEBUG]: Partition table created for /dev/nvme0n1
2024-05-09 15:39:26,923 - util.py[DEBUG]: Creating partition on /dev/nvme0n1 took 5.156 seconds
2024-05-09 15:39:26,923 - handlers.py[DEBUG]: finish: init-network/config-disk_setup: SUCCESS: config-disk_setup ran successfully

If run again we get:

2024-05-09 15:48:47,676 - modules.py[DEBUG]: Running module disk_setup (<module 'cloudinit.config.cc_disk_setup' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_disk_setup.py'>) with frequency once-per-instance
2024-05-09 15:48:47,676 - handlers.py[DEBUG]: start: init-network/config-disk_setup: running config-disk_setup with frequency once-per-instance
2024-05-09 15:48:47,676 - util.py[DEBUG]: Writing to /var/lib/cloud/instances/myid/sem/config_disk_setup - wb: [644] 24 bytes
2024-05-09 15:48:47,676 - helpers.py[DEBUG]: Running config-disk_setup using lock (<FileLock using file '/var/lib/cloud/instances/myid/sem/config_disk_setup'>)
2024-05-09 15:48:47,676 - cc_disk_setup.py[DEBUG]: Partitioning disks: {'/dev/nvme0n1': {'layout': [[5, 82], [95, 83]], 'overwrite': True, 'table_type': 'gpt'}}
2024-05-09 15:48:47,676 - cc_disk_setup.py[DEBUG]: Creating new partition table/disk
2024-05-09 15:48:47,676 - subp.py[DEBUG]: Running command ['udevadm', 'settle'] with allowed return codes [0] (shell=False, capture=True)
2024-05-09 15:48:47,681 - cc_disk_setup.py[DEBUG]: Checking values for /dev/nvme0n1 definition
2024-05-09 15:48:47,681 - cc_disk_setup.py[DEBUG]: Checking against default devices
2024-05-09 15:48:47,681 - cc_disk_setup.py[DEBUG]: Checking if device /dev/nvme0n1 is a valid device
2024-05-09 15:48:47,681 - subp.py[DEBUG]: Running command ['/usr/bin/lsblk', '--pairs', '--output', 'NAME,TYPE,FSTYPE,LABEL', '/dev/nvme0n1', '--nodeps'] with allowed return codes [0] (shell=False, capture=True)
2024-05-09 15:48:47,685 - cc_disk_setup.py[DEBUG]: Checking if device layout matches
2024-05-09 15:48:47,685 - subp.py[DEBUG]: Running command ['/usr/sbin/sgdisk', '-p', '/dev/nvme0n1'] with allowed return codes [0] (shell=False, capture=True)
2024-05-09 15:48:47,698 - cc_disk_setup.py[DEBUG]: called check_partition_gpt_layout(/dev/nvme0n1, [[5, 82], [95, 83]]), returned: ['82', '83']
2024-05-09 15:48:47,698 - cc_disk_setup.py[DEBUG]: Layout types=['82', '83']. Found types=['82', '83']
2024-05-09 15:48:47,698 - cc_disk_setup.py[DEBUG]: Device partitioning layout matches
2024-05-09 15:48:47,698 - util.py[DEBUG]: Creating partition on /dev/nvme0n1 took 0.022 seconds
2024-05-09 15:48:47,699 - handlers.py[DEBUG]: finish: init-network/config-disk_setup: SUCCESS: config-disk_setup ran successfully

Am I missing something? My reproducer (a bit verbose because I copied the base pieces from pycloudlib):

mkdir /tmp/qemu-test
genisoimage -output /tmp/qemu-test/seed.iso -volid cidata -joliet -rock -input-charset UTF-8 /home/james/a/user-data /home/james/a/meta-data

rm /tmp/qemu-test/inst.qcow2; qemu-img create -f qcow2 -b /home/james/images/noble/20240505/noble-server-cloudimg-amd64.img -F qcow2 /tmp/qemu-test/inst.qcow2 10G

qemu-system-x86_64 -enable-kvm -cpu host -smp cpus=2 -m size=5000 -net nic -net user,hostfwd=tcp::18000-:22 -hda /tmp/qemu-test/inst.qcow2 -nographic -chardev socket,id=char0,logfile=/tmp/qemu-test/console.log,host=0.0.0.0,port=18001,telnet=on,server=on,wait=off,mux=on -serial chardev:char0 -qmp unix:/tmp/qemu-test/qmp-socket,server,wait=off -no-shutdown -drive driver=raw,file=/tmp/qemu-test/seed.iso,if=virtio -drive driver=raw,file=/tmp/qemu-test/nvme.img,if=none,id=nvm -device nvme,serial=deadbeef,drive=nvm

[dermotbradley]

Am I missing something?

The 1st log message in his bug report only appears when "fs_setup" is also specified in user-data:

cc_disk_setup.py[DEBUG]: Creating new filesystem.

https://github.com/canonical/cloud-init/blob/main/cloudinit/config/cc_disk_setup.py#L145-L155

[holmanb]

@TheRealFalcon requesting re-review, the reporter updated the bug with more details about the reproducer.

[TheRealFalcon]

LGTM!

[holmanb]

Force pushed to update commit messages and squash one commit