Duplicate USR-A partition label after 1122.2.0 > 1122.3.0 update #1628

johan-adriaans · Oct 21, 2016

Issue Report

Bug

CoreOS Version

NAME=CoreOS
ID=coreos
VERSION=1122.3.0
VERSION_ID=1122.3.0
BUILD_ID=2016-10-20-2045
PRETTY_NAME="CoreOS 1122.3.0 (MoreOS)"
ANSI_COLOR="1;32"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://github.com/coreos/bugs/issues"

Environment

First seen on KVM and reproduced in VirtualBox 5.0.26 r108824

Expected Behavior

To have the blkid output show a USR-A and a USR-B label. Just like the cgpt show output.

Actual Behavior

After updating the blkid command shows 2 USR-A labels.

$ sudo blkid
/dev/vda9: LABEL="ROOT" UUID="af53899e-c105-4d3b-8df0-e6702ceebf51" TYPE="ext4" PARTLABEL="ROOT" PARTUUID="fca2e041-ca3e-452f-8522-336454d8a5d2"
/dev/vda1: SEC_TYPE="msdos" LABEL="EFI-SYSTEM" UUID="A21F-3BD3" TYPE="vfat" PARTLABEL="EFI-SYSTEM" PARTUUID="a9cc9ab9-9c69-417b-ba53-899ace0f1bd4"
/dev/vda2: PARTLABEL="BIOS-BOOT" PARTUUID="a20129c2-a818-4813-b0c9-9763093547e0"
/dev/vda3: LABEL="USR-A" UUID="5ee351e8-5b3e-4d88-afef-ab3e7bd9d222" SEC_TYPE="ext2" TYPE="ext4" PARTLABEL="USR-A" PARTUUID="7130c94a-213a-4e5a-8e26-6cce9662f132"
/dev/vda4: LABEL="USR-A" UUID="49f1b399-5af4-4836-8db2-6e112ed28701" SEC_TYPE="ext2" TYPE="ext4" PARTLABEL="USR-B" PARTUUID="e03dd35c-7c2d-4a47-b3fe-27f15780a57c"
/dev/vda6: LABEL="OEM" UUID="698df2df-9d80-4bbf-af46-4926cd4bb7ea" TYPE="ext4" PARTLABEL="OEM" PARTUUID="9c8c0102-5608-4918-932c-069cbad92bdf"
/dev/vda7: PARTLABEL="OEM-CONFIG" PARTUUID="cafb9e2b-6fd3-41dc-bc20-551fb170b985"
/dev/loop0: UUID="41d194a2-fc3d-4393-9649-329fc902267b" TYPE="swap"

Systemd starts throwing these errors:

systemd[2211]: dev-disk-by\x2dlabel-USR\x2dA.device: Dev dev-disk-by\x2dlabel-USR\x2dA.device appeared twice with different sysfs paths /sys/devices/pci0000:00/0000:00:05.0/virtio1/block/vda/vda3 and /sys/devices/pci0000:00/0000:00:05.0/virtio1/block/vda/vda4

Reproduction Steps

Start with a 1122.2.0 system

#### Before update

$ cat /etc/os-release
NAME=CoreOS
ID=coreos
VERSION=1122.2.0
VERSION_ID=1122.2.0
BUILD_ID=2016-09-06-1449
PRETTY_NAME="CoreOS 1122.2.0 (MoreOS)"
ANSI_COLOR="1;32"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://github.com/coreos/bugs/issues"

$ sudo blkid
/dev/vda9: LABEL="ROOT" UUID="af53899e-c105-4d3b-8df0-e6702ceebf51" TYPE="ext4"
/dev/vda1: SEC_TYPE="msdos" LABEL="EFI-SYSTEM" UUID="A21F-3BD3" TYPE="vfat" PARTLABEL="EFI-SYSTEM" PARTUUID="a9cc9ab9-9c69-417b-ba53-899ace0f1bd4"
/dev/vda2: PARTLABEL="BIOS-BOOT" PARTUUID="a20129c2-a818-4813-b0c9-9763093547e0"
/dev/vda3: LABEL="USR-A" UUID="5ee351e8-5b3e-4d88-afef-ab3e7bd9d222" SEC_TYPE="ext2" TYPE="ext4" PARTLABEL="USR-A" PARTUUID="7130c94a-213a-4e5a-8e26-6cce9662f132"
/dev/vda4: PARTLABEL="USR-B" PARTUUID="e03dd35c-7c2d-4a47-b3fe-27f15780a57c"
/dev/vda6: LABEL="OEM" UUID="698df2df-9d80-4bbf-af46-4926cd4bb7ea" TYPE="ext4" PARTLABEL="OEM" PARTUUID="9c8c0102-5608-4918-932c-069cbad92bdf"
/dev/vda7: PARTLABEL="OEM-CONFIG" PARTUUID="cafb9e2b-6fd3-41dc-bc20-551fb170b985"
/dev/loop0: UUID="34cd2711-4663-4e4a-8f7b-aae825339e8e" TYPE="swap"

$ sudo cgpt show /dev/vda
       start        size    part  contents
           0           1          Hybrid MBR
           1           1          Pri GPT header
           2          32          Pri GPT table
        4096      262144       1  Label: "EFI-SYSTEM"
                                  Type: EFI System Partition
                                  UUID: A9CC9AB9-9C69-417B-BA53-899ACE0F1BD4
                                  Attr: Legacy BIOS Bootable
      266240        4096       2  Label: "BIOS-BOOT"
                                  Type: BIOS Boot Partition
                                  UUID: A20129C2-A818-4813-B0C9-9763093547E0
      270336     2097152       3  Label: "USR-A"
                                  Type: Alias for coreos-rootfs
                                  UUID: 7130C94A-213A-4E5A-8E26-6CCE9662F132
                                  Attr: priority=1 tries=0 successful=1
     2367488     2097152       4  Label: "USR-B"
                                  Type: Alias for coreos-rootfs
                                  UUID: E03DD35C-7C2D-4A47-B3FE-27F15780A57C
                                  Attr: priority=0 tries=0 successful=0
     4464640      262144       6  Label: "OEM"
                                  Type: Alias for linux-data
                                  UUID: 9C8C0102-5608-4918-932C-069CBAD92BDF
     4726784      131072       7  Label: "OEM-CONFIG"
                                  Type: CoreOS reserved
                                  UUID: CAFB9E2B-6FD3-41DC-BC20-551FB170B985
     4857856   624287711       9  Label: "ROOT"
                                  Type: CoreOS auto-resize
                                  UUID: FCA2E041-CA3E-452F-8522-336454D8A5D2
   629145567          32          Sec GPT table
   629145599           1          Sec GPT header

#### Update using

$ sudo update_agent_client -update

####  After update, before reboot

$ sudo blkid
/dev/vda9: LABEL="ROOT" UUID="af53899e-c105-4d3b-8df0-e6702ceebf51" TYPE="ext4"
/dev/vda1: SEC_TYPE="msdos" LABEL="EFI-SYSTEM" UUID="A21F-3BD3" TYPE="vfat" PARTLABEL="EFI-SYSTEM" PARTUUID="a9cc9ab9-9c69-417b-ba53-899ace0f1bd4"
/dev/vda3: LABEL="USR-A" UUID="5ee351e8-5b3e-4d88-afef-ab3e7bd9d222" SEC_TYPE="ext2" TYPE="ext4" PARTLABEL="USR-A" PARTUUID="7130c94a-213a-4e5a-8e26-6cce9662f132"
/dev/vda6: LABEL="OEM" UUID="698df2df-9d80-4bbf-af46-4926cd4bb7ea" TYPE="ext4" PARTLABEL="OEM" PARTUUID="9c8c0102-5608-4918-932c-069cbad92bdf"
/dev/loop0: UUID="34cd2711-4663-4e4a-8f7b-aae825339e8e" TYPE="swap"
/dev/vda2: PARTLABEL="BIOS-BOOT" PARTUUID="a20129c2-a818-4813-b0c9-9763093547e0"
/dev/vda4: LABEL="USR-A" UUID="49f1b399-5af4-4836-8db2-6e112ed28701" SEC_TYPE="ext2" TYPE="ext4" PARTLABEL="USR-B" PARTUUID="e03dd35c-7c2d-4a47-b3fe-27f15780a57c"
/dev/vda7: PARTLABEL="OEM-CONFIG" PARTUUID="cafb9e2b-6fd3-41dc-bc20-551fb170b985"

$ sudo cgpt show /dev/vda
       start        size    part  contents
           0           1          Hybrid MBR
           1           1          Pri GPT header
           2          32          Pri GPT table
        4096      262144       1  Label: "EFI-SYSTEM"
                                  Type: EFI System Partition
                                  UUID: A9CC9AB9-9C69-417B-BA53-899ACE0F1BD4
                                  Attr: Legacy BIOS Bootable
      266240        4096       2  Label: "BIOS-BOOT"
                                  Type: BIOS Boot Partition
                                  UUID: A20129C2-A818-4813-B0C9-9763093547E0
      270336     2097152       3  Label: "USR-A"
                                  Type: Alias for coreos-rootfs
                                  UUID: 7130C94A-213A-4E5A-8E26-6CCE9662F132
                                  Attr: priority=1 tries=0 successful=1
     2367488     2097152       4  Label: "USR-B"
                                  Type: Alias for coreos-rootfs
                                  UUID: E03DD35C-7C2D-4A47-B3FE-27F15780A57C
                                  Attr: priority=2 tries=1 successful=0
     4464640      262144       6  Label: "OEM"
                                  Type: Alias for linux-data
                                  UUID: 9C8C0102-5608-4918-932C-069CBAD92BDF
     4726784      131072       7  Label: "OEM-CONFIG"
                                  Type: CoreOS reserved
                                  UUID: CAFB9E2B-6FD3-41DC-BC20-551FB170B985
     4857856   624287711       9  Label: "ROOT"
                                  Type: CoreOS auto-resize
                                  UUID: FCA2E041-CA3E-452F-8522-336454D8A5D2
   629145567          32          Sec GPT table
   629145599           1          Sec GPT header

#### After update and reboot

$ sudo blkid
/dev/vda9: LABEL="ROOT" UUID="af53899e-c105-4d3b-8df0-e6702ceebf51" TYPE="ext4" PARTLABEL="ROOT" PARTUUID="fca2e041-ca3e-452f-8522-336454d8a5d2"
/dev/vda1: SEC_TYPE="msdos" LABEL="EFI-SYSTEM" UUID="A21F-3BD3" TYPE="vfat" PARTLABEL="EFI-SYSTEM" PARTUUID="a9cc9ab9-9c69-417b-ba53-899ace0f1bd4"
/dev/vda2: PARTLABEL="BIOS-BOOT" PARTUUID="a20129c2-a818-4813-b0c9-9763093547e0"
/dev/vda3: LABEL="USR-A" UUID="5ee351e8-5b3e-4d88-afef-ab3e7bd9d222" SEC_TYPE="ext2" TYPE="ext4" PARTLABEL="USR-A" PARTUUID="7130c94a-213a-4e5a-8e26-6cce9662f132"
/dev/vda4: LABEL="USR-A" UUID="49f1b399-5af4-4836-8db2-6e112ed28701" SEC_TYPE="ext2" TYPE="ext4" PARTLABEL="USR-B" PARTUUID="e03dd35c-7c2d-4a47-b3fe-27f15780a57c"
/dev/vda6: LABEL="OEM" UUID="698df2df-9d80-4bbf-af46-4926cd4bb7ea" TYPE="ext4" PARTLABEL="OEM" PARTUUID="9c8c0102-5608-4918-932c-069cbad92bdf"
/dev/vda7: PARTLABEL="OEM-CONFIG" PARTUUID="cafb9e2b-6fd3-41dc-bc20-551fb170b985"
/dev/loop0: UUID="41d194a2-fc3d-4393-9649-329fc902267b" TYPE="swap"

$ sudo cgpt show /dev/vda
       start        size    part  contents
           0           1          Hybrid MBR
           1           1          Pri GPT header
           2          32          Pri GPT table
        4096      262144       1  Label: "EFI-SYSTEM"
                                  Type: EFI System Partition
                                  UUID: A9CC9AB9-9C69-417B-BA53-899ACE0F1BD4
                                  Attr: Legacy BIOS Bootable
      266240        4096       2  Label: "BIOS-BOOT"
                                  Type: BIOS Boot Partition
                                  UUID: A20129C2-A818-4813-B0C9-9763093547E0
      270336     2097152       3  Label: "USR-A"
                                  Type: Alias for coreos-rootfs
                                  UUID: 7130C94A-213A-4E5A-8E26-6CCE9662F132
                                  Attr: priority=1 tries=0 successful=1
     2367488     2097152       4  Label: "USR-B"
                                  Type: Alias for coreos-rootfs
                                  UUID: E03DD35C-7C2D-4A47-B3FE-27F15780A57C
                                  Attr: priority=2 tries=0 successful=1
     4464640      262144       6  Label: "OEM"
                                  Type: Alias for linux-data
                                  UUID: 9C8C0102-5608-4918-932C-069CBAD92BDF
     4726784      131072       7  Label: "OEM-CONFIG"
                                  Type: CoreOS reserved
                                  UUID: CAFB9E2B-6FD3-41DC-BC20-551FB170B985
     4857856   624287711       9  Label: "ROOT"
                                  Type: CoreOS auto-resize
                                  UUID: FCA2E041-CA3E-452F-8522-336454D8A5D2
   629145567          32          Sec GPT table
   629145599           1          Sec GPT header


#### Errors in journal

systemd[2211]: dev-disk-by\x2dlabel-USR\x2dA.device: Dev dev-disk-by\x2dlabel-USR\x2dA.device appeared twice with different sysfs paths /sys/devices/pci0000:00/0000:00:05.0/virtio1/block/vda/vda3 and /sys/devices/pci0000:00/0000:00:05.0/virtio1/block/vda/vda4
systemd[2211]: dev-disk-by\x2dlabel-USR\x2dA.device: Dev dev-disk-by\x2dlabel-USR\x2dA.device appeared twice with different sysfs paths /sys/devices/pci0000:00/0000:00:05.0/virtio1/block/vda/vda3 and /sys/devices/pci0000:00/0000:00:05.0/virtio1/block/vda/vda4

Other Information

Note that i have a swap file connected to /dev/loop0

Here is the update log itself

update_engine[708]: [1021/130852:INFO:multi_range_http_fetcher.cc(150)] Received transfer complete.
update_engine[708]: [1021/130852:INFO:multi_range_http_fetcher.cc(107)] TransferEnded w/ code 200
update_engine[708]: [1021/130852:INFO:multi_range_http_fetcher.cc(141)] Done w/ all transfers
locksmithd[1727]: LastCheckedTime=1477055275 Progress=1 CurrentOperation="UPDATE_STATUS_DOWNLOADING" NewVersion=0.0.0.0 NewSize=213129575
update_engine[708]: [1021/130852:INFO:update_attempter.cc(408)] Download status: inactive
update_engine[708]: [1021/130852:INFO:delta_performer.cc(771)] Verifying delta payload using public key: /usr/share/update_engine/update-payload-key.pub.pem
systemd-udevd[4590]: Process '/usr/lib/udev/scripts/iscsidev.sh' failed with exit code 1.
systemd[1]: dev-disk-by\x2dlabel-USR\x2dA.device: Dev dev-disk-by\x2dlabel-USR\x2dA.device appeared twice with different sysfs paths /sys/devices/pci0000:00/0000:00:01.1/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda3 and /sys/devices/pci0000:00/0000:0
:0:0/block/sda/sda4
update_engine[708]: [1021/130852:INFO:payload_signer.cc(265)] signature size = 528
update_engine[708]: [1021/130852:INFO:payload_state.cc(70)] Payload downloaded successfully
update_engine[708]: [1021/130852:INFO:payload_state.cc(246)] Incrementing the payload attempt number
update_engine[708]: [1021/130852:INFO:payload_state.cc(377)] Payload Attempt Number = 1
update_engine[708]: [1021/130852:INFO:payload_state.cc(282)] Resetting backoff expiry time as payload backoff is disabled
update_engine[708]: [1021/130852:INFO:payload_state.cc(448)] Backoff Expiry Time = 1/1/1601 0:00:00 GMT
update_engine[708]: [1021/130852:INFO:action_processor.cc(82)] ActionProcessor::ActionComplete: finished DownloadAction, starting OmahaRequestAction
update_engine[708]: [1021/130852:INFO:omaha_request_action.cc(257)] Posting an Omaha request to https://public.update.core-os.net/v1/update/
update_engine[708]: [1021/130852:INFO:omaha_request_action.cc(258)] Request: <?xml version="1.0" encoding="UTF-8"?>
update_engine[708]: <request protocol="3.0" version="CoreOSUpdateEngine-0.1.0.0" updaterversion="CoreOSUpdateEngine-0.1.0.0" installsource="scheduler" ismachine="1">
update_engine[708]:     <os version="Chateau" platform="CoreOS" sp="1122.2.0_x86_64"></os>
update_engine[708]:     <app appid="{e96281a6-d1af-4bde-9a0a-97b76e56dc57}" version="1122.2.0" track="stable" bootid="{8794055b-d844-42dd-8305-4b874538f620}" oem="vagrant" oemversion="0.0.3" alephversion="1122.2.0" machineid="4467910ad1d14842b1aead4e
64-usr" hardware_class="" delta_okay="false" >
update_engine[708]:         <event eventtype="14" eventresult="1"></event>
update_engine[708]:     </app>
update_engine[708]: </request>
update_engine[708]: [1021/130852:INFO:libcurl_http_fetcher.cc(48)] Starting/Resuming transfer
update_engine[708]: [1021/130852:INFO:libcurl_http_fetcher.cc(164)] Setting up curl options for HTTPS
locksmithd[1727]: LastCheckedTime=1477055275 Progress=0 CurrentOperation="UPDATE_STATUS_FINALIZING" NewVersion=0.0.0.0 NewSize=213129575
update_engine[708]: [1021/130852:INFO:libcurl_http_fetcher.cc(427)] Setting up timeout source: 1 seconds.
update_engine[708]: [1021/130852:INFO:libcurl_http_fetcher.cc(240)] HTTP response code: 200
update_engine[708]: [1021/130852:INFO:libcurl_http_fetcher.cc(297)] Transfer completed (200), 267 bytes downloaded
update_engine[708]: [1021/130852:INFO:omaha_request_action.cc(574)] Omaha request response: <?xml version="1.0" encoding="UTF-8"?>
update_engine[708]: <response protocol="3.0" server="update.core-os.net">
update_engine[708]:  <daystart elapsed_seconds="0"></daystart>
update_engine[708]:  <app appid="e96281a6-d1af-4bde-9a0a-97b76e56dc57" status="ok">
update_engine[708]:   <updatecheck status="noupdate"></updatecheck>
update_engine[708]:  </app>
update_engine[708]: </response>
update_engine[708]: [1021/130852:INFO:action_processor.cc(82)] ActionProcessor::ActionComplete: finished OmahaRequestAction, starting FilesystemCopierAction
update_engine[708]: [1021/130852:INFO:filesystem_copier_action.cc(315)] Filesystem size: 1065345024
update_engine[708]: [1021/130900:INFO:filesystem_copier_action.cc(283)] Hash: iIxHDnUg9iuoWerQq7ZnYpAodSevpPlFWKRNzEA9uWg=
update_engine[708]: [1021/130900:INFO:action_processor.cc(82)] ActionProcessor::ActionComplete: finished FilesystemCopierAction, starting PostinstallRunnerAction
kernel: EXT4-fs (sda4): mounting ext2 file system using the ext4 subsystem
kernel: EXT4-fs (sda4): mounted filesystem without journal. Opts: (null)
systemd-udevd[4622]: Process '/usr/lib/udev/scripts/iscsidev.sh' failed with exit code 1.
update_engine[708]: [1021/130900:INFO:subprocess.cc(46)] Subprocess output:
update_engine[708]: '/tmp/au_postint_mount.NamK6I/boot/vmlinuz' -> '/boot/coreos/vmlinuz-b'
update_engine[708]:        start        size    part  contents
update_engine[708]:      2367488     2097152       4  Label: "USR-B"
update_engine[708]:                                   Type: Alias for coreos-rootfs
update_engine[708]:                                   UUID: E03DD35C-7C2D-4A47-B3FE-27F15780A57C
update_engine[708]:                                   Attr: priority=2 tries=1 successful=0
update_engine[708]: COREOS_RELEASE_VERSION=1122.3.0
update_engine[708]: Setup USR-B (/dev/sda4) for next boot.
update_engine[708]: [1021/130900:INFO:postinstall_runner_action.cc(79)] Postinst command succeeded
update_engine[708]: [1021/130900:INFO:action_processor.cc(82)] ActionProcessor::ActionComplete: finished PostinstallRunnerAction, starting OmahaRequestAction
update_engine[708]: [1021/130900:INFO:omaha_request_action.cc(257)] Posting an Omaha request to https://public.update.core-os.net/v1/update/
update_engine[708]: [1021/130900:INFO:omaha_request_action.cc(258)] Request: <?xml version="1.0" encoding="UTF-8"?>
update_engine[708]: <request protocol="3.0" version="CoreOSUpdateEngine-0.1.0.0" updaterversion="CoreOSUpdateEngine-0.1.0.0" installsource="scheduler" ismachine="1">
update_engine[708]:     <os version="Chateau" platform="CoreOS" sp="1122.2.0_x86_64"></os>
update_engine[708]:     <app appid="{e96281a6-d1af-4bde-9a0a-97b76e56dc57}" version="1122.2.0" track="stable" bootid="{8794055b-d844-42dd-8305-4b874538f620}" oem="vagrant" oemversion="0.0.3" alephversion="1122.2.0" machineid="4467910ad1d14842b1aead4e
64-usr" hardware_class="" delta_okay="false" >
update_engine[708]:         <event eventtype="3" eventresult="1"></event>
update_engine[708]:     </app>
update_engine[708]: </request>
update_engine[708]: [1021/130900:INFO:libcurl_http_fetcher.cc(48)] Starting/Resuming transfer
update_engine[708]: [1021/130900:INFO:libcurl_http_fetcher.cc(164)] Setting up curl options for HTTPS
systemd-udevd[4622]: Process '/usr/lib/udev/scripts/iscsidev.sh' failed with exit code 1.
update_engine[708]: [1021/130900:INFO:libcurl_http_fetcher.cc(427)] Setting up timeout source: 1 seconds.
systemd-udevd[4630]: Process '/usr/lib/udev/scripts/iscsidev.sh' failed with exit code 1.
systemd-udevd[4632]: Process '/usr/lib/udev/scripts/iscsidev.sh' failed with exit code 1.
systemd-udevd[4629]: Process '/usr/lib/udev/scripts/iscsidev.sh' failed with exit code 1.
systemd-udevd[4628]: Process '/usr/lib/udev/scripts/iscsidev.sh' failed with exit code 1.
systemd[4559]: dev-disk-by\x2dlabel-USR\x2dA.device: Dev dev-disk-by\x2dlabel-USR\x2dA.device appeared twice with different sysfs paths /sys/devices/pci0000:00/0000:00:01.1/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda4 and /sys/devices/pci0000:00/000
0:0:0:0/block/sda/sda3
systemd[4559]: dev-disk-by\x2duuid-5ee351e8\x2d5b3e\x2d4d88\x2dafef\x2dab3e7bd9d222.device: Dev dev-disk-by\x2duuid-5ee351e8\x2d5b3e\x2d4d88\x2dafef\x2dab3e7bd9d222.device appeared twice with different sysfs paths /sys/devices/pci0000:00/0000:00:01.1
lock/sda/sda4 and /sys/devices/pci0000:00/0000:00:01.1/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda3
systemd-udevd[4627]: Process '/usr/lib/udev/scripts/iscsidev.sh' failed with exit code 1.
systemd-udevd[4631]: Process '/usr/lib/udev/scripts/iscsidev.sh' failed with exit code 1.
systemd-udevd[4627]: Process '/usr/lib/udev/scripts/iscsidev.sh' failed with exit code 1.
systemd-udevd[4632]: Process '/usr/lib/udev/scripts/iscsidev.sh' failed with exit code 1.
systemd-udevd[4628]: Process '/usr/lib/udev/scripts/iscsidev.sh' failed with exit code 1.
systemd[4559]: dev-disk-by\x2duuid-5ee351e8\x2d5b3e\x2d4d88\x2dafef\x2dab3e7bd9d222.device: Dev dev-disk-by\x2duuid-5ee351e8\x2d5b3e\x2d4d88\x2dafef\x2dab3e7bd9d222.device appeared twice with different sysfs paths /sys/devices/pci0000:00/0000:00:01.1
lock/sda/sda4 and /sys/devices/pci0000:00/0000:00:01.1/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda3
systemd-udevd[4627]: Process '/usr/lib/udev/scripts/iscsidev.sh' failed with exit code 1.
systemd-udevd[4622]: Process '/usr/lib/udev/scripts/iscsidev.sh' failed with exit code 1.
systemd-udevd[4630]: Process '/usr/lib/udev/scripts/iscsidev.sh' failed with exit code 1.
systemd[1]: dev-disk-by\x2dlabel-USR\x2dA.device: Dev dev-disk-by\x2dlabel-USR\x2dA.device appeared twice with different sysfs paths /sys/devices/pci0000:00/0000:00:01.1/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda3 and /sys/devices/pci0000:00/0000:0
:0:0/block/sda/sda4
systemd-udevd[4629]: Process '/usr/lib/udev/scripts/iscsidev.sh' failed with exit code 1.
systemd-udevd[4631]: Process '/usr/lib/udev/scripts/iscsidev.sh' failed with exit code 1.
update_engine[708]: [1021/130900:INFO:libcurl_http_fetcher.cc(240)] HTTP response code: 200
update_engine[708]: [1021/130900:INFO:libcurl_http_fetcher.cc(297)] Transfer completed (200), 267 bytes downloaded
update_engine[708]: [1021/130900:INFO:omaha_request_action.cc(574)] Omaha request response: <?xml version="1.0" encoding="UTF-8"?>
update_engine[708]: <response protocol="3.0" server="update.core-os.net">                                                                                                                                                                        [202/413]
update_engine[708]:  <daystart elapsed_seconds="0"></daystart>
update_engine[708]:  <app appid="e96281a6-d1af-4bde-9a0a-97b76e56dc57" status="ok">
update_engine[708]:   <updatecheck status="noupdate"></updatecheck>
update_engine[708]:  </app>
update_engine[708]: </response>
update_engine[708]: [1021/130900:INFO:action_processor.cc(65)] ActionProcessor::ActionComplete: finished last action of type OmahaRequestAction
update_engine[708]: [1021/130900:INFO:action_processor.cc(73)] ActionProcessor::ActionComplete: finished last action of type OmahaRequestAction
update_engine[708]: [1021/130900:INFO:update_attempter.cc(283)] Processing Done.
update_engine[708]: [1021/130900:INFO:update_attempter.cc(309)] Update successfully applied, waiting to reboot.
update_engine[708]: [1021/130900:INFO:update_check_scheduler.cc(82)] Next update check in 49m51s
locksmithd[1727]: LastCheckedTime=1477055275 Progress=0 CurrentOperation="UPDATE_STATUS_UPDATED_NEED_REBOOT" NewVersion=0.0.0.0 NewSize=213129575
locksmithd[1727]: Waiting for 85h50m58.999957367s to reboot.

And finally, 2 of my (workers, so np there) servers failed to restart after this automatic update. After a hard system reset they came back up without any abnormal errors: Here is the log for one of them.

....
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:omaha_request_action.cc(463)] Processing first of 1 package(s)
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:omaha_request_action.cc(470)] Omaha Response package name = update.gz
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:omaha_request_action.cc(482)] Url0: https://update.release.core-os.net/amd64-usr/1122.3.0/update.gz
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:omaha_request_action.cc(494)] Payload size = 213129575 bytes
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:omaha_request_action.cc(518)] Found 1 action(s). Processing the postinstall action.
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:payload_state.cc(51)] Resetting all persisted state as this is a new response
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:payload_state.cc(356)] Current Response Signature =
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: NumURLs = 1
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: Url0 = https://update.release.core-os.net/amd64-usr/1122.3.0/update.gz
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: Payload Size = 213129575
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: Payload Sha256 Hash = aYnn+P/Z4jioLp7i24Eah77r0Ns6s7z/0Y9tP0712Oc=
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: Is Delta Payload = 0
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: Max Failure Count Per Url = 10
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: Disable Payload Backoff = 1
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:payload_state.cc(377)] Payload Attempt Number = 0
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:payload_state.cc(400)] Current URL Index = 0
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:payload_state.cc(421)] Current URL (Url0)'s Failure Count = 0
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:payload_state.cc(282)] Resetting backoff expiry time as payload backoff is disabled
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:payload_state.cc(448)] Backoff Expiry Time = 1/1/1601 0:00:00 GMT
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:payload_state.cc(199)] Payload backoff logic is disabled. Can proceed with the download
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:action_processor.cc(82)] ActionProcessor::ActionComplete: finished OmahaRequestAction, starting OmahaResponseHandlerAction
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:omaha_response_handler_action.cc(43)] Using Url0 as the download url this time
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:omaha_response_handler_action.cc(134)] Waiving payload hash checks since Omaha response only has HTTPS URL(s)
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:prefs.cc(51)] update-state-next-operation not present in /var/lib/update_engine/prefs
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:ERROR:delta_performer.cc(941)] prefs->GetInt64(kPrefsUpdateStateNextOperation, &next_operation) && next_operation != kUpdateStateOperationInvalid && next_operation > 0 failed.
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:omaha_response_handler_action.cc(71)] Using this install plan:
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:install_plan.cc(52)] InstallPlan: , new_update, url: https://update.release.core-os.net/amd64-usr/1122.3.0/update.gz, payload size: 213129575, payload hash: aYnn+P/Z4jioLp7i24Eah77r0Ns6s7z/0Y9tP0712Oc=, install_p
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:utils.cc(633)] Setting cgroup cpu shares to  2
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:ERROR:utils.cc(118)] 0 == writer.Open(path, O_WRONLY | O_CREAT | O_TRUNC, 0600) failed: No such file or directory
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:ERROR:utils.cc(638)] Failed to change cgroup cpu shares to 2 using /sys/fs/cgroup/cpu/update-engine/cpu.shares
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:action_processor.cc(82)] ActionProcessor::ActionComplete: finished OmahaResponseHandlerAction, starting FilesystemCopierAction
Oct 21 07:03:32 cluster-worker-1 update_engine[983]: [1021/070332:INFO:filesystem_copier_action.cc(323)] Filesystem size: 1065345024 bytes (260094x4096).
Oct 21 07:03:32 cluster-worker-1 locksmithd[1937]: LastCheckedTime=1477033412 Progress=0 CurrentOperation="UPDATE_STATUS_UPDATE_AVAILABLE" NewVersion=0.0.0.0 NewSize=213129575
-- Reboot --
Oct 21 10:20:50 localhost systemd-journald[113]: Runtime journal (/run/log/journal/) is 8.0M, max 197.8M, 189.8M free.
Oct 21 10:20:50 localhost kernel: Linux version 4.7.0-coreos (jenkins@jenkins-os-executor-1.c.coreos-gce-testing.internal) (gcc version 4.9.3 (Gentoo Hardened 4.9.3 p1.5, pie-0.6.4) ) #1 SMP Tue Sep 6 14:39:20 UTC 2016
Oct 21 10:20:50 localhost kernel: Command line: BOOT_IMAGE=/coreos/vmlinuz-a mount.usr=PARTUUID=7130c94a-213a-4e5a-8e26-6cce9662f132 rootflags=rw mount.usrflags=ro consoleblank=0 root=LABEL=ROOT console=ttyS0,115200n8 console=tty0 verity.usrhash=859e54440447a43d9a74eb6e965c083c0396e
Oct 21 10:20:50 localhost kernel: x86/fpu: Legacy x87 FPU detected.
....

jumanjiman · Oct 21, 2016

also on digital ocean:

core@coreos-2gb-nyc1-01 ~ $ sudo blkid
/dev/vda9: LABEL="ROOT" UUID="af53899e-c105-4d3b-8df0-e6702ceebf51" TYPE="ext4" PARTLABEL="ROOT" PARTUUID="8ba0b56e-4f8b-4353-92bb-40ebab275dd9"
/dev/vda1: SEC_TYPE="msdos" LABEL="EFI-SYSTEM" UUID="A21F-3BD3" TYPE="vfat" PARTLABEL="EFI-SYSTEM" PARTUUID="2013ab12-720a-4bb3-8964-9371c9120753"
/dev/vda2: PARTLABEL="BIOS-BOOT" PARTUUID="3114d3e7-2546-4705-9a3c-dbd2b84e0376"
/dev/vda3: LABEL="USR-A" UUID="5ee351e8-5b3e-4d88-afef-ab3e7bd9d222" SEC_TYPE="ext2" TYPE="ext4" PARTLABEL="USR-A" PARTUUID="7130c94a-213a-4e5a-8e26-6cce9662f132"
/dev/vda4: LABEL="USR-A" UUID="49f1b399-5af4-4836-8db2-6e112ed28701" SEC_TYPE="ext2" TYPE="ext4" PARTLABEL="USR-B" PARTUUID="e03dd35c-7c2d-4a47-b3fe-27f15780a57c"
/dev/vda6: LABEL="OEM" UUID="698df2df-9d80-4bbf-af46-4926cd4bb7ea" TYPE="ext4" PARTLABEL="OEM" PARTUUID="f1d996f1-3613-46b0-9893-4d2852300b45"
/dev/vda7: PARTLABEL="OEM-CONFIG" PARTUUID="7bb6cc92-64c3-4ee2-b601-ea4f45c80550"

takhyon · Oct 21, 2016

also on aws:

core@ip-192-168-254-21 ~ $ sudo blkid
/dev/xvda9: LABEL="ROOT" UUID="42724924-e503-4742-8f51-001657ac8284" UUID_SUB="bc33bdc1-613c-462c-a31c-eb941dfb51fe" TYPE="btrfs" PARTLABEL="ROOT" PARTUUID="34101eb0-e409-49d2-9dbe-92084ff15046"
/dev/xvdb: UUID="8b9157f0-336d-46db-bccf-7a3659d38fb2" SEC_TYPE="ext2" TYPE="ext3"
/dev/xvda1: SEC_TYPE="msdos" LABEL="EFI-SYSTEM" UUID="BDFE-81B5" TYPE="vfat" PARTLABEL="EFI-SYSTEM" PARTUUID="5c878c07-59b6-4d1f-aad1-58f4e4e1db83"
/dev/xvda2: PARTLABEL="BOOT-B" PARTUUID="4080a1e6-2d8b-47bb-b5b9-ede5b5112589"
/dev/xvda3: LABEL="USR-A" UUID="5ee351e8-5b3e-4d88-afef-ab3e7bd9d222" SEC_TYPE="ext2" TYPE="ext4" PARTLABEL="USR-A" PARTUUID="7130c94a-213a-4e5a-8e26-6cce9662f132"
/dev/xvda4: LABEL="USR-A" UUID="49f1b399-5af4-4836-8db2-6e112ed28701" SEC_TYPE="ext2" TYPE="ext4" PARTLABEL="USR-B" PARTUUID="e03dd35c-7c2d-4a47-b3fe-27f15780a57c"
/dev/xvda6: LABEL="OEM" UUID="d16789d6-53e6-4ec5-823b-4683b880bf75" TYPE="ext4" PARTLABEL="OEM" PARTUUID="728d08fb-f273-49a5-a901-a57a46ba99f1"

crawford · Oct 21, 2016

While annoying, this shouldn't lead to any loss in functionality. For what it's worth, my machine has been complaining about the same thing for a while now:

Jul 03 01:56:01 apollo systemd[1]: dev-disk-by\x2dlabel-USR\x2dA.device: Dev dev-disk-by\x2dlabel-USR\x2dA.device appeared twice with different sysfs paths /sys/devices/pci0000:00/0000:00:06.0/virtio3/block/vda/vda4 and /sys/devices/pci0000:00/0000:00:06.0/virtio3/block/vda/vda3

Having briefly thought about it, I wouldn't be surprised if this behavior has always been present. We don't do much manipulation to the update payload before it is written to disk. Since we don't know at the time we generate the payload whether it will go into A or B, we can't accurately label the filesystem; so it gets USR-A. The partlabels and partuuids, on the other hand, will be accurate, since they are unaffected by updates.

crawford · Oct 21, 2016

As I suspected, this has been around for a long time:

$ cat /etc/os-release 
NAME=CoreOS
ID=coreos
VERSION=444.5.0
VERSION_ID=444.5.0
BUILD_ID=
PRETTY_NAME="CoreOS 444.5.0"
ANSI_COLOR="1;32"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://github.com/coreos/bugs/issues"

$ sudo blkid
/dev/xvda9: LABEL="ROOT" UUID="c7d88fef-6335-4162-8ddf-7a3513fdeb15" UUID_SUB="e630eab6-7c54-4d18-b139-310a10f9c3e6" TYPE="btrfs" PARTLABEL="ROOT" PARTUUID="d52885de-9f00-4b5f-a30a-4f49160fee39" 
/dev/xvda1: SEC_TYPE="msdos" LABEL="EFI-SYSTEM" UUID="631E-274A" TYPE="vfat" PARTLABEL="EFI-SYSTEM" PARTUUID="e5cbfa22-2443-4ad7-afa3-a8f604bdafd7" 
/dev/xvda2: PARTLABEL="BOOT-B" PARTUUID="e54b4ce8-4ce5-413d-87dc-a56a0ef44035" 
/dev/xvda3: LABEL="USR-A" UUID="f1234c7c-9878-4519-9331-9d8931a0f751" SEC_TYPE="ext2" TYPE="ext4" PARTLABEL="USR-A" PARTUUID="7130c94a-213a-4e5a-8e26-6cce9662f132" 
/dev/xvda4: LABEL="USR-A" UUID="4616fbcf-44f2-4104-abac-0241acad152e" SEC_TYPE="ext2" TYPE="ext4" PARTLABEL="USR-B" PARTUUID="e03dd35c-7c2d-4a47-b3fe-27f15780a57c" 
/dev/xvda6: LABEL="OEM" UUID="fa264e5a-d571-4139-af7f-b27c5e9805bd" TYPE="ext4" PARTLABEL="OEM" PARTUUID="b788d9a1-0d34-4553-a94f-ea9b68774eac" 
/dev/xvda7: PARTLABEL="OEM-CONFIG" PARTUUID="608b2fac-7fcb-43b7-bbad-6a91f60cdcc1"

I suspect it's only now just being noticed because of the warning systemd added to the logs.

marineam · Dec 1, 2016

This is something I broke very early on, originally in ChromeOS the root (our /usr) partition filesystem has no label at all. In rewriting some of that code to adapt from root to our /usr scheme I lost that special case and just let the filesystem always get labeled with the partition label. At the time it didn't raise a warning/error and we always use partition labels, not filesystem labels, so I wasn't concerned about the issue at the time.

crawford added priority/P2 kind/friction component/distro team/os area/stability labels Oct 21, 2016

crawford added this to the CoreOS Alpha 1284.0.0 milestone Dec 19, 2016

crawford modified the milestones: CoreOS Alpha 1284.0.0, CoreOS Alpha 1300.0.0 Jan 5, 2017

dm0- referenced this issue Jan 19, 2017
Merged
disk_util: differentiate between partition and FS labels #632

dm0- closed this in coreos/scripts#632 Jan 19, 2017

coreos/bugs

Duplicate USR-A partition label after 1122.2.0 > 1122.3.0 update #1628

johan-adriaans commented Oct 21, 2016 •

edited

This comment has been minimized.

jumanjiman commented Oct 21, 2016

This comment has been minimized.

takhyon commented Oct 21, 2016

crawford added priority/P2 kind/friction component/distro team/os area/stability labels Oct 21, 2016

This comment has been minimized.

crawford commented Oct 21, 2016

This comment has been minimized.

crawford commented Oct 21, 2016

This comment has been minimized.

marineam commented Dec 1, 2016

crawford added this to the CoreOS Alpha 1284.0.0 milestone Dec 19, 2016

crawford modified the milestones: CoreOS Alpha 1284.0.0, CoreOS Alpha 1300.0.0 Jan 5, 2017

dm0- referenced this issue Jan 19, 2017

disk_util: differentiate between partition and FS labels #632

dm0- closed this in coreos/scripts#632 Jan 19, 2017

coreos/bugs

Join GitHub today

Duplicate USR-A partition label after 1122.2.0 > 1122.3.0 update #1628

Comments

johan-adriaans commented Oct 21, 2016 • edited

Issue Report

Bug

CoreOS Version

Environment

Expected Behavior

Actual Behavior

Reproduction Steps

Other Information

This comment has been minimized.

jumanjiman commented Oct 21, 2016

This comment has been minimized.

takhyon commented Oct 21, 2016

crawford added priority/P2 kind/friction component/distro team/os area/stability labels Oct 21, 2016

This comment has been minimized.

crawford commented Oct 21, 2016

This comment has been minimized.

crawford commented Oct 21, 2016

This comment has been minimized.

marineam commented Dec 1, 2016

crawford added this to the CoreOS Alpha 1284.0.0 milestone Dec 19, 2016

crawford modified the milestones: CoreOS Alpha 1284.0.0, CoreOS Alpha 1300.0.0 Jan 5, 2017

dm0- referenced this issue Jan 19, 2017

disk_util: differentiate between partition and FS labels #632

dm0- closed this in coreos/scripts#632 Jan 19, 2017

johan-adriaans commented Oct 21, 2016 •

edited