/systemd

cryptsetup errors on shutdown #5527

Closed
mbiebl opened this Issue Mar 3, 2017 · 5 comments

Comments

Assignees
No one assigned
Labels
bug cryptsetup
Projects
None yet
Milestone
No milestone
3 participants
@mbiebl @poettering @arvidjaar
@mbiebl
Contributor

mbiebl commented Mar 3, 2017 •
Edited 1 time

Submission type

  • Bug report
  • Request for enhancement (RFE)

systemd version the issue has been seen with

v232

Used distribution

Debian sid

I have a test VM using a standard Debian luks encrypted LVM partitioning.

# lvdisplay 
  --- Logical volume ---
  LV Path                /dev/debian-vg/root
  LV Name                root
  VG Name                debian-vg
  LV UUID                jRMDEA-h7Cc-rHRd-t0nq-47oW-Vj7P-QjfqCF
  LV Write Access        read/write
  LV Creation host, time debian, 2017-02-10 02:53:13 +0100
  LV Status              available
  # open                 1
  LV Size                2,23 GiB
  Current LE             570
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:1
   
  --- Logical volume ---
  LV Path                /dev/debian-vg/var
  LV Name                var
  VG Name                debian-vg
  LV UUID                bIHGCW-4rSP-2KOl-DjnZ-9z0b-4jmo-eqmg2z
  LV Write Access        read/write
  LV Creation host, time debian, 2017-02-10 02:53:13 +0100
  LV Status              available
  # open                 1
  LV Size                1,05 GiB
  Current LE             269
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:2
   
  --- Logical volume ---
  LV Path                /dev/debian-vg/swap_1
  LV Name                swap_1
  VG Name                debian-vg
  LV UUID                BErrye-p61K-37oL-HkYW-nsMX-UAEG-fOlrCz
  LV Write Access        read/write
  LV Creation host, time debian, 2017-02-10 02:53:13 +0100
  LV Status              available
  # open                 2
  LV Size                1020,00 MiB
  Current LE             255
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:3
   
  --- Logical volume ---
  LV Path                /dev/debian-vg/tmp
  LV Name                tmp
  VG Name                debian-vg
  LV UUID                Tnzbeh-XvTj-M4Le-xCdA-ekMF-SDeS-eQkKOQ
  LV Write Access        read/write
  LV Creation host, time debian, 2017-02-10 02:53:13 +0100
  LV Status              available
  # open                 1
  LV Size                264,00 MiB
  Current LE             66
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:4
   
  --- Logical volume ---
  LV Path                /dev/debian-vg/home
  LV Name                home
  VG Name                debian-vg
  LV UUID                RhGWra-so1e-HzCI-ZbrX-P7EZ-Ga5w-Xc2jCe
  LV Write Access        read/write
  LV Creation host, time debian, 2017-02-10 02:53:14 +0100
  LV Status              available
  # open                 1
  LV Size                5,22 GiB
  Current LE             1337
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:5

# cat /etc/crypttab 
vda5_crypt UUID=56892e58-2603-44c0-8681-d8aec1645016 none luks

# cat /etc/fstab
/dev/mapper/debian--vg-root /               ext4    errors=remount-ro 0       1
# /boot was on /dev/vda1 during installation
UUID=e7870192-32b7-427a-a259-2842f242423d /boot           ext2    defaults        0       2
/dev/mapper/debian--vg-home /home           ext4    defaults        0       2
/dev/mapper/debian--vg-tmp /tmp            ext4    defaults        0       2
/dev/mapper/debian--vg-var /var            ext4    defaults        0       2
/dev/mapper/debian--vg-swap_1 none            swap    sw              0       0
/dev/sr0        /media/cdrom0   udf,iso9660 user,noauto     0       0

During shutdown I get the following error

[  387.421061] systemd[1]: Stopped (with error) /dev/disk/by-id/dm-name-vda5_crypt.
[FAILED] Stopped (with error) /dev/disk/by-id/dm-name-vda5_crypt.
[  387.422586] systemd[1]: Stopped (with error) /dev/mapper/vda5_crypt.
[FAILED] Stopped (with error) /dev/mapper/vda5_crypt.
[  387.423929] systemd[1]: Stopped (with error) /sys/devices/virtual/block/dm-0.
[FAILED] Stopped (with error) /sys/devices/virtual/block/dm-0.
[  387.425630] systemd[1]: Stopped (with error) /dev/dm-0.
[FAILED] Stopped (with error) /dev/dm-0.
[FAILED] Stopped (with error) /dev/disk/by-i…0344c08681d8aec1645016-vda5_crypt.
[FAILED] Stopped (with error) /dev/disk/by-i…r-0srK-HqO4-eIyK-X1AF-wtox-W7SPa5.

The complete shutdown log is

# halt
         Stopping User Manager for UID 0...
[  OK  ] Stopped target Timers.
[  OK  ] Stopped Daily Cleanup of Temporary Directories.
         Stopping Session 1 of user root.
[  OK  ] Stopped Daily apt activities.
[  OK  ] Stopped target Sound Card.
[  OK  ] Stopped target Graphical Interface.
[  OK  ] Stopped target Multi-User System.
[  OK  ] Stopped target Login Prompts.
         Stopping Getty on tty1...
         Stopping Serial Getty on ttyS0...
         Stopping OpenBSD Secure Shell server...
         Stopping LSB: exim Mail Transport Agent...
         Stopping D-Bus System Message Bus...
         Stopping System Logging Service...
         Stopping Regular background program processing daemon...
[  OK  ] Stopped D-Bus System Message Bus.
[  OK  ] Stopped System Logging Service.
[  OK  ] Stopped Regular background program processing daemon.
[  OK  ] Stopped OpenBSD Secure Shell server.
[  OK  ] Stopped Getty on tty1.
[  OK  ] Stopped Serial Getty on ttyS0.
[  OK  ] Stopped User Manager for UID 0.
[  OK  ] Stopped Session 1 of user root.
[  OK  ] Removed slice User Slice of root.
         Stopping Login Service...
[  OK  ] Removed slice system-serial\x2dgetty.slice.
[  OK  ] Removed slice system-getty.slice.
         Stopping Permit User Sessions...
[  OK  ] Stopped Login Service.
[  OK  ] Stopped Permit User Sessions.
[  OK  ] Stopped LSB: exim Mail Transport Agent.
[  OK  ] Stopped target Basic System.
[  OK  ] Stopped target Paths.
[  OK  ] Stopped target Slices.
[  OK  ] Removed slice User and Session Slice.
[  OK  ] Stopped target Sockets.
[  OK  ] Closed Syslog Socket.
[  OK  ] Closed D-Bus System Message Bus Socket.
[  OK  ] Stopped target System Initialization.
         Stopping Load/Save Random Seed...
[  OK  ] Stopped target Swap.
         Deactivating swap /dev/mapper/debian--vg-swap_1...
         Stopping Update UTMP about System Boot/Shutdown...
[  OK  ] Stopped target Encrypted Volumes.
         Stopping Cryptography Setup for vda5_crypt...
[  OK  ] Stopped Dispatch Password Requests to Console Directory Watch.
[  OK  ] Stopped Forward Password Requests to Wall Directory Watch.
[  OK  ] Stopped target System Time Synchronized.
         Stopping Network Time Synchronization...
[  OK  ] Stopped target Remote File Systems.
[  OK  ] Stopped target Network is Online.
[  OK  ] Stopped target Network.
         Stopping Raise network interfaces...
         Stopping ifup for ens3...
[  OK  ] Deactivated swap /dev/disk/by-uuid/…d03e0-736a-48ab-afce-0864f64c6154.
[  OK  ] Deactivated swap /dev/disk/by-id/dm…mBErryep61K37oLHkYWnsMXUAEGfOlrCz.
[  OK  ] Deactivated swap /dev/disk/by-id/dm-name-debian--vg-swap_1.
[  OK  ] Deactivated swap /dev/debian-vg/swap_1.
[  OK  ] Deactivated swap /dev/dm-3.
[  OK  ] Stopped Network Time Synchronization.
[  OK  ] Stopped Load/Save Random Seed.
[  OK  ] Deactivated swap /dev/mapper/debian--vg-swap_1.
[  OK  ] Stopped Update UTMP about System Boot/Shutdown.
[  387.421061] systemd[1]: Stopped (with error) /dev/disk/by-id/dm-name-vda5_crypt.
[FAILED] Stopped (with error) /dev/disk/by-id/dm-name-vda5_crypt.
[  387.422586] systemd[1]: Stopped (with error) /dev/mapper/vda5_crypt.
[FAILED] Stopped (with error) /dev/mapper/vda5_crypt.
[  387.423929] systemd[1]: Stopped (with error) /sys/devices/virtual/block/dm-0.
[FAILED] Stopped (with error) /sys/devices/virtual/block/dm-0.
[  387.425630] systemd[1]: Stopped (with error) /dev/dm-0.
[FAILED] Stopped (with error) /dev/dm-0.
[FAILED] Stopped (with error) /dev/disk/by-i…0344c08681d8aec1645016-vda5_crypt.
[FAILED] Stopped (with error) /dev/disk/by-i…r-0srK-HqO4-eIyK-X1AF-wtox-W7SPa5.
[  OK  ] Stopped Create Volatile Files and Directories.
[  OK  ] Stopped Flush Journal to Persistent Storage.
[  OK  ] Stopped ifup for ens3.
[  OK  ] Stopped Raise network interfaces.
[  OK  ] Stopped target Local File Systems.
         Unmounting /home...
         Unmounting /boot...
         Unmounting /var...
         Unmounting /tmp...
         Unmounting /run/user/0...
[  OK  ] Stopped Apply Kernel Variables.
[  OK  ] Stopped Load Kernel Modules.
[  OK  ] Unmounted /home.
[  OK  ] Unmounted /boot.
[  OK  ] Unmounted /run/user/0.
[  OK  ] Stopped File System Check on /dev/d…70192-32b7-427a-a259-2842f242423d.
[  OK  ] Stopped File System Check on /dev/mapper/debian--vg-home.
[  OK  ] Unmounted /var.
[  OK  ] Unmounted /tmp.
[  OK  ] Stopped File System Check on /dev/mapper/debian--vg-tmp.
[  OK  ] Stopped File System Check on /dev/mapper/debian--vg-var.
[  OK  ] Removed slice system-systemd\x2dfsck.slice.
[  OK  ] Stopped target Local File Systems (Pre).
         Stopping Monitoring of LVM2 mirrors…ng dmeventd or progress polling...
[  OK  ] Stopped Create Static Device Nodes in /dev.
[  OK  ] Stopped Remount Root and Kernel File Systems.
[  OK  ] Stopped Monitoring of LVM2 mirrors,…sing dmeventd or progress polling.
         Stopping LVM2 metadata daemon...
[  OK  ] Stopped LVM2 metadata daemon.
[  OK  ] Stopped Cryptography Setup for vda5_crypt.
[  OK  ] Reached target Unmount All Filesystems.
[  OK  ] Removed slice system-systemd\x2dcryptsetup.slice.
[  OK  ] Reached target Shutdown.
[  392.185070] systemd-shutdow: 36 output lines suppressed due to ratelimiting
[  392.198292] systemd-shutdown[1]: Sending SIGTERM to remaining processes...
[  392.204935] systemd-journald[289]: Received SIGTERM from PID 1 (systemd-shutdow).
[  392.208084] systemd-shutdown[1]: Sending SIGKILL to remaining processes...
[  392.212411] systemd-shutdown[1]: Unmounting file systems.
[  392.214006] systemd-shutdown[1]: Remounting '/' read-only with options 'errors=remount-ro,data=ordered'.
[  392.227597] EXT4-fs (dm-1): re-mounted. Opts: errors=remount-ro,data=ordered
[  392.230965] systemd-shutdown[1]: Remounting '/' read-only with options 'errors=remount-ro,data=ordered'.
[  392.233194] EXT4-fs (dm-1): re-mounted. Opts: errors=remount-ro,data=ordered
[  392.234766] systemd-shutdown[1]: All filesystems unmounted.
[  392.236036] systemd-shutdown[1]: Deactivating swaps.
[  392.237193] systemd-shutdown[1]: All swaps deactivated.
[  392.238351] systemd-shutdown[1]: Detaching loop devices.
[  392.240402] systemd-shutdown[1]: All loop devices detached.
[  392.328723] reboot: System halted

@mbiebl mbiebl added cryptsetup bug labels Mar 3, 2017

@poettering
Owner

poettering commented Mar 3, 2017

You appear to be using LVM. As soon as LVM is involved we can't really help you upstream. Please contact the LVM folks.

(Do note though that #867 remains an issue, hence if you split off /var into a separate file system, than it will result in unmount errors that mostly cosmetical, and evreything you stacked below it the same.)

(Also note: if your root partition is on complex storage, then the backing processes for it need to be part of the initrd and stay around until final shutdown. It appears from your logs this isn't the case for you: https://www.freedesktop.org/wiki/Software/systemd/RootStorageDaemons/)

Either way, please direct LVM questions to LVM people, it's a complex piece of software that interferes with systemd's and udev's usual ways in non-trivial ways, and we really can't support that here. Thank you for understanding.

@poettering poettering closed this Mar 3, 2017

@mbiebl
Contributor

mbiebl commented Mar 3, 2017

Well, the problem doesn't seem to be LVM but rather the user of LUKS/cryptsetup and it is systemd which ships a cryptsetup generator (which I suspect generates wrong dependencies)
@mbiebl
Contributor

mbiebl commented Mar 4, 2017

So, I've just upgraded that test VM to v233 and the errors were gone on shutdown.
Running a git bisect showed that it was fixed by c5a97ed

Posting it here, just in case someone else runs into this problem as well
@arvidjaar
Contributor

arvidjaar commented Mar 5, 2017

@mbiebl

Running a git bisect showed that it was fixed by c5a97ed

Does it also fix #1620?
@mbiebl
Contributor

mbiebl commented Mar 5, 2017

@arvidjaar you probably need to ask the bug submitter

@mbiebl mbiebl referenced this issue Jul 5, 2017

Open

Stopped (with error) /dev/dm-1 #1620

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment