ensure libguestfs mounts are cleaned up

Make a few attempts to umount the libguestfs
mounts, in case other processes are accessing
the mounts (like udev etc.)

If we still can't umount after 5 attempts
(with average delay of about 1s between each),
then initiate a lazy umount so that the mounts
might be automatically cleaned up at some stage.
We wait a further 10s after initiating the lazy
umount, before raising an exception.

Addresses the original issue in bug 1013689

Change-Id: Ib5ede9f705c833825a19308c140f99c5bf3a776f
(cherry picked from commit 5eea887)

nova/virt/disk/guestfs.py

@@ -17,6 +17,7 @@
 
 import os
 
+from nova import exception
 from nova import utils
 from nova.virt.disk import mount
 
@@ -88,9 +89,18 @@ def mnt_dev(self):
     def unmnt_dev(self):
         if not self.mounted:
             return
-        # root users don't need a specific unmnt_dev()
-        # but ordinary users do
-        utils.execute('fusermount', '-u', self.mount_dir, run_as_root=True)
+        umount_cmd = ['fusermount', '-u', self.mount_dir]
+        try:
+            # We make a few attempts to work around other
+            # processes temporarily scanning the mount_dir etc.
+            utils.execute(*umount_cmd, attempts=5, run_as_root=True)
+        except exception.ProcessExecutionError:
+            # If we still can't umount, then do a lazy umount
+            # (in the background), so that mounts might eventually
+            # be cleaned up. Note we'll wait 10s below for the umount to
+            # complete, after which we'll raise an exception.
+            umount_cmd.insert(1, '-z')
+            utils.execute(*umount_cmd, run_as_root=True)
 
         # Unfortunately FUSE has an issue where it doesn't wait
         # for processes associated with the mount to terminate.