Wait iput_async before evict_inodes to prevent race

Wait for iput_async before entering evict_inodes in
generic_shutdown_super. The reason we must finish before
evict_inodes is when lazytime is on, or when zfs_purgedir calls
zfs_zget, iput would bump i_count from 0 to 1. This would race
with the i_count check in evict_inodes.  This means it could
destroy the inode while we are still using it.

Signed-off-by: Chunwei Chen <david.chen@osnexus.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #4854

module/zfs/zfs_vfsops.c

@@ -1415,6 +1415,24 @@ zfs_preumount(struct super_block *sb)
 
 	if (zsb)
 		zfsctl_destroy(sb->s_fs_info);
+	/*
+	 * Wait for iput_async before entering evict_inodes in
+	 * generic_shutdown_super. The reason we must finish before
+	 * evict_inodes is when lazytime is on, or when zfs_purgedir calls
+	 * zfs_zget, iput would bump i_count from 0 to 1. This would race
+	 * with the i_count check in evict_inodes.  This means it could
+	 * destroy the inode while we are still using it.
+	 *
+	 * We wait for two passes.  xattr directories in the first pass may
+	 * add xattr entries in zfs_purgedir, so in the second pass we wait
+	 * for them.  We don't use taskq_wait here because it is a pool wide
+	 * taskq.  Other mounted filesystems can constantly do iput_async
+	 * and there's no guarantee when taskq will be empty.
+	 */
+	taskq_wait_outstanding(dsl_pool_iput_taskq(
+	    dmu_objset_pool(zsb->z_os)), 0);
+	taskq_wait_outstanding(dsl_pool_iput_taskq(
+	    dmu_objset_pool(zsb->z_os)), 0);
 }
 EXPORT_SYMBOL(zfs_preumount);