Prevent incorrect datasets being mounted

During a mount, zpl_mount_impl(), uses sget() with the callback
zpl_test_super() to find a super_block with a matching objset,
stored in z_os. It does so without taking the teardown lock on
the zfsvfs.

The problem is that operations like rollback will replace the
z_os. And, there is a window where the objset in the rollback
is freed, but z_os still points to it. Then, a mount like
operation, for instance a clone, can reallocate that exact same
pointer and zpl_test_super() will then match the super_block
associated with the rollback as opposed to the clone.

This fix tests for a match and if so, takes the teardown lock
before doing the final match test.

Reviewed-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: John Poduska <jpoduska@datto.com>
Closes #14518

module/os/linux/zfs/zpl_super.c

@@ -20,6 +20,7 @@
  */
 /*
  * Copyright (c) 2011, Lawrence Livermore National Security, LLC.
+ * Copyright (c) 2023, Datto Inc. All rights reserved.
  */
 
 
@@ -276,11 +277,28 @@ zpl_test_super(struct super_block *s, void *data)
 {
 	zfsvfs_t *zfsvfs = s->s_fs_info;
 	objset_t *os = data;
+	int match;
 
-	if (zfsvfs == NULL)
+	/*
+	 * If the os doesn't match the z_os in the super_block, assume it is
+	 * not a match. Matching would imply a multimount of a dataset. It is
+	 * possible that during a multimount, there is a simultaneous operation
+	 * that changes the z_os, e.g., rollback, where the match will be
+	 * missed, but in that case the user will get an EBUSY.
+	 */
+	if (zfsvfs == NULL || os != zfsvfs->z_os)
 		return (0);
 
-	return (os == zfsvfs->z_os);
+	/*
+	 * If they do match, recheck with the lock held to prevent mounting the
+	 * wrong dataset since z_os can be stale when the teardown lock is held.
+	 */
+	if (zpl_enter(zfsvfs, FTAG) != 0)
+		return (0);
+	match = (os == zfsvfs->z_os);
+	zpl_exit(zfsvfs, FTAG);
+
+	return (match);
 }
 
 static struct super_block *