zfs/cmd
Matthew Ahrens a57f954226
zdb leak detection fails with in-progress device removal
When a device removal is in progress, there are 2 locations for the data
that's already been moved: the original location, on the device that's
being removed; and the new location, which is pointed to by the indirect
mapping.  When doing leak detection, zdb needs to know about both
locations.  To determine what's already been copied, we load the
spacemaps of the removing vdev, omit the blocks that are yet to be
copied, and then use the vdev's remap op to find the new location.

The problem is with an optimization to the spacemap-loading code in zdb.
When processing the log spacemaps, we ignore entries that are not
relevant because they are past the point that's been copied.  However,
entries which span the point that's been copied (i.e. they are partly
relevant and partly irrelevant) are processed normally.  This can lead
to an illegal spacemap operation, for example if offsets up to 100KB
have been copied, and the spacemap log has the following entries:

	ALLOC 50KB-150KB (partly relevant)
	FREE 50KB-100KB (entirely relevant)
	FREE 100KB-150KB (entirely irrlevant - ignored)
	ALLOC 50KB-150KB (partly relevant)

Because the entirely irrelevant entry was ignored, its space remains in
the spacemap.  When the last entry is processed, we attempt to add it to
the spacemap, but it partially overlaps with the 100-150KB entry that
was left over.

This problem was discovered by ztest/zloop.

One solution would be to also ignore the irrelevant parts of
partially-irrelevant entries (i.e. when processing the ALLOC 50-150, to
only add 50-100 to the spacemap).  However, this commit implements a
simpler solution, which is to remove this optimization entirely.  I.e.
to process the entire spacemap log, without regard for the point that's
been copied.  After reconstructing the entire allocatable range tree,
there's already code to remove the parts that have not yet been copied.

Reviewed-by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
External-issue: DLPX-71820
Closes #10920
2020-09-17 10:55:30 -07:00
..
arc_summary Add L2ARC arcstats for MFU/MRU buffers and buffer content type 2020-09-14 10:10:44 -07:00
arcstat Add L2ARC arcstats for MFU/MRU buffers and buffer content type 2020-09-14 10:10:44 -07:00
dbufstat Add zstd support to zfs 2020-08-20 10:30:06 -07:00
fsck_zfs Fix typos in cmd/ 2019-08-30 09:43:30 -07:00
mount_zfs Force the use of '.' as decimal separator. 2020-09-09 10:14:04 -07:00
raidz_test Unconditionally enable debugging for libzpool 2020-07-10 15:30:31 -07:00
vdev_id Silence 'make checkbashisms' 2020-08-20 13:45:47 -07:00
zdb zdb leak detection fails with in-progress device removal 2020-09-17 10:55:30 -07:00
zed ZED: Do not offline a missing device if no spare is available 2020-08-18 22:13:17 -07:00
zfs Force the use of '.' as decimal separator. 2020-09-09 10:14:04 -07:00
zfs_ids_to_path Use abs_top_builddir when referencing libraries 2020-07-10 14:26:32 -07:00
zgenhostid cmd/zgenhostid: replace with simple c implementation 2020-09-16 12:25:12 -07:00
zhack Unconditionally enable debugging for libzpool 2020-07-10 15:30:31 -07:00
zinject Use abs_top_builddir when referencing libraries 2020-07-10 14:26:32 -07:00
zpool Force the use of '.' as decimal separator. 2020-09-09 10:14:04 -07:00
zstream Add support to decode a resume token 2020-07-23 17:44:03 -07:00
zstreamdump Minor `zstream redup` command fixes 2020-04-10 21:10:09 -07:00
ztest Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
zvol_id OpenZFS restructuring - libspl 2019-10-02 10:39:48 -07:00
zvol_wait zvol_wait should ignore redacted zvols 2019-11-06 10:51:19 -08:00
Makefile.am Small program that converts a dataset id and an object id to a path 2020-05-20 10:05:33 -07:00