OpenZFS on Linux and FreeBSD
Go to file
Matthew Ahrens 66ccc9b75f zdb leak detection fails with in-progress device removal
When a device removal is in progress, there are 2 locations for the data
that's already been moved: the original location, on the device that's
being removed; and the new location, which is pointed to by the indirect
mapping.  When doing leak detection, zdb needs to know about both
locations.  To determine what's already been copied, we load the
spacemaps of the removing vdev, omit the blocks that are yet to be
copied, and then use the vdev's remap op to find the new location.

The problem is with an optimization to the spacemap-loading code in zdb.
When processing the log spacemaps, we ignore entries that are not
relevant because they are past the point that's been copied.  However,
entries which span the point that's been copied (i.e. they are partly
relevant and partly irrelevant) are processed normally.  This can lead
to an illegal spacemap operation, for example if offsets up to 100KB
have been copied, and the spacemap log has the following entries:

	ALLOC 50KB-150KB (partly relevant)
	FREE 50KB-100KB (entirely relevant)
	FREE 100KB-150KB (entirely irrlevant - ignored)
	ALLOC 50KB-150KB (partly relevant)

Because the entirely irrelevant entry was ignored, its space remains in
the spacemap.  When the last entry is processed, we attempt to add it to
the spacemap, but it partially overlaps with the 100-150KB entry that
was left over.

This problem was discovered by ztest/zloop.

One solution would be to also ignore the irrelevant parts of
partially-irrelevant entries (i.e. when processing the ALLOC 50-150, to
only add 50-100 to the spacemap).  However, this commit implements a
simpler solution, which is to remove this optimization entirely.  I.e.
to process the entire spacemap log, without regard for the point that's
been copied.  After reconstructing the entire allocatable range tree,
there's already code to remove the parts that have not yet been copied.

Reviewed-by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
External-issue: DLPX-71820
Closes #10920
2020-09-18 12:38:24 -07:00
.github Add Stale-bot 2020-08-24 12:04:38 -07:00
cmd zdb leak detection fails with in-progress device removal 2020-09-18 12:38:24 -07:00
config config/zfs-build.m4: never define _initramfs in RPM_DEFINE_UTIL 2020-09-15 18:36:03 -07:00
contrib cmd/zgenhostid: replace with simple c implementation 2020-09-18 12:37:54 -07:00
etc Fix another dependency loop 2020-08-30 14:18:45 -07:00
include Rename acltype=posixacl to acltype=posix 2020-09-18 12:38:00 -07:00
lib libzfsbootenv: lzbe_nvlist_set needs to store bootenv version VB_NVLIST 2020-09-18 12:38:04 -07:00
man Rename acltype=posixacl to acltype=posix 2020-09-18 12:38:00 -07:00
module FreeBSD: Do not copy vp into f_data for DTYPE_VNODE files 2020-09-18 12:38:14 -07:00
rpm zfs label bootenv should store data as nvlist 2020-09-15 18:36:12 -07:00
scripts Add a missing option prefix `-` in zfs-tests.sh usage() 2020-09-09 10:26:02 -07:00
tests Rename acltype=posixacl to acltype=posix 2020-09-18 12:38:00 -07:00
udev Centralize variable substitution 2020-07-14 17:33:44 -07:00
.editorconfig Add an .editorconfig; document git whitespace settings 2020-01-27 13:32:52 -08:00
.gitignore Add FreeBSD support to OpenZFS 2020-04-14 11:36:28 -07:00
.gitmodules Add zimport.sh compatibility test script 2014-02-21 12:10:31 -08:00
AUTHORS Add zstd support to zfs 2020-08-20 10:30:06 -07:00
CODE_OF_CONDUCT.md Add CODE_OF_CONDUCT.md 2019-04-30 10:58:45 -07:00
COPYRIGHT Fix typos 2020-06-09 21:24:09 -07:00
LICENSE Update build system and packaging 2018-05-29 16:00:33 -07:00
META Tag 2.0.0-rc1 2020-08-25 13:11:25 -07:00
Makefile.am Silence 'make checkbashisms' 2020-08-20 13:45:47 -07:00
NEWS Fix NEWS file 2020-08-27 16:07:01 -07:00
NOTICE Update build system and packaging 2018-05-29 16:00:33 -07:00
README.md Update wiki links with new address 2020-06-03 19:46:31 -07:00
TEST Remove CI builder customization from TEST 2020-03-16 10:46:03 -07:00
autogen.sh Cause autogen.sh to fail if autoreconf fails 2018-07-06 09:27:37 -07:00
configure.ac zfs label bootenv should store data as nvlist 2020-09-15 18:36:12 -07:00
copy-builtin Move zfs_gitrev.h to build directory 2020-06-24 18:19:28 -07:00
cppcheck-suppressions.txt Import ZStandard v1.4.5 2020-08-20 10:30:06 -07:00
zfs.release.in Move zfs.release generation to configure step 2012-07-12 12:22:51 -07:00

README.md

img

OpenZFS is an advanced file system and volume manager which was originally developed for Solaris and is now maintained by the OpenZFS community. This repository contains the code for running OpenZFS on Linux and FreeBSD.

codecov coverity

Official Resources

Installation

Full documentation for installing OpenZFS on your favorite Linux distribution can be found at the ZoL Site.

Contribute & Develop

We have a separate document with contribution guidelines.

We have a Code of Conduct.

Release

OpenZFS is released under a CDDL license. For more details see the NOTICE, LICENSE and COPYRIGHT files; UCRL-CODE-235197

Supported Kernels

  • The META file contains the officially recognized supported Linux kernel versions.
  • Supported FreeBSD versions are 12-STABLE and 13-CURRENT.