zfs/include/sys
Tom Caputi d4a72f2386 Sequential scrub and resilvers
Currently, scrubs and resilvers can take an extremely
long time to complete. This is largely due to the fact
that zfs scans process pools in logical order, as
determined by each block's bookmark. This makes sense
from a simplicity perspective, but blocks in zfs are
often scattered randomly across disks, particularly
due to zfs's copy-on-write mechanisms.

This patch improves performance by splitting scrubs
and resilvers into a metadata scanning phase and an IO
issuing phase. The metadata scan reads through the
structure of the pool and gathers an in-memory queue
of I/Os, sorted by size and offset on disk. The issuing
phase will then issue the scrub I/Os as sequentially as
possible, greatly improving performance.

This patch also updates and cleans up some of the scan
code which has not been updated in several years.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Authored-by: Saso Kiselkov <saso.kiselkov@nexenta.com>
Authored-by: Alek Pinchuk <apinchuk@datto.com>
Authored-by: Tom Caputi <tcaputi@datto.com>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Closes #3625 
Closes #6256
2017-11-15 17:27:01 -08:00
..
crypto
fm Native Encryption for ZFS on Linux 2017-08-14 10:36:48 -07:00
fs Sequential scrub and resilvers 2017-11-15 17:27:01 -08:00
sysevent Emit history events for 'zpool create' 2017-10-23 09:45:59 -07:00
Makefile.am Encryption patch follow-up 2017-10-11 16:54:48 -04:00
abd.h OpenZFS 8416 - abd.h is not C++ friendly 2017-06-30 11:11:01 -07:00
arc.h Sequential scrub and resilvers 2017-11-15 17:27:01 -08:00
arc_impl.h Fixes for #6639 2017-10-11 16:55:50 -04:00
avl.h Remove dead code from AVL tree 2017-10-05 19:28:00 -07:00
avl_impl.h
blkptr.h OpenZFS 8067 - zdb should be able to dump literal embedded block pointer 2017-07-07 11:28:01 -07:00
bplist.h
bpobj.h
bptree.h
bqueue.h
dbuf.h OpenZFS 7531 - Assign correct flags to prefetched buffers 2017-11-11 20:24:34 -08:00
ddt.h Native Encryption for ZFS on Linux 2017-08-14 10:36:48 -07:00
dmu.h Fixes for #6639 2017-10-11 16:55:50 -04:00
dmu_impl.h OpenZFS 7793 - ztest fails assertion in dmu_tx_willuse_space 2017-03-07 09:51:59 -08:00
dmu_objset.h Native Encryption for ZFS on Linux 2017-08-14 10:36:48 -07:00
dmu_send.h Free objects when receiving full stream as clone 2017-10-10 15:30:51 -07:00
dmu_traverse.h Native Encryption for ZFS on Linux 2017-08-14 10:36:48 -07:00
dmu_tx.h OpenZFS 7793 - ztest fails assertion in dmu_tx_willuse_space 2017-03-07 09:51:59 -08:00
dmu_zfetch.h
dnode.h Improved dnode allocation and dmu_hold_impl() 2017-09-05 16:15:04 -07:00
dsl_bookmark.h
dsl_crypt.h Native Encryption for ZFS on Linux 2017-08-14 10:36:48 -07:00
dsl_dataset.h Typo in dsl_dataset.h 2017-10-12 17:10:38 -07:00
dsl_deadlist.h
dsl_deleg.h Native Encryption for ZFS on Linux 2017-08-14 10:36:48 -07:00
dsl_destroy.h
dsl_dir.h Native Encryption for ZFS on Linux 2017-08-14 10:36:48 -07:00
dsl_pool.h Sequential scrub and resilvers 2017-11-15 17:27:01 -08:00
dsl_prop.h
dsl_scan.h Sequential scrub and resilvers 2017-11-15 17:27:01 -08:00
dsl_synctask.h
dsl_userhold.h
edonr.h
efi_partition.h Fix spelling 2017-01-03 11:31:18 -06:00
hkdf.h Encryption patch follow-up 2017-10-11 16:54:48 -04:00
metaslab.h OpenZFS 7303 - dynamic metaslab selection 2017-01-12 11:52:56 -08:00
metaslab_impl.h OpenZFS 7613 - ms_freetree[4] is only used in syncing context 2017-01-26 15:27:19 -08:00
mmp.h Add callback for zfs_multihost_interval 2017-07-25 13:22:20 -04:00
mntent.h
multilist.h OpenZFS 7968 - multi-threaded spa_sync() 2017-03-20 18:36:00 -07:00
nvpair.h
nvpair_impl.h
pathname.h
policy.h
range_tree.h Sequential scrub and resilvers 2017-11-15 17:27:01 -08:00
refcount.h OpenZFS 8081 - Compiler warnings in zdb 2017-10-27 12:46:35 -07:00
rrwlock.h
sa.h Remove unused sa_update_from_cb() 2016-12-01 16:39:06 -07:00
sa_impl.h
sdt.h Add line info and SET_ERROR() to ZFS debug log 2017-07-25 23:09:48 -07:00
sha2.h
skein.h
spa.h Undo c89 workarounds to match with upstream 2017-11-04 13:25:13 -07:00
spa_boot.h
spa_checksum.h
spa_impl.h Sequential scrub and resilvers 2017-11-15 17:27:01 -08:00
space_map.h
space_reftree.h
sysevent.h OpenZFS 6939 - add sysevents to zfs core for commands 2017-07-12 21:28:13 -07:00
trace.h
trace_acl.h Fix build-it compilation regression 2017-01-24 08:50:15 -08:00
trace_arc.h Fix build-it compilation regression 2017-01-24 08:50:15 -08:00
trace_common.h
trace_dbgmsg.h Add line info and SET_ERROR() to ZFS debug log 2017-07-25 23:09:48 -07:00
trace_dbuf.h Crash in dbuf_evict_one with DTRACE_PROBE 2017-08-09 11:04:41 -07:00
trace_dmu.h OpenZFS 7793 - ztest fails assertion in dmu_tx_willuse_space 2017-03-07 09:51:59 -08:00
trace_dnode.h Fix build-it compilation regression 2017-01-24 08:50:15 -08:00
trace_multilist.h Fix build-it compilation regression 2017-01-24 08:50:15 -08:00
trace_txg.h Fix build-it compilation regression 2017-01-24 08:50:15 -08:00
trace_zil.h OpenZFS 7578 - Fix/improve some aspects of ZIL writing 2017-06-09 09:15:37 -07:00
trace_zio.h Use cstyle -cpP in `make cstyle` check 2016-12-12 10:46:26 -08:00
trace_zrlock.h Use cstyle -cpP in `make cstyle` check 2016-12-12 10:46:26 -08:00
txg.h OpenZFS 8063 - verify that we do not attempt to access inactive txg 2017-05-10 13:52:22 -04:00
txg_impl.h Fix spelling 2017-01-03 11:31:18 -06:00
u8_textprep.h
u8_textprep_data.h
uberblock.h Multi-modifier protection (MMP) 2017-07-13 13:54:00 -04:00
uberblock_impl.h OpenZFS 8491 - uberblock on-disk padding to reserve space for smoothly merging zpool checkpoint & MMP in ZFS 2017-07-24 13:47:51 -04:00
uio_impl.h
unique.h
uuid.h
vdev.h vdev_mirror: load balancing fixes 2017-08-21 10:05:16 -07:00
vdev_disk.h
vdev_file.h Use a dedicated taskq for vdev_file 2016-12-21 10:47:15 -08:00
vdev_impl.h Sequential scrub and resilvers 2017-11-15 17:27:01 -08:00
vdev_raidz.h Use cstyle -cpP in `make cstyle` check 2016-12-12 10:46:26 -08:00
vdev_raidz_impl.h Use cstyle -cpP in `make cstyle` check 2016-12-12 10:46:26 -08:00
xvattr.h Fix spelling 2017-01-03 11:31:18 -06:00
zap.h OpenZFS 1300 - filename normalization doesn't work for removes 2017-02-02 14:13:41 -08:00
zap_impl.h OpenZFS 7793 - ztest fails assertion in dmu_tx_willuse_space 2017-03-07 09:51:59 -08:00
zap_leaf.h
zfeature.h
zfs_acl.h Rename zfs_sb_t -> zfsvfs_t 2017-03-10 09:51:33 -08:00
zfs_context.h OpenZFS 640 - number_to_scaled_string is duplicated in several commands 2017-10-30 14:47:20 -07:00
zfs_ctldir.h Rename zfs_sb_t -> zfsvfs_t 2017-03-10 09:51:33 -08:00
zfs_debug.h Add line info and SET_ERROR() to ZFS debug log 2017-07-25 23:09:48 -07:00
zfs_delay.h
zfs_dir.h Rename zfs_sb_t -> zfsvfs_t 2017-03-10 09:51:33 -08:00
zfs_fuid.h Rename zfs_sb_t -> zfsvfs_t 2017-03-10 09:51:33 -08:00
zfs_ioctl.h Send / Recv Fixes following b52563 2017-08-23 16:54:24 -07:00
zfs_onexit.h
zfs_ratelimit.h Add missing *_destroy/*_fini calls 2017-05-04 19:26:28 -04:00
zfs_rlock.h Rename zfs_sb_t -> zfsvfs_t 2017-03-10 09:51:33 -08:00
zfs_sa.h
zfs_stat.h
zfs_vfsops.h Linux 4.12 compat: super_setup_bdi_name() 2017-05-02 09:46:18 -07:00
zfs_vnops.h Rename zfs_* functions 2017-03-10 09:51:35 -08:00
zfs_znode.h OpenZFS 8081 - Compiler warnings in zdb 2017-10-27 12:46:35 -07:00
zil.h OpenZFS 8081 - Compiler warnings in zdb 2017-10-27 12:46:35 -07:00
zil_impl.h OpenZFS 8558, 8602 - lwp_create() returns EAGAIN 2017-10-26 12:57:53 -07:00
zio.h Add corruption failure option to zinject(8) 2017-08-14 15:17:15 -07:00
zio_checksum.h Remove dependency on linear ABD 2017-03-29 12:24:51 -07:00
zio_compress.h DLPX-44812 integrate EP-220 large memory scalability 2016-11-29 14:34:27 -08:00
zio_crypt.h Encryption patch follow-up 2017-10-11 16:54:48 -04:00
zio_impl.h Native Encryption for ZFS on Linux 2017-08-14 10:36:48 -07:00
zio_priority.h
zpl.h Use cstyle -cpP in `make cstyle` check 2016-12-12 10:46:26 -08:00
zrlock.h OpenZFS 6328 - Fix cstyle errors in zfs codebase 2017-01-12 09:42:11 -08:00
zvol.h Add port of FreeBSD 'volmode' property 2017-07-12 13:05:37 -07:00