zfs/lib/libzfs
Rob Norris db2b1fdb79 ddt: add FDT feature and support for legacy and new on-disk formats
This is the supporting infrastructure for the upcoming dedup features.

Traditionally, dedup objects live directly in the MOS root. While their
details vary (checksum, type and class), they are all the same "kind" of
thing - a store of dedup entries.

The new features are more varied than that, and are better thought of as
a set of related stores for the overall state of a dedup table.

This adds a new feature flag, SPA_FEATURE_FAST_DEDUP. Enabling this will
cause new DDTs to be created as a ZAP in the MOS root, named
DDT-<checksum>. The is used as the root object for the normal type/class
store objects, but will also be a place for any storage required by new
features.

This commit adds two new fields to ddt_t, for version and flags. These
are intended to describe the structure and features of the overall dedup
table, and are stored as-is in the DDT root. In this commit, flags are
always zero, but the intent is that they can be used to hang optional
logic or state onto for new dedup features. Version is always 1.

For a "legacy" dedup table, where no DDT root directory exists, the
version will be 0.

ddt_configure() is expected to determine the version and flags features
currently in operation based on whether or not the fast_dedup feature is
enabled, and from what's available on disk. In this way, its possible to
support both old and new tables.

This also provides a migration path. A legacy setup can be upgraded to
FDT by creating the DDT root ZAP, moving the existing objects into it,
and setting version and flags appropriately. There's no support for that
here, but it would be straightforward to add later and allows the
possibility that newer features could be applied to existing dedup
tables.

Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Co-authored-by: Allan Jude <allan@klarasystems.com>
Signed-off-by: Rob Norris <rob.norris@klarasystems.com>
Sponsored-by: Klara, Inc.
Sponsored-by: iXsystems, Inc.
Closes #15892
2024-08-16 11:58:59 -07:00
..
os Replace P2ALIGN with P2ALIGN_TYPED and delete P2ALIGN. 2024-05-10 08:47:21 -07:00
.gitignore Clean up lib dependencies 2020-07-10 14:26:00 -07:00
Makefile.am libzfs: sendrecv: send_progress_thread: handle SIGINFO/SIGUSR1 2023-08-08 09:35:35 -07:00
THIRDPARTYLICENSE.openssl Fix typos in lib/ 2019-09-02 17:53:27 -07:00
THIRDPARTYLICENSE.openssl.descrip Encryption patch follow-up 2017-10-11 16:54:48 -04:00
libzfs.abi ddt: add FDT feature and support for legacy and new on-disk formats 2024-08-16 11:58:59 -07:00
libzfs.pc.in Spruce up pkg-config files for libzfs/libzfs_core 2020-09-04 11:11:18 -07:00
libzfs.suppr Library ABI tracking with abigail 2020-11-17 09:18:52 -08:00
libzfs_changelist.c Add '-u' - nomount flag for zfs set 2023-10-02 16:58:54 -07:00
libzfs_config.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
libzfs_crypto.c libzfs: use zfs_strerror() in place of strerror() 2024-01-29 09:54:57 -08:00
libzfs_dataset.c Overflowing refreservation is bad 2024-04-29 11:32:49 -07:00
libzfs_diff.c libzfs: use zfs_strerror() in place of strerror() 2024-01-29 09:54:57 -08:00
libzfs_impl.h ddt: add support for prefetching tables into the ARC 2024-07-26 09:16:18 -07:00
libzfs_import.c Fix "out of memory" error 2024-01-12 12:35:29 -08:00
libzfs_iter.c libzfs: add v2 iterator interfaces 2023-04-10 11:53:02 -07:00
libzfs_mount.c Better control the thread pool size when mounting datasets 2024-05-14 09:36:21 -07:00
libzfs_pool.c Fix incorrect error report on vdev attach/replace 2024-08-15 12:39:44 -07:00
libzfs_sendrecv.c Fix printf typo for `zfs receive -cv` (#16295) 2024-07-17 17:18:12 -07:00
libzfs_status.c nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
libzfs_util.c JSON output support for zpool get 2024-08-06 12:47:00 -07:00