cd69ba3d49
Adds a log/journal to dedup. At the end of txg, instead of writing the entry directly to the ZAP, instead its adding to an in-memory tree and appended to an on-disk object. The on-disk object is only read at import, to reload the in-memory tree. Lookups first go the the log tree before going to the ZAP, so recently-used entries will remain close by in memory. This vastly reduces overhead from dedup IO, as it will not have to do so many read/update/write cycles on ZAP leaf nodes. A flushing facility is added at end of txg, to push logged entries out to the ZAP. There's actually two separate "logs" (in-memory tree and on-disk object), one active (recieving updated entries) and one flushing (writing out to disk). These are swapped (ie flushing begins) based on memory used by the in-memory log trees and time since we last flushed something. The flushing facility monitors the amount of entries coming in and being flushed out, and calibrates itself to try to flush enough each txg to keep up with the ingest rate without competing too much with other IO. Multiple tuneables are provided to control the flushing facility. All the histograms and stats are update to accomodate the log as a separate entry store. zdb gains knowledge of how to count them and dump them. Documentation included! Reviewed-by: Alexander Motin <mav@FreeBSD.org> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Co-authored-by: Allan Jude <allan@klarasystems.com> Signed-off-by: Rob Norris <rob.norris@klarasystems.com> Sponsored-by: Klara, Inc. Sponsored-by: iXsystems, Inc. Closes #15895 |
||
|---|---|---|
| .github | ||
| cmd | ||
| config | ||
| contrib | ||
| etc | ||
| include | ||
| lib | ||
| man | ||
| module | ||
| rpm | ||
| scripts | ||
| tests | ||
| udev | ||
| .cirrus.yml | ||
| .editorconfig | ||
| .gitignore | ||
| .gitmodules | ||
| .mailmap | ||
| AUTHORS | ||
| CODE_OF_CONDUCT.md | ||
| COPYRIGHT | ||
| LICENSE | ||
| META | ||
| Makefile.am | ||
| NEWS | ||
| NOTICE | ||
| README.md | ||
| RELEASES.md | ||
| TEST | ||
| autogen.sh | ||
| configure.ac | ||
| copy-builtin | ||
| zfs.release.in | ||
README.md
OpenZFS is an advanced file system and volume manager which was originally developed for Solaris and is now maintained by the OpenZFS community. This repository contains the code for running OpenZFS on Linux and FreeBSD.
Official Resources
- Documentation - for using and developing this repo
- ZoL Site - Linux release info & links
- Mailing lists
- OpenZFS site - for conference videos and info on other platforms (illumos, OSX, Windows, etc)
Installation
Full documentation for installing OpenZFS on your favorite operating system can be found at the Getting Started Page.
Contribute & Develop
We have a separate document with contribution guidelines.
We have a Code of Conduct.
Release
OpenZFS is released under a CDDL license.
For more details see the NOTICE, LICENSE and COPYRIGHT files; UCRL-CODE-235197
Supported Kernels
- The
METAfile contains the officially recognized supported Linux kernel versions. - Supported FreeBSD versions are any supported branches and releases starting from 13.0-RELEASE.