600a02b884
Previous flushing algorithm limited only total number of log blocks to the minimum of 256K and 4x number of metaslabs in the pool. As result, system with 1500 disks with 1000 metaslabs each, touching several new metaslabs each TXG could grow spacemap log to huge size without much benefits. We've observed one of such systems importing pool for about 45 minutes. This patch improves the situation from five sides: - By limiting maximum period for each metaslab to be flushed to 1000 TXGs, that effectively limits maximum number of per-TXG spacemap logs to load to the same number. - By making flushing more smooth via accounting number of metaslabs that were touched after the last flush and actually need another flush, not just ms_unflushed_txg bump. - By applying zfs_unflushed_log_block_pct to the number of metaslabs that were touched after the last flush, not all metaslabs in the pool. - By aggressively prefetching per-TXG spacemap logs up to 16 TXGs in advance, making log spacemap load process for wide HDD pool CPU-bound, accelerating it by many times. - By reducing zfs_unflushed_log_block_max from 256K to 128K, reducing single-threaded by nature log processing time from ~10 to ~5 minutes. As further optimization we could skip bumping ms_unflushed_txg for metaslabs not touched since the last flush, but that would be an incompatible change, requiring new pool feature. Reviewed-by: Matthew Ahrens <mahrens@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Alexander Motin <mav@FreeBSD.org> Sponsored-By: iXsystems, Inc. Closes #12789 |
||
|---|---|---|
| .github | ||
| cmd | ||
| config | ||
| contrib | ||
| etc | ||
| include | ||
| lib | ||
| man | ||
| module | ||
| rpm | ||
| scripts | ||
| tests | ||
| udev | ||
| .editorconfig | ||
| .gitignore | ||
| .gitmodules | ||
| AUTHORS | ||
| CODE_OF_CONDUCT.md | ||
| COPYRIGHT | ||
| LICENSE | ||
| META | ||
| Makefile.am | ||
| NEWS | ||
| NOTICE | ||
| README.md | ||
| RELEASES.md | ||
| TEST | ||
| autogen.sh | ||
| configure.ac | ||
| copy-builtin | ||
| zfs.release.in | ||
README.md
OpenZFS is an advanced file system and volume manager which was originally developed for Solaris and is now maintained by the OpenZFS community. This repository contains the code for running OpenZFS on Linux and FreeBSD.
Official Resources
- Documentation - for using and developing this repo
- ZoL Site - Linux release info & links
- Mailing lists
- OpenZFS site - for conference videos and info on other platforms (illumos, OSX, Windows, etc)
Installation
Full documentation for installing OpenZFS on your favorite operating system can be found at the Getting Started Page.
Contribute & Develop
We have a separate document with contribution guidelines.
We have a Code of Conduct.
Release
OpenZFS is released under a CDDL license.
For more details see the NOTICE, LICENSE and COPYRIGHT files; UCRL-CODE-235197
Supported Kernels
- The
METAfile contains the officially recognized supported Linux kernel versions. - Supported FreeBSD versions are any supported branches and releases starting from 12.2-RELEASE.