OpenZFS on Linux and FreeBSD
Go to file
Alexander Motin 82e3117095 Optimize microzaps
Microzap on-disk format does not include a hash tree, expecting one to
be built in RAM during mzap_open().  The built tree is linked to DMU
user buffer, freed when original DMU buffer is dropped from cache. I've
found that workloads accessing many large directories and having active
eviction from DMU cache spend significant amount of time building and
then destroying the trees.  I've also found that for each 64 byte mzap
element additional 64 byte tree element is allocated, that is a waste
of memory and CPU caches.

Improve memory efficiency of the hash tree by switching from AVL-tree
to B-tree.  It allows to save 24 bytes per element just on pointers.
Save 32 bits on mze_hash by storing only upper 32 bits since lower 32
bits are always zero for microzaps.  Save 16 bits on mze_chunkid, since
microzap can never have so many elements.  Respectively with the 16 bits
there can be no more than 16 bits of collision differentiators.  As
result, struct mzap_ent now drops from 48 (rounded to 64) to 8 bytes.

Tune B-trees for small data.  Reduce BTREE_CORE_ELEMS from 128 to 126
to allow struct zfs_btree_core in case of 8 byte elements to pack into
2KB instead of 4KB.  Aside of the microzaps it should also help 32bit
range trees.  Allow custom B-tree leaf size to reduce memmove() time.

Split zap_name_alloc() into zap_name_alloc() and zap_name_init_str().
It allows to not waste time allocating/freeing memory when processing
multiple names in a loop during mzap_open().

Together on a pool with 10K directories of 1800 files each and DMU
cache limited to 128MB this reduces time of `find . -name zzz` by 41%
from 7.63s to 4.47s, and saves additional ~30% of CPU time on the DMU
cache reclamation.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Signed-off-by:	Alexander Motin <mav@FreeBSD.org>
Sponsored by:	iXsystems, Inc.
Closes 

(cherry picked from commit 9dcdee7889)
2023-03-02 14:37:07 -08:00
.github Use Ubuntu 20.04 and remove Ubuntu 18.04 from workflows 2022-12-09 10:57:10 -08:00
cmd Change ZEVENT_POOL_GUID to ZEVENT_POOL to display pool names 2023-01-19 12:50:36 -08:00
config autoconf: add support for openEuler 2023-03-01 14:41:19 -08:00
contrib initramfs: Make mountpoint=none work 2023-02-09 12:43:03 -08:00
etc systemd: set restart=always for zfs-zed.service 2023-01-19 12:50:42 -08:00
include Optimize microzaps 2023-03-02 14:37:07 -08:00
lib Add color output to zfs diff. 2023-01-19 12:50:36 -08:00
man Add color output to zfs diff. 2023-01-19 12:50:36 -08:00
module Optimize microzaps 2023-03-02 14:37:07 -08:00
rpm rpm: add support for openEuler 2023-03-01 14:41:19 -08:00
scripts scripts/enum-extract.pl should not hard code perl path 2022-12-01 12:39:41 -08:00
tests Activate filesystem features only in syncing context 2023-01-19 12:50:42 -08:00
udev Udev rules: use match (==) rather than assign (=) for PROGRAM 2021-09-14 12:23:10 -07:00
.editorconfig Add an .editorconfig; document git whitespace settings 2020-01-27 13:32:52 -08:00
.gitignore Add FreeBSD support to OpenZFS 2020-04-14 11:36:28 -07:00
.gitmodules .gitmodules: link to openzfs github repository 2021-04-14 13:23:08 -07:00
AUTHORS Add zstd support to zfs 2020-08-20 10:30:06 -07:00
CODE_OF_CONDUCT.md Documentation corrections 2023-01-19 12:50:42 -08:00
COPYRIGHT Fix typos 2020-06-09 21:24:09 -07:00
LICENSE Update build system and packaging 2018-05-29 16:00:33 -07:00
META Tag zfs-2.1.9 2023-01-24 15:41:54 -08:00
Makefile.am config: check for parallel(1), use it for cstyle 2022-09-14 11:23:25 -07:00
NEWS Fix NEWS file 2020-08-26 21:44:41 -07:00
NOTICE Update build system and packaging 2018-05-29 16:00:33 -07:00
README.md README: Update OpenZFS website url 2022-02-16 17:58:55 -08:00
RELEASES.md Add RELEASES.md file 2021-04-07 13:26:58 -07:00
TEST Remove CI builder customization from TEST 2020-03-16 10:46:03 -07:00
autogen.sh Cause autogen.sh to fail if autoreconf fails 2018-07-06 09:27:37 -07:00
configure.ac Linux 5.16 compat: don't use XSTATE_XSAVE to save FPU state 2022-02-16 17:58:55 -08:00
copy-builtin copy-builtin: posix conformance 2021-05-10 12:18:54 -07:00
zfs.release.in Move zfs.release generation to configure step 2012-07-12 12:22:51 -07:00

README.md

img

OpenZFS is an advanced file system and volume manager which was originally developed for Solaris and is now maintained by the OpenZFS community. This repository contains the code for running OpenZFS on Linux and FreeBSD.

codecov coverity

Official Resources

Installation

Full documentation for installing OpenZFS on your favorite operating system can be found at the Getting Started Page.

Contribute & Develop

We have a separate document with contribution guidelines.

We have a Code of Conduct.

Release

OpenZFS is released under a CDDL license. For more details see the NOTICE, LICENSE and COPYRIGHT files; UCRL-CODE-235197

Supported Kernels

  • The META file contains the officially recognized supported Linux kernel versions.
  • Supported FreeBSD versions are any supported branches and releases starting from 12.2-RELEASE.