Archive-Team
/
zfs
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 103 Wiki Activity
zfs/include/sys
History
Attila Fülöp 31b160f0a6
ICP: Improve AES-GCM performance 
Currently SIMD accelerated AES-GCM performance is limited by two
factors:

a. The need to disable preemption and interrupts and save the FPU
state before using it and to do the reverse when done. Due to the
way the code is organized (see (b) below) we have to pay this price
twice for each 16 byte GCM block processed.

b. Most processing is done in C, operating on single GCM blocks.
The use of SIMD instructions is limited to the AES encryption of the
counter block (AES-NI) and the Galois multiplication (PCLMULQDQ).
This leads to the FPU not being fully utilized for crypto
operations.

To solve (a) we do crypto processing in larger chunks while owning
the FPU. An `icp_gcm_avx_chunk_size` module parameter was introduced
to make this chunk size tweakable. It defaults to 32 KiB. This step
alone roughly doubles performance. (b) is tackled by porting and
using the highly optimized openssl AES-GCM assembler routines, which
do all the processing (CTR, AES, GMULT) in a single routine. Both
steps together result in up to 32x reduction of the time spend in
the en/decryption routines, leading up to approximately 12x
throughput increase for large (128 KiB) blocks.

Lastly, this commit changes the default encryption algorithm from
AES-CCM to AES-GCM when setting the `encryption=on` property.

Reviewed-By: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-By: Jason King <jason.king@joyent.com>
Reviewed-By: Tom Caputi <tcaputi@datto.com>
Reviewed-By: Richard Laager <rlaager@wiktel.com>
Signed-off-by: Attila Fülöp <attila@fueloep.org>
Closes #9749
 		2020-02-10 12:59:50 -08:00
..
crypto Add support for selecting encryption backend 2018-08-02 11:59:24 -07:00
fm Add zpool status -s (slow I/Os) and -p (parseable) 2018-11-08 16:47:24 -08:00
fs Restore aclmode and remove acltype on FreeBSD 2020-02-04 08:40:07 -08:00
lua Fix typos in include/ 2019-08-30 09:53:15 -07:00
sysevent Add TRIM support 2019-03-29 09:13:20 -07:00
Makefile.am Relocate common quota functions to shared code 2019-12-11 12:12:08 -08:00
abd.h Minor diff reduction with ZoF in include/sys 2019-11-27 11:11:03 -08:00
aggsum.h Reduce number of atomic_add() calls in aggsum 2020-02-06 13:21:06 -08:00
arc.h Fix typos in include/ 2019-08-30 09:53:15 -07:00
arc_impl.h Fix for ARC sysctls ignored at runtime 2019-10-26 15:22:19 -07:00
avl.h Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
avl_impl.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
bitops.h Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
blkptr.h OpenZFS 8067 - zdb should be able to dump literal embedded block pointer 2017-07-07 11:28:01 -07:00
bplist.h Fast Clone Deletion 2019-07-26 10:54:14 -07:00
bpobj.h Fast Clone Deletion 2019-07-26 10:54:14 -07:00
bptree.h Illumos 4914 - zfs on-disk bookmark structure should be named *_phys_t 2014-08-06 14:48:41 -07:00
bqueue.h Implement Redacted Send/Receive 2019-06-19 09:48:12 -07:00
btree.h Function name and comment updates 2019-10-11 10:13:21 -07:00
cityhash.h OpenZFS 8484 - Implement aggregate sum and use for arc counters 2018-06-06 09:35:59 -07:00
dataset_kstats.h port async unlinked drain from illumos-nexenta 2019-02-12 10:41:15 -08:00
dbuf.h Few microoptimizations to dbuf layer 2020-02-05 11:08:44 -08:00
ddt.h Remove dedupditto functionality 2019-06-19 14:54:02 -07:00
dmu.h Minor diff reduction with ZoF in include/sys 2019-11-27 11:11:03 -08:00
dmu_impl.h Implement Redacted Send/Receive 2019-06-19 09:48:12 -07:00
dmu_objset.h Fast Clone Deletion 2019-07-26 10:54:14 -07:00
dmu_recv.h Fix errata #4 handling for resuming streams 2020-01-14 12:25:20 -08:00
dmu_redact.h Implement Redacted Send/Receive 2019-06-19 09:48:12 -07:00
dmu_send.h Add 'zfs send --saved' flag 2020-01-10 10:16:58 -08:00
dmu_traverse.h Implement Redacted Send/Receive 2019-06-19 09:48:12 -07:00
dmu_tx.h Linux 4.19-rc3+ compat: Remove refcount_t compat 2018-09-26 10:29:26 -07:00
dmu_zfetch.h Replace zf_rwlock with a mutex 2019-07-25 11:57:58 -07:00
dnode.h Fix zil replay panic when TX_REMOVE followed by TX_CREATE 2019-08-28 10:42:02 -07:00
dsl_bookmark.h Fix comments on zfs_bookmark_phys 2019-06-22 16:32:26 -07:00
dsl_crypt.h Allow unencrypted children of encrypted datasets 2019-06-20 12:29:51 -07:00
dsl_dataset.h Implement Redacted Send/Receive 2019-06-19 09:48:12 -07:00
dsl_deadlist.h Add fast path for zfs_ioc_space_snaps() handling of empty_bpobj 2019-08-20 11:34:52 -07:00
dsl_deleg.h Remove code for zfs remap 2019-06-24 16:44:01 -07:00
dsl_destroy.h Fast Clone Deletion 2019-07-26 10:54:14 -07:00
dsl_dir.h Fast Clone Deletion 2019-07-26 10:54:14 -07:00
dsl_pool.h Eliminate Linux specific inode usage from common code 2019-12-11 11:53:57 -08:00
dsl_prop.h Support inheriting properties in channel programs 2020-01-22 17:03:17 -08:00
dsl_scan.h Prevent unnecessary resilver restarts 2019-11-27 10:15:01 -08:00
dsl_synctask.h OpenZFS 9425 - channel programs can be interrupted 2019-06-22 16:51:46 -07:00
dsl_userhold.h Illumos #3740 2013-11-04 11:17:48 -08:00
edonr.h OpenZFS 4185 - add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R 2016-10-03 14:51:15 -07:00
efi_partition.h Fix typos in include/ 2019-08-30 09:53:15 -07:00
frame.h Suppress incorrect objtool warnings 2017-12-07 10:28:50 -08:00
hkdf.h Encryption patch follow-up 2017-10-11 16:54:48 -04:00
metaslab.h Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
metaslab_impl.h Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
mmp.h MMP interval and fail_intervals in uberblock 2019-03-21 12:47:57 -07:00
mntent.h Add FreeBSD required defines to mntent.h 2019-11-30 15:49:09 -08:00
mod.h Wrap Linux module macros 2019-11-01 10:41:03 -07:00
multilist.h Avoid extra taskq_dispatch() calls by DMU 2019-06-25 12:03:38 -07:00
note.h Update build system and packaging 2018-05-29 16:00:33 -07:00
nvpair.h Add new fnvlist_lookup_* functions 2018-10-03 15:30:55 -07:00
nvpair_impl.h OpenZFS 9580 - Add a hash-table on top of nvlist to speed-up operations 2018-07-30 11:30:03 -07:00
objlist.h Implement Redacted Send/Receive 2019-06-19 09:48:12 -07:00
pathname.h Disable unused pathname::pn_path* (unneeded in Linux) 2019-07-15 13:57:56 -07:00
qat.h QAT related bug fixes 2019-09-12 13:33:44 -07:00
range_tree.h Don't cast away const 2019-10-31 10:38:03 -07:00
refcount.h Prevent race in blkptr_verify against device removal 2019-08-13 21:24:43 -06:00
rrwlock.h OpenZFS restructuring - zfs_ioctl 2019-09-27 10:46:28 -07:00
sa.h Fix typos in include/ 2019-08-30 09:53:15 -07:00
sa_impl.h Linux 4.19-rc3+ compat: Remove refcount_t compat 2018-09-26 10:29:26 -07:00
skein.h OpenZFS 4185 - add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R 2016-10-03 14:51:15 -07:00
spa.h Refactor deadman set failmode to be cross platform 2019-12-05 12:40:45 -08:00
spa_boot.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
spa_checkpoint.h Serialize ZTHR operations to eliminate races 2019-01-13 10:09:46 -08:00
spa_checksum.h Implementation of AVX2 optimized Fletcher-4 2016-06-02 14:30:51 -07:00
spa_impl.h Refactor deadman set failmode to be cross platform 2019-12-05 12:40:45 -08:00
spa_log_spacemap.h Log Spacemap Project 2019-07-16 10:11:49 -07:00
space_map.h Log Spacemap Project 2019-07-16 10:11:49 -07:00
space_reftree.h Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
sysevent.h OpenZFS 6939 - add sysevents to zfs core for commands 2017-07-12 21:28:13 -07:00
txg.h OpenZFS 9425 - channel programs can be interrupted 2019-06-22 16:51:46 -07:00
txg_impl.h Fix typos in include/ 2019-08-30 09:53:15 -07:00
u8_textprep.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
u8_textprep_data.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
uberblock.h Multi-modifier protection (MMP) 2017-07-13 13:54:00 -04:00
uberblock_impl.h MMP interval and fail_intervals in uberblock 2019-03-21 12:47:57 -07:00
uio_impl.h deadlock between mm_sem and tx assign in zfs_write() and page fault 2018-10-16 11:11:24 -07:00
unique.h Illumos #3742 2013-11-04 10:55:25 -08:00
uuid.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
vdev.h Prevent unnecessary resilver restarts 2019-11-27 10:15:01 -08:00
vdev_disk.h Add support for autoexpand property 2018-07-23 15:40:15 -07:00
vdev_file.h Add zfs_file_* interface, remove vnodes 2019-11-21 09:32:57 -08:00
vdev_impl.h Add zfs_file_* interface, remove vnodes 2019-11-21 09:32:57 -08:00
vdev_indirect_births.h OpenZFS 7614, 9064 - zfs device evacuation/removal 2018-04-14 12:16:17 -07:00
vdev_indirect_mapping.h OpenZFS 7614, 9064 - zfs device evacuation/removal 2018-04-14 12:16:17 -07:00
vdev_initialize.h Add TRIM support 2019-03-29 09:13:20 -07:00
vdev_raidz.h Linux 5.0 compat: SIMD compatibility 2019-07-12 09:31:20 -07:00
vdev_raidz_impl.h Add AltiVec RAID-Z 2020-01-23 11:01:24 -08:00
vdev_removal.h panic in removal_remap test on 4K devices 2019-06-13 13:12:39 -07:00
vdev_trim.h Add TRIM support 2019-03-29 09:13:20 -07:00
xvattr.h Linux 4.18 compat: inode timespec -> timespec64 2018-06-19 21:51:18 -07:00
zap.h fat zap should prefetch when iterating 2019-06-12 13:13:09 -07:00
zap_impl.h OpenZFS 7793 - ztest fails assertion in dmu_tx_willuse_space 2017-03-07 09:51:59 -08:00
zap_leaf.h Fix ENOSPC in "Handle zap_add() failures in ..." 2018-04-18 14:19:50 -07:00
zcp.h async zvol minor node creation interferes with receive 2020-02-03 09:33:14 -08:00
zcp_global.h OpenZFS 7431 - ZFS Channel Programs 2018-02-08 15:28:18 -08:00
zcp_iter.h OpenZFS 7431 - ZFS Channel Programs 2018-02-08 15:28:18 -08:00
zcp_prop.h OpenZFS 7431 - ZFS Channel Programs 2018-02-08 15:28:18 -08:00
zfeature.h Revert "zhack: Add 'feature disable' command" 2016-05-17 11:52:07 -07:00
zfs_acl.h Return an error code from zfs_acl_chmod_setattr 2019-11-01 10:19:11 -07:00
zfs_context.h Refactor zfs_context.h to build on FreeBSD 2019-12-04 13:12:57 -08:00
zfs_debug.h Remove sdt.h 2019-10-25 13:38:37 -07:00
zfs_delay.h Update build system and packaging 2018-05-29 16:00:33 -07:00
zfs_file.h Add zfs_file_* interface, remove vnodes 2019-11-21 09:32:57 -08:00
zfs_fuid.h Relocate common quota functions to shared code 2019-12-11 12:12:08 -08:00
zfs_ioctl.h Resolve ZoF differences in zfs_ioctl.h 2019-11-30 15:35:54 -08:00
zfs_ioctl_impl.h Abstract away platform specific superblock references 2019-12-10 09:21:07 -08:00
zfs_onexit.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
zfs_project.h Minor diff reduction with ZoF in include/sys 2019-11-27 11:11:03 -08:00
zfs_quota.h Relocate common quota functions to shared code 2019-12-11 12:12:08 -08:00
zfs_ratelimit.h Change checksum & IO delay ratelimit values 2018-03-04 17:34:51 -08:00
zfs_rlock.h Prefix struct rangelock 2019-11-01 10:37:33 -07:00
zfs_sa.h Project Quota on ZFS 2018-02-13 14:54:54 -08:00
zfs_stat.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
zfs_sysfs.h Fix in-kernel sysfs entries 2018-09-06 21:44:52 -07:00
zfs_znode.h Move platform specific parts of zfs_znode.h to platform code 2019-11-06 10:54:25 -08:00
zil.h Add zfs_file_* interface, remove vnodes 2019-11-21 09:32:57 -08:00
zil_impl.h make zil max block size tunable 2019-06-10 11:48:42 -07:00
zio.h ICP: Improve AES-GCM performance 2020-02-10 12:59:50 -08:00
zio_checksum.h Remove dependency on linear ABD 2017-03-29 12:24:51 -07:00
zio_compress.h lz4_decompress_abd declared but not defined 2019-06-13 13:14:34 -07:00
zio_crypt.h Add FreeBSD support to zio_crypto.h 2019-11-30 15:38:16 -08:00
zio_impl.h Fix typos in include/ 2019-08-30 09:53:15 -07:00
zio_priority.h Add TRIM support 2019-03-29 09:13:20 -07:00
zrlock.h OpenZFS 6328 - Fix cstyle errors in zfs codebase 2017-01-12 09:42:11 -08:00
zthr.h Fast Clone Deletion 2019-07-26 10:54:14 -07:00
zvol.h async zvol minor node creation interferes with receive 2020-02-03 09:33:14 -08:00
zvol_impl.h Prefix struct rangelock 2019-11-01 10:37:33 -07:00