Archive-Team
/
zfs
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 103 Wiki Activity

Limit dbuf cache sizes based only on ARC target size by default 

Set the initial max sizes to ULONG_MAX to allow the caches to grow
with the ARC.

Recalculate the metadata cache size on demand so it can adapt, too.

Update descriptions in zfs-module-parameters(5).

Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Matt Ahrens <matt@delphix.com>
Signed-off-by: Ryan Moeller <ryan@iXsystems.com>
Closes #10563 
Closes #10610
This commit is contained in:
Ryan Moeller 2020-07-24 23:38:48 -04:00 committed by GitHub
parent 4fbdb10c7b
commit 8348fac30c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 34 additions and 34 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
man/man5/zfs-module-parameters.5
View File

@ -32,13 +32,12 @@ Description of the different parameters to the ZFS module.
\fBdbuf_cache_max_bytes\fR (ulong)
.ad
.RS 12n
Maximum size in bytes of the dbuf cache. When \fB0\fR this value will default
to \fB1/2^dbuf_cache_shift\fR (1/32) of the target ARC size, otherwise the
provided value in bytes will be used. The behavior of the dbuf cache and its
associated settings can be observed via the \fB/proc/spl/kstat/zfs/dbufstats\fR
kstat.
Maximum size in bytes of the dbuf cache. The target size is determined by the
MIN versus \fB1/2^dbuf_cache_shift\fR (1/32) of the target ARC size. The
behavior of the dbuf cache and its associated settings can be observed via the
\fB/proc/spl/kstat/zfs/dbufstats\fR kstat.
.sp
Default value: \fB0\fR.
Default value: \fBULONG_MAX\fR.
.RE
.sp
@ -47,13 +46,12 @@ Default value: \fB0\fR.
\fBdbuf_metadata_cache_max_bytes\fR (ulong)
.ad
.RS 12n
Maximum size in bytes of the metadata dbuf cache. When \fB0\fR this value will
default to \fB1/2^dbuf_cache_shift\fR (1/16) of the target ARC size, otherwise
the provided value in bytes will be used. The behavior of the metadata dbuf
cache and its associated settings can be observed via the
\fB/proc/spl/kstat/zfs/dbufstats\fR kstat.
Maximum size in bytes of the metadata dbuf cache. The target size is
determined by the MIN versus \fB1/2^dbuf_metadata_cache_shift\fR (1/64) of the
target ARC size. The behavior of the metadata dbuf cache and its associated
settings can be observed via the \fB/proc/spl/kstat/zfs/dbufstats\fR kstat.
.sp
Default value: \fB0\fR.
Default value: \fBULONG_MAX\fR.
.RE
.sp

46
module/zfs/dbuf.c
View File

@ -207,12 +207,16 @@ typedef struct dbuf_cache {
dbuf_cache_t dbuf_caches[DB_CACHE_MAX];
/* Size limits for the caches */
unsigned long dbuf_cache_max_bytes = 0;
unsigned long dbuf_metadata_cache_max_bytes = 0;
unsigned long dbuf_cache_max_bytes = ULONG_MAX;
unsigned long dbuf_metadata_cache_max_bytes = ULONG_MAX;
/* Set the default sizes of the caches to log2 fraction of arc size */
int dbuf_cache_shift = 5;
int dbuf_metadata_cache_shift = 6;
static unsigned long dbuf_cache_target_bytes(void);
static unsigned long dbuf_metadata_cache_target_bytes(void);
/*
* The LRU dbuf cache uses a three-stage eviction policy:
* - A low water marker designates when the dbuf eviction thread
@ -432,7 +436,7 @@ dbuf_include_in_metadata_cache(dmu_buf_impl_t *db)
*/
if (zfs_refcount_count(
&dbuf_caches[DB_DBUF_METADATA_CACHE].size) >
dbuf_metadata_cache_max_bytes) {
dbuf_metadata_cache_target_bytes()) {
DBUF_STAT_BUMP(metadata_cache_overflow);
return (B_FALSE);
}
@ -610,11 +614,26 @@ dbuf_cache_multilist_index_func(multilist_t *ml, void *obj)
multilist_get_num_sublists(ml));
}
/*
* The target size of the dbuf cache can grow with the ARC target,
* unless limited by the tunable dbuf_cache_max_bytes.
*/
static inline unsigned long
dbuf_cache_target_bytes(void)
{
return MIN(dbuf_cache_max_bytes,
arc_target_bytes() >> dbuf_cache_shift);
return (MIN(dbuf_cache_max_bytes,
arc_target_bytes() >> dbuf_cache_shift));
}
/*
* The target size of the dbuf metadata cache can grow with the ARC target,
* unless limited by the tunable dbuf_metadata_cache_max_bytes.
*/
static inline unsigned long
dbuf_metadata_cache_target_bytes(void)
{
return (MIN(dbuf_metadata_cache_max_bytes,
arc_target_bytes() >> dbuf_metadata_cache_shift));
}
static inline uint64_t
@ -805,23 +824,6 @@ retry:
dbuf_stats_init(h);
/*
* Setup the parameters for the dbuf caches. We set the sizes of the
* dbuf cache and the metadata cache to 1/32nd and 1/16th (default)
* of the target size of the ARC. If the values has been specified as
* a module option and they're not greater than the target size of the
* ARC, then we honor that value.
*/
if (dbuf_cache_max_bytes == 0 ||
dbuf_cache_max_bytes >= arc_target_bytes()) {
dbuf_cache_max_bytes = arc_target_bytes() >> dbuf_cache_shift;
}
if (dbuf_metadata_cache_max_bytes == 0 ||
dbuf_metadata_cache_max_bytes >= arc_target_bytes()) {
dbuf_metadata_cache_max_bytes =
arc_target_bytes() >> dbuf_metadata_cache_shift;
}
/*
* All entries are queued via taskq_dispatch_ent(), so min/maxalloc
* configuration is not required.