Archive-Team
/
zfs
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 103 Wiki Activity
zfs/module/zstd
History
Brian Behlendorf 4c9c96aba4 Silence unused-but-set-variable warnings 
Clang 13.0.0 added support for `Wunused-but-set-parameter` and
`-Wunused-but-set-variable` which correctly detects two unused
variables in zstd resulting in a build failure.  This commit
annotates these instances accordingly.

  https://releases.llvm.org/13.0.1/tools/clang/docs/ReleaseNotes.html#id6

In FSE_createCTable(), malloc() is intentionally defined as NULL when
compiled in the kernel so the variable is unused.

  zstd/lib/compress/fse_compress.c:307:12: error: variable 'size'
  set but not used [-Werror,-Wunused-but-set-variable]

Additionally, in ZSTD_seqDecompressedSize() the assert is compiled
out similarly resulting in an unused variable.

  zstd/lib/compress/zstd_compress_superblock.c:412:12: error: variable
  'litLengthSum' set but not used [-Werror,-Wunused-but-set-variable]

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
 		2022-05-02 15:42:58 -07:00
..
include Linux 5.16: Resolve ZSTD_isError symbol collision in Linux kernel 2021-12-07 13:14:24 -08:00
lib Silence unused-but-set-variable warnings 2022-05-02 15:42:58 -07:00
Makefile.in Fix cross-endian interoperability of zstd 2021-09-14 15:05:55 -07:00
README.md Fix various typos 2021-04-07 13:27:11 -07:00
zfs_zstd.c Fix cross-endian interoperability of zstd 2021-09-14 15:05:55 -07:00
zstd-in.c Import ZStandard v1.4.5 2020-08-20 10:30:06 -07:00
zstd_sparc.c Fix cross-endian interoperability of zstd 2021-09-14 15:05:55 -07:00

README.md

ZSTD-On-ZFS Library Manual

Introduction

This subtree contains the ZSTD library used in ZFS. It is heavily cut-down by dropping any unneeded files, and combined into a single file, but otherwise is intentionally unmodified. Please do not alter the file containing the zstd library, besides upgrading to a newer ZSTD release.

Tree structure:

  • zfs_zstd.c is the actual zzstd kernel module.
  • lib/ contains the unmodified, "amalgamated" version of the Zstandard library, generated from our template file
  • zstd-in.c is our template file for generating the library
  • include/: This directory contains supplemental includes for platform compatibility, which are not expected to be used by ZFS elsewhere in the future. Thus we keep them private to ZSTD.

Updating ZSTD

To update ZSTD the following steps need to be taken:

  1. Grab the latest release of ZSTD.
  2. Update module/zstd/zstd-in.c if required. (see zstd/contrib/single_file_libs/zstd-in.c in the zstd repository)
  3. Generate the "single-file-library" and put it to module/zstd/lib/.
  4. Copy the following files to module/zstd/lib/:
    • zstd/lib/zstd.h
    • zstd/lib/common/zstd_errors.h

This can be done using a few shell commands from inside the zfs repo:

cd PATH/TO/ZFS

url="https://github.com/facebook/zstd"
release="$(curl -s "${url}"/releases/latest | grep -oP '(?<=v)[\d\.]+')"
zstd="/tmp/zstd-${release}/"

wget -O /tmp/zstd.tar.gz \
    "${url}/releases/download/v${release}/zstd-${release}.tar.gz"
tar -C /tmp -xzf /tmp/zstd.tar.gz

cp ${zstd}/lib/zstd.h module/zstd/lib/
cp ${zstd}/lib/zstd_errors.h module/zstd/lib/
${zstd}/contrib/single_file_libs/combine.sh \
    -r ${zstd}/lib -o module/zstd/lib/zstd.c module/zstd/zstd-in.c

Note: if the zstd library for zfs is updated to a newer version, the macro list in include/zstd_compat_wrapper.h usually needs to be updated. this can be done with some hand crafting of the output of the following script: nm zstd.o | awk '{print "#define "$3 " zfs_" $3}' > macrotable

Altering ZSTD and breaking changes

If ZSTD made changes that break compatibility or you need to make breaking changes to the way we handle ZSTD, it is required to maintain backwards compatibility.

We already save the ZSTD version number within the block header to be used to add future compatibility checks and/or fixes. However, currently it is not actually used in such a way.