zfs/module/Makefile.in

143 lines
4.0 KiB
Makefile
Raw Normal View History

Cleanup linux module kbuild files The linux module can be built either as an external module, or compiled into the kernel, using copy-builtin. The source and build directories are slightly different between the two cases, and currently, compiling into the kernel still refers to some files from the configured ZFS source tree, instead of the copies inside the kernel source tree. There is also duplication between copy-builtin, which creates a Kbuild file to build ZFS inside the kernel tree, and the top-level module/Makefile.in. Fix this by moving the list of modules and the CFLAGS settings into a new module/Kbuild.in, which will be used by the kernel kbuild infrastructure, and using KBUILD_EXTMOD to distinguish the two cases within the Makefiles, in order to choose appropriate include directories etc. Module CFLAGS setting is simplified by using subdir-ccflags-y (available since 2.6.30) to set them in the top-level Kbuild instead of each individual module. The disabling of -Wunused-but-set-variable is removed from the lua and zfs modules. The variable that the Makefile uses is actually not defined, so this has no effect; and the warning has long been disabled by the kernel Makefile itself. The target_cpu definition in module/{zfs,zcommon} is removed as it was replaced by use of CONFIG_SPARC64 in commit 70835c5b755e ("Unify target_cpu handling") os/linux/{spl,zfs} are removed from obj-m, as they are not modules in themselves, but are included by the Makefile in the spl and zfs module directories. The vestigial Makefiles in os and os/linux are removed. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu> Closes #10379 Closes #10421
2020-06-07 21:03:12 +00:00
include Kbuild
INSTALL_MOD_DIR ?= extra
Add zstd support to zfs This PR adds two new compression types, based on ZStandard: - zstd: A basic ZStandard compression algorithm Available compression. Levels for zstd are zstd-1 through zstd-19, where the compression increases with every level, but speed decreases. - zstd-fast: A faster version of the ZStandard compression algorithm zstd-fast is basically a "negative" level of zstd. The compression decreases with every level, but speed increases. Available compression levels for zstd-fast: - zstd-fast-1 through zstd-fast-10 - zstd-fast-20 through zstd-fast-100 (in increments of 10) - zstd-fast-500 and zstd-fast-1000 For more information check the man page. Implementation details: Rather than treat each level of zstd as a different algorithm (as was done historically with gzip), the block pointer `enum zio_compress` value is simply zstd for all levels, including zstd-fast, since they all use the same decompression function. The compress= property (a 64bit unsigned integer) uses the lower 7 bits to store the compression algorithm (matching the number of bits used in a block pointer, as the 8th bit was borrowed for embedded block pointers). The upper bits are used to store the compression level. It is necessary to be able to determine what compression level was used when later reading a block back, so the concept used in LZ4, where the first 32bits of the on-disk value are the size of the compressed data (since the allocation is rounded up to the nearest ashift), was extended, and we store the version of ZSTD and the level as well as the compressed size. This value is returned when decompressing a block, so that if the block needs to be recompressed (L2ARC, nop-write, etc), that the same parameters will be used to result in the matching checksum. All of the internal ZFS code ( `arc_buf_hdr_t`, `objset_t`, `zio_prop_t`, etc.) uses the separated _compress and _complevel variables. Only the properties ZAP contains the combined/bit-shifted value. The combined value is split when the compression_changed_cb() callback is called, and sets both objset members (os_compress and os_complevel). The userspace tools all use the combined/bit-shifted value. Additional notes: zdb can now also decode the ZSTD compression header (flag -Z) and inspect the size, version and compression level saved in that header. For each record, if it is ZSTD compressed, the parameters of the decoded compression header get printed. ZSTD is included with all current tests and new tests are added as-needed. Per-dataset feature flags now get activated when the property is set. If a compression algorithm requires a feature flag, zfs activates the feature when the property is set, rather than waiting for the first block to be born. This is currently only used by zstd but can be extended as needed. Portions-Sponsored-By: The FreeBSD Foundation Co-authored-by: Allan Jude <allanjude@freebsd.org> Co-authored-by: Brian Behlendorf <behlendorf1@llnl.gov> Co-authored-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Co-authored-by: Kjeld Schouten-Lebbing <kjeld@schouten-lebbing.nl> Co-authored-by: Michael Niewöhner <foss@mniewoehner.de> Signed-off-by: Allan Jude <allan@klarasystems.com> Signed-off-by: Allan Jude <allanjude@freebsd.org> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Signed-off-by: Kjeld Schouten-Lebbing <kjeld@schouten-lebbing.nl> Signed-off-by: Michael Niewöhner <foss@mniewoehner.de> Closes #6247 Closes #9024 Closes #10277 Closes #10278
2020-08-18 17:10:17 +00:00
SUBDIR_TARGETS = icp lua zstd
all: modules
distclean maintainer-clean: clean
install: modules_install
uninstall: modules_uninstall
check:
.PHONY: all distclean maintainer-clean install uninstall check distdir \
modules modules-Linux modules-FreeBSD modules-unknown \
clean clean-Linux clean-FreeBSD \
modules_install modules_install-Linux modules_install-FreeBSD \
2021-01-22 20:54:34 +00:00
modules_uninstall modules_uninstall-Linux modules_uninstall-FreeBSD \
cppcheck cppcheck-Linux cppcheck-FreeBSD
# For FreeBSD, use debug options from ./configure if not overridden.
export WITH_DEBUG ?= @WITH_DEBUG@
export WITH_INVARIANTS ?= @WITH_INVARIANTS@
# Filter out options that FreeBSD make doesn't understand
getflags = ( \
set -- \
$(filter-out --%,$(firstword $(MFLAGS))) \
$(filter -I%,$(MFLAGS)) \
$(filter -j%,$(MFLAGS)); \
fmakeflags=""; \
while getopts :deiI:j:knqrstw flag; do \
case $$flag in \
\?) :;; \
:) if [ $$OPTARG = "j" ]; then \
ncpus=$$(sysctl -n kern.smp.cpus 2>/dev/null || :); \
if [ -n "$$ncpus" ]; then fmakeflags="$$fmakeflags -j$$ncpus"; fi; \
fi;; \
d) fmakeflags="$$fmakeflags -dA";; \
*) fmakeflags="$$fmakeflags -$$flag$$OPTARG";; \
esac; \
done; \
echo $$fmakeflags \
)
FMAKEFLAGS = -C @abs_srcdir@ -f Makefile.bsd $(shell $(getflags))
ifneq (@abs_srcdir@,@abs_builddir@)
FMAKEFLAGS += MAKEOBJDIR=@abs_builddir@
endif
FMAKE = env -u MAKEFLAGS make $(FMAKEFLAGS)
modules-Linux:
list='$(SUBDIR_TARGETS)'; for targetdir in $$list; do \
$(MAKE) -C $$targetdir; \
done
$(MAKE) -C @LINUX_OBJ@ M=`pwd` @KERNEL_MAKE@ CONFIG_ZFS=m modules
modules-FreeBSD:
+$(FMAKE)
modules-unknown:
@true
modules: modules-@ac_system@
clean-Linux:
@# Only cleanup the kernel build directories when CONFIG_KERNEL
@# is defined. This indicates that kernel modules should be built.
@CONFIG_KERNEL_TRUE@ $(MAKE) -C @LINUX_OBJ@ M=`pwd` @KERNEL_MAKE@ clean
if [ -f @LINUX_SYMBOLS@ ]; then $(RM) @LINUX_SYMBOLS@; fi
if [ -f Module.markers ]; then $(RM) Module.markers; fi
find . -name '*.ur-safe' -type f -print | xargs $(RM)
clean-FreeBSD:
+$(FMAKE) clean
clean: clean-@ac_system@
modules_install-Linux:
Support custom build directories and move includes One of the neat tricks an autoconf style project is capable of is allow configurion/building in a directory other than the source directory. The major advantage to this is that you can build the project various different ways while making changes in a single source tree. For example, this project is designed to work on various different Linux distributions each of which work slightly differently. This means that changes need to verified on each of those supported distributions perferably before the change is committed to the public git repo. Using nfs and custom build directories makes this much easier. I now have a single source tree in nfs mounted on several different systems each running a supported distribution. When I make a change to the source base I suspect may break things I can concurrently build from the same source on all the systems each in their own subdirectory. wget -c http://github.com/downloads/behlendorf/zfs/zfs-x.y.z.tar.gz tar -xzf zfs-x.y.z.tar.gz cd zfs-x-y-z ------------------------- run concurrently ---------------------- <ubuntu system> <fedora system> <debian system> <rhel6 system> mkdir ubuntu mkdir fedora mkdir debian mkdir rhel6 cd ubuntu cd fedora cd debian cd rhel6 ../configure ../configure ../configure ../configure make make make make make check make check make check make check This change also moves many of the include headers from individual incude/sys directories under the modules directory in to a single top level include directory. This has the advantage of making the build rules cleaner and logically it makes a bit more sense.
2010-09-04 20:26:23 +00:00
@# Install the kernel modules
$(MAKE) -C @LINUX_OBJ@ M=`pwd` modules_install \
INSTALL_MOD_PATH=$(DESTDIR)$(INSTALL_MOD_PATH) \
INSTALL_MOD_DIR=$(INSTALL_MOD_DIR) \
KERNELRELEASE=@LINUX_VERSION@
@# Remove extraneous build products when packaging
kmoddir=$(DESTDIR)$(INSTALL_MOD_PATH)/lib/modules/@LINUX_VERSION@; \
if [ -n "$(DESTDIR)" ]; then \
find $$kmoddir -name 'modules.*' | xargs $(RM); \
fi
sysmap=$(DESTDIR)$(INSTALL_MOD_PATH)/boot/System.map-@LINUX_VERSION@; \
if [ -f $$sysmap ]; then \
depmod -ae -F $$sysmap @LINUX_VERSION@; \
fi
modules_install-FreeBSD:
@# Install the kernel modules
+$(FMAKE) install
modules_install: modules_install-@ac_system@
modules_uninstall-Linux:
Support custom build directories and move includes One of the neat tricks an autoconf style project is capable of is allow configurion/building in a directory other than the source directory. The major advantage to this is that you can build the project various different ways while making changes in a single source tree. For example, this project is designed to work on various different Linux distributions each of which work slightly differently. This means that changes need to verified on each of those supported distributions perferably before the change is committed to the public git repo. Using nfs and custom build directories makes this much easier. I now have a single source tree in nfs mounted on several different systems each running a supported distribution. When I make a change to the source base I suspect may break things I can concurrently build from the same source on all the systems each in their own subdirectory. wget -c http://github.com/downloads/behlendorf/zfs/zfs-x.y.z.tar.gz tar -xzf zfs-x.y.z.tar.gz cd zfs-x-y-z ------------------------- run concurrently ---------------------- <ubuntu system> <fedora system> <debian system> <rhel6 system> mkdir ubuntu mkdir fedora mkdir debian mkdir rhel6 cd ubuntu cd fedora cd debian cd rhel6 ../configure ../configure ../configure ../configure make make make make make check make check make check make check This change also moves many of the include headers from individual incude/sys directories under the modules directory in to a single top level include directory. This has the advantage of making the build rules cleaner and logically it makes a bit more sense.
2010-09-04 20:26:23 +00:00
@# Uninstall the kernel modules
kmoddir=$(DESTDIR)$(INSTALL_MOD_PATH)/lib/modules/@LINUX_VERSION@; \
for objdir in $(ZFS_MODULES); do \
$(RM) -R $$kmoddir/$(INSTALL_MOD_DIR)/$$objdir; \
done
modules_uninstall-FreeBSD:
@false
modules_uninstall: modules_uninstall-@ac_system@
2021-01-22 20:54:34 +00:00
cppcheck-Linux:
@CPPCHECK@ -j@CPU_COUNT@ --std=c99 --quiet --force --error-exitcode=2 \
--inline-suppr \
--suppress=unmatchedSuppression \
--suppress=noValidConfiguration \
2021-01-22 20:54:34 +00:00
--enable=warning,information -D_KERNEL \
--include=@LINUX_OBJ@/include/generated/autoconf.h \
--include=@top_srcdir@/zfs_config.h \
--config-exclude=@LINUX_OBJ@/include \
-I @LINUX_OBJ@/include \
-I @top_srcdir@/include/os/linux/kernel \
-I @top_srcdir@/include/os/linux/spl \
-I @top_srcdir@/include/os/linux/zfs \
-I @top_srcdir@/include \
avl icp lua nvpair spl unicode zcommon zfs zstd os/linux
cppcheck-FreeBSD:
@true
cppcheck: cppcheck-@ac_system@
distdir:
(cd @srcdir@ && find $(ZFS_MODULES) os -name '*.[chS]') | \
while read path; do \
mkdir -p $$distdir/$${path%/*}; \
cp @srcdir@/$$path $$distdir/$$path; \
done; \
cp @srcdir@/Makefile.bsd $$distdir/Makefile.bsd