2009-11-02 Brian Behlendorf * : Tag zfs-0.4.6 - Use 'git log --no-merges' for full change log. * Rebased to ZFS b121 from OpenSolaris. * module/zfs/vdev_disk.c: Finally a feature complete implementation: - Handle dynamic bio merge_bdev limitations when constructing the bio set associated with a dio. This previously prevented us from layering cleanly on the md and dm virtual devices. - Removed hard coded 512 byte sector size. - Correctly determine the device size when using a partition. - Hold and extra dio reference when submitting bio's using bio_submit() to prevent a completion race. * lib/libefi/*: Added fully function libefi library from Solaris. This allows us to properly create and access GPT style partition tables which are used when a whole device is added to a zpool. * cmd/zpool/zpool_vdev.c: Fully integrated zpool with Linux package libblkid. This allows zpool to identify existing devices of various types to prevent devices from accidentally being used. When given a whole device with a GPT partition table all partitions will be checked for existing filesystems. At the moment MBR style partition tables cannot be check and the force option must be used. * cmd/zpool/zpool_vdev.c: Solaris devid support has been removed in favor of Linux's udev. This means that a zpool device will always be opened using the path provided at configuration time. This may initially seem limiting but it has certain advantages: - When creating a zpool where the physical location of the device is NOT import simply create the pool using the /dev/disk/by-id paths. This will ensure that regardless of physical location the device will be properly located. - When creating a zpool where the physical location of the device is important use the /dev/disk/by-path paths. This will ensure that devices are never accidentally detected and used in an incorrect location which would compromise the redundancy of the system. - Ever better you can create use your own udev rules file to setup any mapping and naming convention you desire. One example of a custom rule is to map physical device locations using grid with numbers are letter for coordinates. Each letter might represent a specific bus/channel and each number a specific device. For large configurations this provides an easy way to identify devices. * module/zpios/zpios.c: Update to use kobject_set_name() for increased portability. * modules/*/*: Update module init/exit access points to use spl_module_{init,exit}() macro API. This ensures the cwd is immediately set to '/' and may be leveraged latter for any additional module setup/cleanup which is required. * cmd/ztest/ztest.c: Check ftrucate() return code to prevent warnings when --fortify-source options is used in rpm builds. * config/Rules.am: Set DEBUG/NDEBUG globally when building user space components. * scripts/zconfig.sh: Initial hook for running additional sanity tests are part of 'make check'. Currently, there are only two tests which do some basic configuration checking but they should be extended as much as possible to prevent regressions. Tests should also all be written so they run entirely in-tree. * scripts/zpios-sanity.sh: Initial hook for validating real IO using all block devices and all raid configurations. Supported device types include scsi, ide, md, dm, ram, loop, and file. Supported raid types include raid0, raid10, raidz, and raidz2. * scripts/zpool-config/*: Update dragon and x4550 configs to use custom udev rules file with <1-N> naming convention. Add configs for md, dm, and ram block devices to verify functionality. * zfs-test.spec.in: Added zfs-test package which extends the existing in-tree test infrastructure such that it can be run as part of an installed package. This simplifies the testing of tagged releases. * zfs-modules.spec.in: Various spec file tweaks for the supported distros: RHEL5, RHEL6, SLES10, SLES11, Chaos4, Fedora 11. 2009-08-04 Brian Behlendorf * : Tag zfs-0.4.5 - Use 'git log --no-merges' for full change log. * FC11 and SLES11 support: This includes all compatibility changes to support 2.6.29 based kernels and the required build system improvements. * lib/libspl/asm-generic/atomic.c: Generic user space atomic support. * module/zpios/zpios.c: Register a basic compat ioctl handler for 32-bit user vs 64-bit kernel compatibility. This is the default build environment for all 64-bit SLES systems. Additionally replace the use of 'struct timespec' which uses longs internally and is therefore different sizes on 32-bit vs 64-bit objects with 'struct zpios_timespec_t'.a * config/kernel.m4: Check arch/default path when detecting kernel objects under SLES. We still preferentially use arch/arch if available but if that fails it is acceptable to use default. * config/kernel.m4: Remove LINUXINCLUDE from autoconf wrapper. This breaks the 2.6.28+ kernels build system, all kernel build systems at least post 2.6.16 will set this properly so we should not. * lib/libspl/include/assert.h: Add ASSERTV macro to simplify removing variables (the V in ASSERTV) which are only used in ASSERT(). Also revert all previously modified ASSERT()s to their original definitions and use the ASSERTV macro to handle unused variables. * module/zpios/zpios.c: Use spl device interfaces for portability. * scripts/common.sh: The losetup -f option is unavailable for SLES10 use the unused_loop_device() common function. Additionally LOSETUP is now used to portably reference the losetup binary. * module/zfs/vdev_disk.c: Unused destroy_dirty_buffers arg removed. * module/zfs/vdev_disk.c: BIO_RW_SYNC renamed to BIO_RW_SYNCIO. * module/zfs/vdev_disk.c: open/close_bdev_excl() renamed to open/close_bdev_exclusive(). * module/zfs/vdev_disk.c: Empty write barriers are supported as of linux 2.6.24 and are now used to implement DKIOCFLUSHWRITECACHE. * module/zfs/vdev_disk.c: The bi_end_io API changes make partial IO's impossible handle this case cleanly. * module/zfs/vdev_disk.c: BIO_RW_FAILFAST replaced with BIO_RW_FAILFAST_{DEV|_TRANSPORT|_DRIVER}, use the legacy BIO_RW_FAILFAST flag if it exists for now. * module/zfs/dmu_objset.c, module/zfs/spa_history.c: Excessively large stack frames (>2048) were detected in dmu_objset_snapshot() and spa_history_log() and were reduced by allocating from the heap. * Rebased to ZFS b117 from OpenSolaris: * module/zfs/dmu.c: Update zerocopy patch to be consistent with new flags arg passed to dmu_read(). * module/zpios/zpios.c: Update zpios to be aware of the new flags argument available in dmu_read(). Additionally add a zpios command line flag to set the DMU_READ_NO_PREFETCH flag for a test. * module/zfs/include/sys/zfs_context.h: Wrap new sysevent includes until sysevent or something like it is implemented. * module/zfs/zfs_ioctl.c: Wrap all the ACL interfaces with HAVE_ZPL. They are integrated with the vfs layer which is not yet supported. * module/zcommon/zfs_prop.c: Export new quota related symbols. * lib/libzfs/libzfs_util.c: Increase buffer size for nvlist which is needed for large configurations. * Bug fixes: * lib/libspl/include/sys/zfs_debug.h: Removed duplicate file. * lib/libspl/include/sys/isa_defs.h: Fixed missing macro definitions for little endian and big endian. * cmd/zpios/zpios_main.c: Pretty-up the zpios 'make check' output. * module/zfs/vdev_disk.c: Honor spa_mode() when opening block devs. * module/zfs/vdev_disk.c: Initial error handling added to vdev_disk_io_done() to revalidate the media on EIO. * lib/libzfs/libzfs_util.c: Fix an accidentally introduced formatting issue in zfs_nicenum() output. 2009-07-02 Brian Behlendorf * : Tag zfs-0.4.4 - Use 'git log --no-merges' for full change log. * : Generic distro friendly build system / packaging improvements for RPM based distros including CHAOS, RHEL, Fedora, and SLES. These changes bring the zfs-0.4.4 tree in to compliance with the spl-0.4.4 packaging changes. The bottom line is 2 source rpms and 4 binary rpms will now be generated when creating packages there will be: zfs-.src.rpm - Fully rebuildable source rpm for libzfs and utils. zfs-modules-.src.rpm - Fully rebuildable source rpm for kernel modules. zfs-..rpm - Binary rpm for libzfs and utils. The utils in this package are compatible with all zfs-module rpms of the same version. zfs-devel-..rpm - Binary rpm containing headers for building against libzfs libraries. zfs-modules--.arch.rpm - Binary rpm containing the kernel modules for a specific kernel build. The package name contains the kernel version and you should have one of these packages installed to match every kernel on your system. zfs-modules-devel--.arch.rpm - Binary rpm containing development header and module symbols needed for building additional kernel modules which are dependent on the zfs module stack. * : SLES9/10 distro support added: Several significant updates to the build system were required to support this including: - Autoconf macros updated to be aware of the standard install locations for SLES kernel sources and build objects. Additionally changes were made to support multiple names for Module{s}.symvers. - By default on SLES all user space builds are 32-bit even on 64-bit arches. This means we need to be careful to pass -m64 in the autoconf checks which probe the kernel. Additionally, ioctl_compat handles still need to be added so 32-bit user binarys can perform ioctls with the 64-bit kernel. * : Powerpc64 support added: - 64-bit user space atomic support for power64 was obtained from an old version of OpenSolaris which offered minimal powerpc support. The atomic support is not 100% fully implemented but it's a good first step towards cleanly supporting the architecture. - Added powerpc ISA type. - Explicitly use signed char for portability. On x86/x86_64 systems the default char type is signed, on ppc/ppc64 systems the default char type is unsigned. - Core target arch support for conditional compilation of SUBDIRs. Required by libspl for its arch specific atomic implementations. * COPYRIGHT: Readded accidentally dropped COPYRIGHT, it just references the OPENSOLARIS.LICENSE which was still in the project. * module/zfs/dmu_tx.c: Add EXPORT_SYMBOL(dmu_tx_callback_register). * config/user-zlib.m4: Add basic zlib autoconf check for user space. 2009-03-20 Brian Behlendorf * : Tag zfs-0.4.3 - Use 'git log --no-merges' for full change log. * : Rebased to ZFS b108 from OpenSolaris * configure.ac, *Makefile.am: Build system update. This includes resolving various build issues and adding support for the remaining common build targets. Available targets now include: - make all # Build everything - make install # Install everything - make clean # Clean up build products - make distclean # Clean up everything - make dist # Create package tarball - make srpm # Create package source RPM - make rpm # Create package binary RPMs - make tags # Create ctags and etags for everything Extra care was taken to ensure that the source RPMs are fully rebuildable against Fedora/RHEL/Chaos kernels. To build binary RPMs from the source RPM for your system simply run: rpmbuild --rebuild zfs-x.y.z-1.src.rpm This will produce two binary RPMs with correct 'requires' dependencies for your kernel. One will contain all zfs modules and support utilities, the other is a devel package for compiling additional kernel modules which are dependent on the zfs. zfs-x.y.z-1_.x86_64.rpm zfs-devel-x.y.2-1_.x86_64.rpm * config/kernel.m4, config/Rules.am: Removed the build options -Wno-unused -Wno-missing -Wno-unused -Wno-parentheses and -Wno-unknown-pragmas. All upstream warnings which were being suppressed by these options are now handled in the matching gcc-* topic branch. * zpios: Various zpios and test infrastructure improvements. * feature-commit-cb: New feature commit callbacks. * : Various bug fixes which are more clearly detailed in the git commit logs. For a detailed summary of changes post version 0.4.2, check out the 'top' topic branch and view the commit logs. git checkout top git log --no-merges -53 2009-02-05 Brian Behlendorf * : Tag zfs-0.4.2 * lib/libzfs/libzfs_util.c: Remove mknod() this is now handled by by the SPL with an upcall even without GPL-only symbols. * module/zfs/vdev_diskc: Updated linux vdev_disk interfaces to issue multiple bios if needed due to the maximum request size being smaller than the request size passed down from the spa. * scripts/zpool-config/lo-*: Add loopback based test configs. 2009-01-21 Brian Behlendorf * : Tag zfs-0.4.1 * : Rebased to ZFS b105 from OpenSolaris * : Restructured project and build system to more closely mirror the code organization used in OpenSolaris. This allows us minimize the changes to upstream source needed, and allows concurrent user+kernel space builds of all components. 2008-11-19 Brian Behlendorf * : Tag zfs-0.4.0 * : ZFS project migrated from Subversion which leveraged a quilt based patch stack to Git and a TopGit managed patch stack. The new method treats all patches as Git branches which can be more easily shared for distributed development. Consult the top level GIT file for detailed information on how to properly develop for this package using Git+TopGit. 2008-11-12 Brian Behlendorf * : Tag zfs-0.3.4 * zfs-07-create-dev-zfs.patch: Ricardo M. Correia - Make libzfs create /dev/zfs if it doesn't exist. * zfs-05-check-zvol-size.patch: Ricardo M. Correia - Properly check zvol size under Linux. * zfs-04-no-openat-fdopendir.patch: Ricardo M. Correia - Do not use openat() and fdopendir() since they are not available on older systems. * zfs-03-fix-bio-sync.patch: Ricardo M. Correia - Fix memory corruption in RHEL4 due to synchronous IO becoming asynchronous. 2008-11-06 Brian Behlendorf * zfs-02-zpios-fix-stuck-thread-memleak.patch: Ricardo M. Correia - Fix stuck threads and memory leaks when errors occur while writing. * zfs-01-zpios-arg-corruption.patch: Ricardo M. Correia - Fix zpios cmd line argument corruption problem. * zfs-00-minor-fixes.patch: Ricardo M. Correia - Minor build system improvements - Minor script improvements - Create a full copy and not a link tree with quilt - KPIOS_MAJOR changed from 231 to 232 - BIO_RW_BARRIER flag removed from IO request 2008-06-30 Brian Behlendorf * : Tag zfs-0.3.3 * : Minor script updates and tweaks to be compatible with the latest version of the SPL. 2008-06-13 Brian Behlendorf * vdev_disk.diff: Replace vdev_disk implementation which was based on the kmalloc'ed logical address space with a version which works with vmalloc'ed memory in the virtual address space. This was done to support the new SPL slab implementation which is based on virtual addresses to avoid the need for contigeously allocated memory. 2008-06-05 Brian Behlendorf * arc-vm-integration.diff: Reduce maximum default arc memory usage to 1/4 of total system memory. Because all the bulk data is still allocated on the slab memory fragmentation is a serious concern. To address this in the short term we simply need to leave lots of free memory. * fix-stack.diff: First step towards reducing stack usage so we can run the full ZFS stack using a stock kernel. 2008-06-04 Brian Behlendorf * : Tag zfs-0.3.2 * : Extensive improvements to the build system to detect kernel API changes so we can flexibly build with a wider range of kernel versions. The code has now been testing with the 2.6.18-32chaos and 2.6.25.3-18.fc9 kernels, however we should also be compatible with other kernels in the range of 2.6.18-2.6.25. The only remaining issue preventing us from running with a stock kernel is ZFS stack usage. 2008-05-21 Brian Behlendorf * : Tag zfs-0.3.1 * : License headers including URCL added for release. 2008-05-21 Brian Behlendorf * : Tag zfs-0.3.0 * configure.ac: Improved autotools support and configurable debug. 2008-05-15 Brian Behlendorf * : Updating original ZFS sources to build 89 which includes the new write throttling changes plus support for using ZFS as your root device. Neither of which will work exactly right without some more work but this gets us much closers to the latest source. 2008-02-28 Brian Behlendorf * : First attempt based on SPL module and zfs-lustre sources