Archive-Team
/
zfs
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 103 Wiki Activity
zfs/include/sys/mutex.h

198 lines
5.5 KiB
C
Raw Normal View History

Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
/*
Public Release Prep Updated AUTHORS, COPYING, DISCLAIMER, and INSTALL files. Added standardized headers to all source file to clearly indicate the copyright, license, and to give credit where credit is due.
2010-05-17 22:18:00 +00:00
* Copyright (C) 2007-2010 Lawrence Livermore National Security, LLC.
* Copyright (C) 2007 The Regents of the University of California.
* Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER).
* Written by Brian Behlendorf <behlendorf1@llnl.gov>.
Go through and add a header with the proper UCRL number. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@114 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-26 04:38:26 +00:00
* UCRL-CODE-235197
*
Public Release Prep Updated AUTHORS, COPYING, DISCLAIMER, and INSTALL files. Added standardized headers to all source file to clearly indicate the copyright, license, and to give credit where credit is due.
2010-05-17 22:18:00 +00:00
* This file is part of the SPL, Solaris Porting Layer.
Refresh links to web site Update links to refer to the official ZFS on Linux website instead of @behlendorf's personal fork on github. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-03-05 01:26:55 +00:00
* For details, see <http://zfsonlinux.org/>.
Public Release Prep Updated AUTHORS, COPYING, DISCLAIMER, and INSTALL files. Added standardized headers to all source file to clearly indicate the copyright, license, and to give credit where credit is due.
2010-05-17 22:18:00 +00:00
*
* The SPL is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
* Free Software Foundation; either version 2 of the License, or (at your
* option) any later version.
Go through and add a header with the proper UCRL number. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@114 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-26 04:38:26 +00:00
*
Public Release Prep Updated AUTHORS, COPYING, DISCLAIMER, and INSTALL files. Added standardized headers to all source file to clearly indicate the copyright, license, and to give credit where credit is due.
2010-05-17 22:18:00 +00:00
* The SPL is distributed in the hope that it will be useful, but WITHOUT
Go through and add a header with the proper UCRL number. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@114 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-26 04:38:26 +00:00
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
* for more details.
*
* You should have received a copy of the GNU General Public License along
Public Release Prep Updated AUTHORS, COPYING, DISCLAIMER, and INSTALL files. Added standardized headers to all source file to clearly indicate the copyright, license, and to give credit where credit is due.
2010-05-17 22:18:00 +00:00
* with the SPL. If not, see <http://www.gnu.org/licenses/>.
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
*/
Go through and add a header with the proper UCRL number. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@114 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-26 04:38:26 +00:00
Minor nit, SOLARIS should be SPL git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@17 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-28 00:52:31 +00:00
#ifndef _SPL_MUTEX_H
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
#define _SPL_MUTEX_H
Initial commit. All spl source written up to this point wrapped in an initial reasonable autoconf style build system. This does not yet build but the configure system does appear to work properly and integrate with the kernel. Hopefully the next commit gets us back to a buildable version we can run the test suite against. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@1 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-26 20:36:04 +00:00
Reorganize /include/ to add a /sys/, this way we don't need to muck with #includes in existing Solaris style source to get it to find the right stuff. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@18 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-01 00:45:59 +00:00
#include <sys/types.h>
Reimplement mutexs for Linux lock profiling/analysis For a generic explanation of why mutexs needed to be reimplemented to work with the kernel lock profiling see commits: e811949a57044d60d12953c5c3b808a79a7d36ef and d28db80fd0fd4fd63aec09037c44408e51a222d6 The specific changes made to the mutex implemetation are as follows. The Linux mutex structure is now directly embedded in the kmutex_t. This allows a kmutex_t to be directly case to a mutex struct and passed directly to the Linux primative. Just like with the rwlocks it is critical that these functions be implemented as '#defines to ensure the location information is preserved. The preprocessor can then do a direct replacement of the Solaris primative with the linux primative. Just as with the rwlocks we need to track the lock owner. Here things get a little more interesting because depending on your kernel version, and how you've built your kernel Linux may already do this for you. If your running a 2.6.29 or newer kernel on a SMP system the lock owner will be tracked. This was added to Linux to support adaptive mutexs, more on that shortly. Alternately, your kernel might track the lock owner if you've set CONFIG_DEBUG_MUTEXES in the kernel build. If neither of the above things is true for your kernel the kmutex_t type will include and track the lock owner to ensure correct behavior. This is all handled by a new autoconf check called SPL_AC_MUTEX_OWNER. Concerning adaptive mutexs these are a very recent development and they did not make it in to either the latest FC11 of SLES11 kernels. Ideally, I'd love to see this kernel change appear in one of these distros because it does help performance. From Linux kernel commit: 0d66bf6d3514b35eb6897629059443132992dbd7 "Testing with Ingo's test-mutex application... gave a 345% boost for VFS scalability on my testbox" However, if you don't want to backport this change yourself you can still simply export the task_curr() symbol. The kmutex_t implementation will use this symbol when it's available to provide it's own adaptive mutexs. Finally, DEBUG_MUTEX support was removed including the proc handlers. This was done because now that we are cleanly integrated with the kernel profiling all this information and much much more is available in debug kernel builds. This code was now redundant. Update mutexs validated on: - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-25 21:47:01 +00:00
#include <linux/mutex.h>
Fix compilation error due to undefined ACCESS_ONCE macro. When CONFIG_DEBUG_MUTEXES is turned on in RHEL5's kernel config, the mutexes store the owner for debugging purposes, therefore the SPL will enable HAVE_MUTEX_OWNER. However, the SPL code uses ACCESS_ONCE() to access the owner, and this macro is not defined in the RHEL5 kernel, therefore we define it ourselves in include/linux/compiler_compat.h. Signed-off-by: Ricardo M. Correia <ricardo.correia@oracle.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2010-07-20 20:47:37 +00:00
#include <linux/compiler_compat.h>
Add new lock types MUTEX_NOLOCKDEP, and RW_NOLOCKDEP When running a kernel with CONFIG_LOCKDEP=y, lockdep reports possible recursive locking in some cases and possible circular locking dependency in others, within the SPL and ZFS modules. When lockdep detects these conditions, it disables further lock analysis for all locks. This causes /proc/lock_stats not to reflect full information about lock contention, even in locks without dependency issues. This commit creates a new type of mutex, MUTEX_NOLOCKDEP. This mutex type causes subsequent attempts to take or release those locks to be wrapped in lockdep_off() and lockdep_on(). This commit also creates an RW_NOLOCKDEP type analagous to MUTEX_NOLOCKDEP. MUTEX_NOLOCKDEP and RW_NOLOCKDEP are also defined in zfs, in a commit to that repo, for userspace builds. Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #480
2015-10-06 21:01:46 +00:00
#include <linux/lockdep.h>
Reimplement mutexs for Linux lock profiling/analysis For a generic explanation of why mutexs needed to be reimplemented to work with the kernel lock profiling see commits: e811949a57044d60d12953c5c3b808a79a7d36ef and d28db80fd0fd4fd63aec09037c44408e51a222d6 The specific changes made to the mutex implemetation are as follows. The Linux mutex structure is now directly embedded in the kmutex_t. This allows a kmutex_t to be directly case to a mutex struct and passed directly to the Linux primative. Just like with the rwlocks it is critical that these functions be implemented as '#defines to ensure the location information is preserved. The preprocessor can then do a direct replacement of the Solaris primative with the linux primative. Just as with the rwlocks we need to track the lock owner. Here things get a little more interesting because depending on your kernel version, and how you've built your kernel Linux may already do this for you. If your running a 2.6.29 or newer kernel on a SMP system the lock owner will be tracked. This was added to Linux to support adaptive mutexs, more on that shortly. Alternately, your kernel might track the lock owner if you've set CONFIG_DEBUG_MUTEXES in the kernel build. If neither of the above things is true for your kernel the kmutex_t type will include and track the lock owner to ensure correct behavior. This is all handled by a new autoconf check called SPL_AC_MUTEX_OWNER. Concerning adaptive mutexs these are a very recent development and they did not make it in to either the latest FC11 of SLES11 kernels. Ideally, I'd love to see this kernel change appear in one of these distros because it does help performance. From Linux kernel commit: 0d66bf6d3514b35eb6897629059443132992dbd7 "Testing with Ingo's test-mutex application... gave a 345% boost for VFS scalability on my testbox" However, if you don't want to backport this change yourself you can still simply export the task_curr() symbol. The kmutex_t implementation will use this symbol when it's available to provide it's own adaptive mutexs. Finally, DEBUG_MUTEX support was removed including the proc handlers. This was done because now that we are cleanly integrated with the kernel profiling all this information and much much more is available in debug kernel builds. This code was now redundant. Update mutexs validated on: - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-25 21:47:01 +00:00
typedef enum {
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
MUTEX_DEFAULT = 0,
MUTEX_SPIN = 1,
Add new lock types MUTEX_NOLOCKDEP, and RW_NOLOCKDEP When running a kernel with CONFIG_LOCKDEP=y, lockdep reports possible recursive locking in some cases and possible circular locking dependency in others, within the SPL and ZFS modules. When lockdep detects these conditions, it disables further lock analysis for all locks. This causes /proc/lock_stats not to reflect full information about lock contention, even in locks without dependency issues. This commit creates a new type of mutex, MUTEX_NOLOCKDEP. This mutex type causes subsequent attempts to take or release those locks to be wrapped in lockdep_off() and lockdep_on(). This commit also creates an RW_NOLOCKDEP type analagous to MUTEX_NOLOCKDEP. MUTEX_NOLOCKDEP and RW_NOLOCKDEP are also defined in zfs, in a commit to that repo, for userspace builds. Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #480
2015-10-06 21:01:46 +00:00
MUTEX_ADAPTIVE = 2,
MUTEX_NOLOCKDEP = 3
Reimplement mutexs for Linux lock profiling/analysis For a generic explanation of why mutexs needed to be reimplemented to work with the kernel lock profiling see commits: e811949a57044d60d12953c5c3b808a79a7d36ef and d28db80fd0fd4fd63aec09037c44408e51a222d6 The specific changes made to the mutex implemetation are as follows. The Linux mutex structure is now directly embedded in the kmutex_t. This allows a kmutex_t to be directly case to a mutex struct and passed directly to the Linux primative. Just like with the rwlocks it is critical that these functions be implemented as '#defines to ensure the location information is preserved. The preprocessor can then do a direct replacement of the Solaris primative with the linux primative. Just as with the rwlocks we need to track the lock owner. Here things get a little more interesting because depending on your kernel version, and how you've built your kernel Linux may already do this for you. If your running a 2.6.29 or newer kernel on a SMP system the lock owner will be tracked. This was added to Linux to support adaptive mutexs, more on that shortly. Alternately, your kernel might track the lock owner if you've set CONFIG_DEBUG_MUTEXES in the kernel build. If neither of the above things is true for your kernel the kmutex_t type will include and track the lock owner to ensure correct behavior. This is all handled by a new autoconf check called SPL_AC_MUTEX_OWNER. Concerning adaptive mutexs these are a very recent development and they did not make it in to either the latest FC11 of SLES11 kernels. Ideally, I'd love to see this kernel change appear in one of these distros because it does help performance. From Linux kernel commit: 0d66bf6d3514b35eb6897629059443132992dbd7 "Testing with Ingo's test-mutex application... gave a 345% boost for VFS scalability on my testbox" However, if you don't want to backport this change yourself you can still simply export the task_curr() symbol. The kmutex_t implementation will use this symbol when it's available to provide it's own adaptive mutexs. Finally, DEBUG_MUTEX support was removed including the proc handlers. This was done because now that we are cleanly integrated with the kernel profiling all this information and much much more is available in debug kernel builds. This code was now redundant. Update mutexs validated on: - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-25 21:47:01 +00:00
} kmutex_type_t;
Lots of build fixes. This is turning out to be a very good idea since it forcefully codifing the ABI. Since the shim layer is no longer linked at build time in to the test suite we can;'t cut any corners and get away with it. Everything is working now with the exception of sorting setting Module.symvers properly. This may take a little Makefile reorg. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@5 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-27 19:09:51 +00:00
Initial commit. All spl source written up to this point wrapped in an initial reasonable autoconf style build system. This does not yet build but the configure system does appear to work properly and integrate with the kernel. Hopefully the next commit gets us back to a buildable version we can run the test suite against. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@1 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-26 20:36:04 +00:00
typedef struct {
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
struct mutex m_mutex;
spinlock_t m_lock; /* used for serializing mutex_exit */
Revert "Remove misguided HAVE_MUTEX_OWNER check" This reverts commit d89616fda88bc030aaff758d37ede7d35e58841a which introduced some build failures which need to be resolved before this can be merged. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #633
2017-08-02 19:07:11 +00:00
#ifndef HAVE_MUTEX_OWNER
Use kernel provided mutex owner To reduce mutex footprint, we detect the existence of owner in kernel mutex, and rely on it if it exists. Note that before Linux 3.0, mutex owner is of type thread_info. Also note that, in Linux 3.18, the condition for owner is changed from CONFIG_DEBUG_MUTEXES || CONFIG_SMP to CONFIG_DEBUG_MUTEXES || CONFIG_MUTEX_SPIN_ON_OWNER Signed-off-by: Chunwei Chen <david.chen@osnexus.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #540
2016-04-12 19:05:14 +00:00
/* only when kernel doesn't have owner */
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
kthread_t *m_owner;
Revert "Remove misguided HAVE_MUTEX_OWNER check" This reverts commit d89616fda88bc030aaff758d37ede7d35e58841a which introduced some build failures which need to be resolved before this can be merged. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #633
2017-08-02 19:07:11 +00:00
#endif
Add new lock types MUTEX_NOLOCKDEP, and RW_NOLOCKDEP When running a kernel with CONFIG_LOCKDEP=y, lockdep reports possible recursive locking in some cases and possible circular locking dependency in others, within the SPL and ZFS modules. When lockdep detects these conditions, it disables further lock analysis for all locks. This causes /proc/lock_stats not to reflect full information about lock contention, even in locks without dependency issues. This commit creates a new type of mutex, MUTEX_NOLOCKDEP. This mutex type causes subsequent attempts to take or release those locks to be wrapped in lockdep_off() and lockdep_on(). This commit also creates an RW_NOLOCKDEP type analagous to MUTEX_NOLOCKDEP. MUTEX_NOLOCKDEP and RW_NOLOCKDEP are also defined in zfs, in a commit to that repo, for userspace builds. Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #480
2015-10-06 21:01:46 +00:00
#ifdef CONFIG_LOCKDEP
kmutex_type_t m_type;
#endif /* CONFIG_LOCKDEP */
Initial commit. All spl source written up to this point wrapped in an initial reasonable autoconf style build system. This does not yet build but the configure system does appear to work properly and integrate with the kernel. Hopefully the next commit gets us back to a buildable version we can run the test suite against. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@1 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-26 20:36:04 +00:00
} kmutex_t;
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
#define MUTEX(mp) (&((mp)->m_mutex))
Don't allow shrinking a PF_FSTRANS context Avoid deadlocks when entering the shrinker from a PF_FSTRANS context. This patch also reverts commit d0d5dd7 which added MUTEX_FSTRANS. Its use has been deprecated within ZFS as it was an ineffective mechanism to eliminate deadlocks. Among other things, it introduced the need for strict ordering of mutex locking and unlocking in order that the PF_FSTRANS flag wouldn't set incorrectly. Signed-off-by: Tim Chase <tim@chase2k.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #446
2015-03-31 12:49:15 +00:00
static inline void
spl_mutex_set_owner(kmutex_t *mp)
{
Use kernel provided mutex owner To reduce mutex footprint, we detect the existence of owner in kernel mutex, and rely on it if it exists. Note that before Linux 3.0, mutex owner is of type thread_info. Also note that, in Linux 3.18, the condition for owner is changed from CONFIG_DEBUG_MUTEXES || CONFIG_SMP to CONFIG_DEBUG_MUTEXES || CONFIG_MUTEX_SPIN_ON_OWNER Signed-off-by: Chunwei Chen <david.chen@osnexus.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #540
2016-04-12 19:05:14 +00:00
/*
* kernel will handle its owner, so we don't need to do anything if it
* is defined.
*/
Revert "Remove misguided HAVE_MUTEX_OWNER check" This reverts commit d89616fda88bc030aaff758d37ede7d35e58841a which introduced some build failures which need to be resolved before this can be merged. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #633
2017-08-02 19:07:11 +00:00
#ifndef HAVE_MUTEX_OWNER
Don't allow shrinking a PF_FSTRANS context Avoid deadlocks when entering the shrinker from a PF_FSTRANS context. This patch also reverts commit d0d5dd7 which added MUTEX_FSTRANS. Its use has been deprecated within ZFS as it was an ineffective mechanism to eliminate deadlocks. Among other things, it introduced the need for strict ordering of mutex locking and unlocking in order that the PF_FSTRANS flag wouldn't set incorrectly. Signed-off-by: Tim Chase <tim@chase2k.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #446
2015-03-31 12:49:15 +00:00
mp->m_owner = current;
Revert "Remove misguided HAVE_MUTEX_OWNER check" This reverts commit d89616fda88bc030aaff758d37ede7d35e58841a which introduced some build failures which need to be resolved before this can be merged. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #633
2017-08-02 19:07:11 +00:00
#endif
Don't allow shrinking a PF_FSTRANS context Avoid deadlocks when entering the shrinker from a PF_FSTRANS context. This patch also reverts commit d0d5dd7 which added MUTEX_FSTRANS. Its use has been deprecated within ZFS as it was an ineffective mechanism to eliminate deadlocks. Among other things, it introduced the need for strict ordering of mutex locking and unlocking in order that the PF_FSTRANS flag wouldn't set incorrectly. Signed-off-by: Tim Chase <tim@chase2k.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #446
2015-03-31 12:49:15 +00:00
}
static inline void
spl_mutex_clear_owner(kmutex_t *mp)
{
Revert "Remove misguided HAVE_MUTEX_OWNER check" This reverts commit d89616fda88bc030aaff758d37ede7d35e58841a which introduced some build failures which need to be resolved before this can be merged. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #633
2017-08-02 19:07:11 +00:00
#ifndef HAVE_MUTEX_OWNER
Don't allow shrinking a PF_FSTRANS context Avoid deadlocks when entering the shrinker from a PF_FSTRANS context. This patch also reverts commit d0d5dd7 which added MUTEX_FSTRANS. Its use has been deprecated within ZFS as it was an ineffective mechanism to eliminate deadlocks. Among other things, it introduced the need for strict ordering of mutex locking and unlocking in order that the PF_FSTRANS flag wouldn't set incorrectly. Signed-off-by: Tim Chase <tim@chase2k.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #446
2015-03-31 12:49:15 +00:00
mp->m_owner = NULL;
Revert "Remove misguided HAVE_MUTEX_OWNER check" This reverts commit d89616fda88bc030aaff758d37ede7d35e58841a which introduced some build failures which need to be resolved before this can be merged. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #633
2017-08-02 19:07:11 +00:00
#endif
Don't allow shrinking a PF_FSTRANS context Avoid deadlocks when entering the shrinker from a PF_FSTRANS context. This patch also reverts commit d0d5dd7 which added MUTEX_FSTRANS. Its use has been deprecated within ZFS as it was an ineffective mechanism to eliminate deadlocks. Among other things, it introduced the need for strict ordering of mutex locking and unlocking in order that the PF_FSTRANS flag wouldn't set incorrectly. Signed-off-by: Tim Chase <tim@chase2k.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #446
2015-03-31 12:49:15 +00:00
}
Revert "Remove misguided HAVE_MUTEX_OWNER check" This reverts commit d89616fda88bc030aaff758d37ede7d35e58841a which introduced some build failures which need to be resolved before this can be merged. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #633
2017-08-02 19:07:11 +00:00
#ifdef HAVE_MUTEX_OWNER
Use kernel provided mutex owner To reduce mutex footprint, we detect the existence of owner in kernel mutex, and rely on it if it exists. Note that before Linux 3.0, mutex owner is of type thread_info. Also note that, in Linux 3.18, the condition for owner is changed from CONFIG_DEBUG_MUTEXES || CONFIG_SMP to CONFIG_DEBUG_MUTEXES || CONFIG_MUTEX_SPIN_ON_OWNER Signed-off-by: Chunwei Chen <david.chen@osnexus.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #540
2016-04-12 19:05:14 +00:00
#define mutex_owner(mp) (ACCESS_ONCE(MUTEX(mp)->owner))
Revert "Remove misguided HAVE_MUTEX_OWNER check" This reverts commit d89616fda88bc030aaff758d37ede7d35e58841a which introduced some build failures which need to be resolved before this can be merged. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #633
2017-08-02 19:07:11 +00:00
#else
#define mutex_owner(mp) (ACCESS_ONCE((mp)->m_owner))
#endif
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
#define mutex_owned(mp) (mutex_owner(mp) == current)
#define MUTEX_HELD(mp) mutex_owned(mp)
#define MUTEX_NOT_HELD(mp) (!MUTEX_HELD(mp))
Initial commit. All spl source written up to this point wrapped in an initial reasonable autoconf style build system. This does not yet build but the configure system does appear to work properly and integrate with the kernel. Hopefully the next commit gets us back to a buildable version we can run the test suite against. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@1 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-26 20:36:04 +00:00
Add new lock types MUTEX_NOLOCKDEP, and RW_NOLOCKDEP When running a kernel with CONFIG_LOCKDEP=y, lockdep reports possible recursive locking in some cases and possible circular locking dependency in others, within the SPL and ZFS modules. When lockdep detects these conditions, it disables further lock analysis for all locks. This causes /proc/lock_stats not to reflect full information about lock contention, even in locks without dependency issues. This commit creates a new type of mutex, MUTEX_NOLOCKDEP. This mutex type causes subsequent attempts to take or release those locks to be wrapped in lockdep_off() and lockdep_on(). This commit also creates an RW_NOLOCKDEP type analagous to MUTEX_NOLOCKDEP. MUTEX_NOLOCKDEP and RW_NOLOCKDEP are also defined in zfs, in a commit to that repo, for userspace builds. Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #480
2015-10-06 21:01:46 +00:00
#ifdef CONFIG_LOCKDEP
static inline void
spl_mutex_set_type(kmutex_t *mp, kmutex_type_t type)
{
mp->m_type = type;
}
static inline void
spl_mutex_lockdep_off_maybe(kmutex_t *mp) \
{ \
if (mp && mp->m_type == MUTEX_NOLOCKDEP) \
lockdep_off(); \
}
static inline void
spl_mutex_lockdep_on_maybe(kmutex_t *mp) \
{ \
if (mp && mp->m_type == MUTEX_NOLOCKDEP) \
lockdep_on(); \
}
#else /* CONFIG_LOCKDEP */
#define spl_mutex_set_type(mp, type)
#define spl_mutex_lockdep_off_maybe(mp)
#define spl_mutex_lockdep_on_maybe(mp)
#endif /* CONFIG_LOCKDEP */
Reimplement mutexs for Linux lock profiling/analysis For a generic explanation of why mutexs needed to be reimplemented to work with the kernel lock profiling see commits: e811949a57044d60d12953c5c3b808a79a7d36ef and d28db80fd0fd4fd63aec09037c44408e51a222d6 The specific changes made to the mutex implemetation are as follows. The Linux mutex structure is now directly embedded in the kmutex_t. This allows a kmutex_t to be directly case to a mutex struct and passed directly to the Linux primative. Just like with the rwlocks it is critical that these functions be implemented as '#defines to ensure the location information is preserved. The preprocessor can then do a direct replacement of the Solaris primative with the linux primative. Just as with the rwlocks we need to track the lock owner. Here things get a little more interesting because depending on your kernel version, and how you've built your kernel Linux may already do this for you. If your running a 2.6.29 or newer kernel on a SMP system the lock owner will be tracked. This was added to Linux to support adaptive mutexs, more on that shortly. Alternately, your kernel might track the lock owner if you've set CONFIG_DEBUG_MUTEXES in the kernel build. If neither of the above things is true for your kernel the kmutex_t type will include and track the lock owner to ensure correct behavior. This is all handled by a new autoconf check called SPL_AC_MUTEX_OWNER. Concerning adaptive mutexs these are a very recent development and they did not make it in to either the latest FC11 of SLES11 kernels. Ideally, I'd love to see this kernel change appear in one of these distros because it does help performance. From Linux kernel commit: 0d66bf6d3514b35eb6897629059443132992dbd7 "Testing with Ingo's test-mutex application... gave a 345% boost for VFS scalability on my testbox" However, if you don't want to backport this change yourself you can still simply export the task_curr() symbol. The kmutex_t implementation will use this symbol when it's available to provide it's own adaptive mutexs. Finally, DEBUG_MUTEX support was removed including the proc handlers. This was done because now that we are cleanly integrated with the kernel profiling all this information and much much more is available in debug kernel builds. This code was now redundant. Update mutexs validated on: - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-25 21:47:01 +00:00
/*
* The following functions must be a #define and not static inline.
* This ensures that the native linux mutex functions (lock/unlock)
* will be correctly located in the users code which is important
* for the built in kernel lock analysis tools
*/
Commit adaptive mutexes. This seems to have introduced some new crashes but it's not clear to me yet if these are a problem with the mutex implementation or ZFSs usage of it. Minor taskq fixes to add new tasks to the end of the pending list. Minor enhansements to the debug infrastructure. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@94 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-05 20:18:49 +00:00
#undef mutex_init
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
#define mutex_init(mp, name, type, ibc) \
{ \
static struct lock_class_key __key; \
Add new lock types MUTEX_NOLOCKDEP, and RW_NOLOCKDEP When running a kernel with CONFIG_LOCKDEP=y, lockdep reports possible recursive locking in some cases and possible circular locking dependency in others, within the SPL and ZFS modules. When lockdep detects these conditions, it disables further lock analysis for all locks. This causes /proc/lock_stats not to reflect full information about lock contention, even in locks without dependency issues. This commit creates a new type of mutex, MUTEX_NOLOCKDEP. This mutex type causes subsequent attempts to take or release those locks to be wrapped in lockdep_off() and lockdep_on(). This commit also creates an RW_NOLOCKDEP type analagous to MUTEX_NOLOCKDEP. MUTEX_NOLOCKDEP and RW_NOLOCKDEP are also defined in zfs, in a commit to that repo, for userspace builds. Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #480
2015-10-06 21:01:46 +00:00
ASSERT(type == MUTEX_DEFAULT || type == MUTEX_NOLOCKDEP); \
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
\
Add mutex_enter_nested() which maps to mutex_lock_nested() Also add support for the "name" parameter in mutex_init(). The name allows for better diagnostics, namely in /proc/lock_stats when lock debugging is enabled. Nested mutexes are necessary to support CONFIG_PROVE_LOCKING. ZoL can use mutex_enter_nested()'s "class" argument to to convey the locking hierarchy. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Closes #439
2015-03-20 19:03:26 +00:00
__mutex_init(MUTEX(mp), (name) ? (#name) : (#mp), &__key); \
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
spin_lock_init(&(mp)->m_lock); \
Don't allow shrinking a PF_FSTRANS context Avoid deadlocks when entering the shrinker from a PF_FSTRANS context. This patch also reverts commit d0d5dd7 which added MUTEX_FSTRANS. Its use has been deprecated within ZFS as it was an ineffective mechanism to eliminate deadlocks. Among other things, it introduced the need for strict ordering of mutex locking and unlocking in order that the PF_FSTRANS flag wouldn't set incorrectly. Signed-off-by: Tim Chase <tim@chase2k.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #446
2015-03-31 12:49:15 +00:00
spl_mutex_clear_owner(mp); \
Add new lock types MUTEX_NOLOCKDEP, and RW_NOLOCKDEP When running a kernel with CONFIG_LOCKDEP=y, lockdep reports possible recursive locking in some cases and possible circular locking dependency in others, within the SPL and ZFS modules. When lockdep detects these conditions, it disables further lock analysis for all locks. This causes /proc/lock_stats not to reflect full information about lock contention, even in locks without dependency issues. This commit creates a new type of mutex, MUTEX_NOLOCKDEP. This mutex type causes subsequent attempts to take or release those locks to be wrapped in lockdep_off() and lockdep_on(). This commit also creates an RW_NOLOCKDEP type analagous to MUTEX_NOLOCKDEP. MUTEX_NOLOCKDEP and RW_NOLOCKDEP are also defined in zfs, in a commit to that repo, for userspace builds. Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #480
2015-10-06 21:01:46 +00:00
spl_mutex_set_type(mp, type); \
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
}
Reimplement mutexs for Linux lock profiling/analysis For a generic explanation of why mutexs needed to be reimplemented to work with the kernel lock profiling see commits: e811949a57044d60d12953c5c3b808a79a7d36ef and d28db80fd0fd4fd63aec09037c44408e51a222d6 The specific changes made to the mutex implemetation are as follows. The Linux mutex structure is now directly embedded in the kmutex_t. This allows a kmutex_t to be directly case to a mutex struct and passed directly to the Linux primative. Just like with the rwlocks it is critical that these functions be implemented as '#defines to ensure the location information is preserved. The preprocessor can then do a direct replacement of the Solaris primative with the linux primative. Just as with the rwlocks we need to track the lock owner. Here things get a little more interesting because depending on your kernel version, and how you've built your kernel Linux may already do this for you. If your running a 2.6.29 or newer kernel on a SMP system the lock owner will be tracked. This was added to Linux to support adaptive mutexs, more on that shortly. Alternately, your kernel might track the lock owner if you've set CONFIG_DEBUG_MUTEXES in the kernel build. If neither of the above things is true for your kernel the kmutex_t type will include and track the lock owner to ensure correct behavior. This is all handled by a new autoconf check called SPL_AC_MUTEX_OWNER. Concerning adaptive mutexs these are a very recent development and they did not make it in to either the latest FC11 of SLES11 kernels. Ideally, I'd love to see this kernel change appear in one of these distros because it does help performance. From Linux kernel commit: 0d66bf6d3514b35eb6897629059443132992dbd7 "Testing with Ingo's test-mutex application... gave a 345% boost for VFS scalability on my testbox" However, if you don't want to backport this change yourself you can still simply export the task_curr() symbol. The kmutex_t implementation will use this symbol when it's available to provide it's own adaptive mutexs. Finally, DEBUG_MUTEX support was removed including the proc handlers. This was done because now that we are cleanly integrated with the kernel profiling all this information and much much more is available in debug kernel builds. This code was now redundant. Update mutexs validated on: - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-25 21:47:01 +00:00
Commit adaptive mutexes. This seems to have introduced some new crashes but it's not clear to me yet if these are a problem with the mutex implementation or ZFSs usage of it. Minor taskq fixes to add new tasks to the end of the pending list. Minor enhansements to the debug infrastructure. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@94 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-05 20:18:49 +00:00
#undef mutex_destroy
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
#define mutex_destroy(mp) \
{ \
VERIFY3P(mutex_owner(mp), ==, NULL); \
}
- Add some spinlocks to cover all the private data in the mutex. I don't think this should fix anything but it's a good idea regardless. - Drop the lock before calling the construct/destructor for the slab otherwise we can't sleep in a constructor/destructor and for long running functions we may NMI. - Do something braindead, but safe for the console debug logs for now. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@73 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-15 20:53:36 +00:00
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
#define mutex_tryenter(mp) \
({ \
int _rc_; \
\
Add new lock types MUTEX_NOLOCKDEP, and RW_NOLOCKDEP When running a kernel with CONFIG_LOCKDEP=y, lockdep reports possible recursive locking in some cases and possible circular locking dependency in others, within the SPL and ZFS modules. When lockdep detects these conditions, it disables further lock analysis for all locks. This causes /proc/lock_stats not to reflect full information about lock contention, even in locks without dependency issues. This commit creates a new type of mutex, MUTEX_NOLOCKDEP. This mutex type causes subsequent attempts to take or release those locks to be wrapped in lockdep_off() and lockdep_on(). This commit also creates an RW_NOLOCKDEP type analagous to MUTEX_NOLOCKDEP. MUTEX_NOLOCKDEP and RW_NOLOCKDEP are also defined in zfs, in a commit to that repo, for userspace builds. Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #480
2015-10-06 21:01:46 +00:00
spl_mutex_lockdep_off_maybe(mp); \
Don't allow shrinking a PF_FSTRANS context Avoid deadlocks when entering the shrinker from a PF_FSTRANS context. This patch also reverts commit d0d5dd7 which added MUTEX_FSTRANS. Its use has been deprecated within ZFS as it was an ineffective mechanism to eliminate deadlocks. Among other things, it introduced the need for strict ordering of mutex locking and unlocking in order that the PF_FSTRANS flag wouldn't set incorrectly. Signed-off-by: Tim Chase <tim@chase2k.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #446
2015-03-31 12:49:15 +00:00
if ((_rc_ = mutex_trylock(MUTEX(mp))) == 1) \
spl_mutex_set_owner(mp); \
Add new lock types MUTEX_NOLOCKDEP, and RW_NOLOCKDEP When running a kernel with CONFIG_LOCKDEP=y, lockdep reports possible recursive locking in some cases and possible circular locking dependency in others, within the SPL and ZFS modules. When lockdep detects these conditions, it disables further lock analysis for all locks. This causes /proc/lock_stats not to reflect full information about lock contention, even in locks without dependency issues. This commit creates a new type of mutex, MUTEX_NOLOCKDEP. This mutex type causes subsequent attempts to take or release those locks to be wrapped in lockdep_off() and lockdep_on(). This commit also creates an RW_NOLOCKDEP type analagous to MUTEX_NOLOCKDEP. MUTEX_NOLOCKDEP and RW_NOLOCKDEP are also defined in zfs, in a commit to that repo, for userspace builds. Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #480
2015-10-06 21:01:46 +00:00
spl_mutex_lockdep_on_maybe(mp); \
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
\
_rc_; \
Commit adaptive mutexes. This seems to have introduced some new crashes but it's not clear to me yet if these are a problem with the mutex implementation or ZFSs usage of it. Minor taskq fixes to add new tasks to the end of the pending list. Minor enhansements to the debug infrastructure. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@94 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-05 20:18:49 +00:00
})
Initial commit. All spl source written up to this point wrapped in an initial reasonable autoconf style build system. This does not yet build but the configure system does appear to work properly and integrate with the kernel. Hopefully the next commit gets us back to a buildable version we can run the test suite against. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@1 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-26 20:36:04 +00:00
Add mutex_enter_nested() which maps to mutex_lock_nested() Also add support for the "name" parameter in mutex_init(). The name allows for better diagnostics, namely in /proc/lock_stats when lock debugging is enabled. Nested mutexes are necessary to support CONFIG_PROVE_LOCKING. ZoL can use mutex_enter_nested()'s "class" argument to to convey the locking hierarchy. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Closes #439
2015-03-20 19:03:26 +00:00
#ifdef CONFIG_DEBUG_LOCK_ALLOC
#define mutex_enter_nested(mp, subclass) \
{ \
ASSERT3P(mutex_owner(mp), !=, current); \
Add new lock types MUTEX_NOLOCKDEP, and RW_NOLOCKDEP When running a kernel with CONFIG_LOCKDEP=y, lockdep reports possible recursive locking in some cases and possible circular locking dependency in others, within the SPL and ZFS modules. When lockdep detects these conditions, it disables further lock analysis for all locks. This causes /proc/lock_stats not to reflect full information about lock contention, even in locks without dependency issues. This commit creates a new type of mutex, MUTEX_NOLOCKDEP. This mutex type causes subsequent attempts to take or release those locks to be wrapped in lockdep_off() and lockdep_on(). This commit also creates an RW_NOLOCKDEP type analagous to MUTEX_NOLOCKDEP. MUTEX_NOLOCKDEP and RW_NOLOCKDEP are also defined in zfs, in a commit to that repo, for userspace builds. Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #480
2015-10-06 21:01:46 +00:00
spl_mutex_lockdep_off_maybe(mp); \
Add mutex_enter_nested() which maps to mutex_lock_nested() Also add support for the "name" parameter in mutex_init(). The name allows for better diagnostics, namely in /proc/lock_stats when lock debugging is enabled. Nested mutexes are necessary to support CONFIG_PROVE_LOCKING. ZoL can use mutex_enter_nested()'s "class" argument to to convey the locking hierarchy. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Closes #439
2015-03-20 19:03:26 +00:00
mutex_lock_nested(MUTEX(mp), (subclass)); \
Add new lock types MUTEX_NOLOCKDEP, and RW_NOLOCKDEP When running a kernel with CONFIG_LOCKDEP=y, lockdep reports possible recursive locking in some cases and possible circular locking dependency in others, within the SPL and ZFS modules. When lockdep detects these conditions, it disables further lock analysis for all locks. This causes /proc/lock_stats not to reflect full information about lock contention, even in locks without dependency issues. This commit creates a new type of mutex, MUTEX_NOLOCKDEP. This mutex type causes subsequent attempts to take or release those locks to be wrapped in lockdep_off() and lockdep_on(). This commit also creates an RW_NOLOCKDEP type analagous to MUTEX_NOLOCKDEP. MUTEX_NOLOCKDEP and RW_NOLOCKDEP are also defined in zfs, in a commit to that repo, for userspace builds. Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #480
2015-10-06 21:01:46 +00:00
spl_mutex_lockdep_on_maybe(mp); \
Don't allow shrinking a PF_FSTRANS context Avoid deadlocks when entering the shrinker from a PF_FSTRANS context. This patch also reverts commit d0d5dd7 which added MUTEX_FSTRANS. Its use has been deprecated within ZFS as it was an ineffective mechanism to eliminate deadlocks. Among other things, it introduced the need for strict ordering of mutex locking and unlocking in order that the PF_FSTRANS flag wouldn't set incorrectly. Signed-off-by: Tim Chase <tim@chase2k.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #446
2015-03-31 12:49:15 +00:00
spl_mutex_set_owner(mp); \
Add mutex_enter_nested() which maps to mutex_lock_nested() Also add support for the "name" parameter in mutex_init(). The name allows for better diagnostics, namely in /proc/lock_stats when lock debugging is enabled. Nested mutexes are necessary to support CONFIG_PROVE_LOCKING. ZoL can use mutex_enter_nested()'s "class" argument to to convey the locking hierarchy. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Closes #439
2015-03-20 19:03:26 +00:00
}
#else /* CONFIG_DEBUG_LOCK_ALLOC */
#define mutex_enter_nested(mp, subclass) \
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
{ \
ASSERT3P(mutex_owner(mp), !=, current); \
Add new lock types MUTEX_NOLOCKDEP, and RW_NOLOCKDEP When running a kernel with CONFIG_LOCKDEP=y, lockdep reports possible recursive locking in some cases and possible circular locking dependency in others, within the SPL and ZFS modules. When lockdep detects these conditions, it disables further lock analysis for all locks. This causes /proc/lock_stats not to reflect full information about lock contention, even in locks without dependency issues. This commit creates a new type of mutex, MUTEX_NOLOCKDEP. This mutex type causes subsequent attempts to take or release those locks to be wrapped in lockdep_off() and lockdep_on(). This commit also creates an RW_NOLOCKDEP type analagous to MUTEX_NOLOCKDEP. MUTEX_NOLOCKDEP and RW_NOLOCKDEP are also defined in zfs, in a commit to that repo, for userspace builds. Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #480
2015-10-06 21:01:46 +00:00
spl_mutex_lockdep_off_maybe(mp); \
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
mutex_lock(MUTEX(mp)); \
Add new lock types MUTEX_NOLOCKDEP, and RW_NOLOCKDEP When running a kernel with CONFIG_LOCKDEP=y, lockdep reports possible recursive locking in some cases and possible circular locking dependency in others, within the SPL and ZFS modules. When lockdep detects these conditions, it disables further lock analysis for all locks. This causes /proc/lock_stats not to reflect full information about lock contention, even in locks without dependency issues. This commit creates a new type of mutex, MUTEX_NOLOCKDEP. This mutex type causes subsequent attempts to take or release those locks to be wrapped in lockdep_off() and lockdep_on(). This commit also creates an RW_NOLOCKDEP type analagous to MUTEX_NOLOCKDEP. MUTEX_NOLOCKDEP and RW_NOLOCKDEP are also defined in zfs, in a commit to that repo, for userspace builds. Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #480
2015-10-06 21:01:46 +00:00
spl_mutex_lockdep_on_maybe(mp); \
Don't allow shrinking a PF_FSTRANS context Avoid deadlocks when entering the shrinker from a PF_FSTRANS context. This patch also reverts commit d0d5dd7 which added MUTEX_FSTRANS. Its use has been deprecated within ZFS as it was an ineffective mechanism to eliminate deadlocks. Among other things, it introduced the need for strict ordering of mutex locking and unlocking in order that the PF_FSTRANS flag wouldn't set incorrectly. Signed-off-by: Tim Chase <tim@chase2k.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #446
2015-03-31 12:49:15 +00:00
spl_mutex_set_owner(mp); \
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
}
Add mutex_enter_nested() which maps to mutex_lock_nested() Also add support for the "name" parameter in mutex_init(). The name allows for better diagnostics, namely in /proc/lock_stats when lock debugging is enabled. Nested mutexes are necessary to support CONFIG_PROVE_LOCKING. ZoL can use mutex_enter_nested()'s "class" argument to to convey the locking hierarchy. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Closes #439
2015-03-20 19:03:26 +00:00
#endif /* CONFIG_DEBUG_LOCK_ALLOC */
#define mutex_enter(mp) mutex_enter_nested((mp), 0)
Reimplement mutexs for Linux lock profiling/analysis For a generic explanation of why mutexs needed to be reimplemented to work with the kernel lock profiling see commits: e811949a57044d60d12953c5c3b808a79a7d36ef and d28db80fd0fd4fd63aec09037c44408e51a222d6 The specific changes made to the mutex implemetation are as follows. The Linux mutex structure is now directly embedded in the kmutex_t. This allows a kmutex_t to be directly case to a mutex struct and passed directly to the Linux primative. Just like with the rwlocks it is critical that these functions be implemented as '#defines to ensure the location information is preserved. The preprocessor can then do a direct replacement of the Solaris primative with the linux primative. Just as with the rwlocks we need to track the lock owner. Here things get a little more interesting because depending on your kernel version, and how you've built your kernel Linux may already do this for you. If your running a 2.6.29 or newer kernel on a SMP system the lock owner will be tracked. This was added to Linux to support adaptive mutexs, more on that shortly. Alternately, your kernel might track the lock owner if you've set CONFIG_DEBUG_MUTEXES in the kernel build. If neither of the above things is true for your kernel the kmutex_t type will include and track the lock owner to ensure correct behavior. This is all handled by a new autoconf check called SPL_AC_MUTEX_OWNER. Concerning adaptive mutexs these are a very recent development and they did not make it in to either the latest FC11 of SLES11 kernels. Ideally, I'd love to see this kernel change appear in one of these distros because it does help performance. From Linux kernel commit: 0d66bf6d3514b35eb6897629059443132992dbd7 "Testing with Ingo's test-mutex application... gave a 345% boost for VFS scalability on my testbox" However, if you don't want to backport this change yourself you can still simply export the task_curr() symbol. The kmutex_t implementation will use this symbol when it's available to provide it's own adaptive mutexs. Finally, DEBUG_MUTEX support was removed including the proc handlers. This was done because now that we are cleanly integrated with the kernel profiling all this information and much much more is available in debug kernel builds. This code was now redundant. Update mutexs validated on: - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-25 21:47:01 +00:00
Retire MUTEX_OWNER checks To minimize the size of a kmutex_t a MUTEX_OWNER check was added. It allowed the kmutex_t wrapper to leverage the mutex owner which was already stored in the mutex for certain kernel configurations. The upside to this was that it reduced the size of the kmutex_t wrapper structure by the size of a task_struct pointer (4/8 bytes). The downside was that two mutex implementations needed to be maintained. Depending on your exact kernel configuration the correct one would be selected. Over the years this solution worked but it could be fragile since it depending heavily on assumed kernel mutex implementation details. For example the SPL_AC_MUTEX_OWNER_TASK_STRUCT configure check needed to be added when the kernel changed how the owner was stored. It also made the code more complicated than it needed to be. Therefore, in the name of simplicity and portability this optimization is being retired. It will slightly increase the memory requirements for a kmutex_t but only very slightly. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 18:23:49 +00:00
/*
* The reason for the spinlock:
*
* The Linux mutex is designed with a fast-path/slow-path design such that it
* does not guarantee serialization upon itself, allowing a race where latter
* acquirers finish mutex_unlock before former ones.
*
* The race renders it unsafe to be used for serializing the freeing of an
* object in which the mutex is embedded, where the latter acquirer could go
* on to free the object while the former one is still doing mutex_unlock and
* causing memory corruption.
*
* However, there are many places in ZFS where the mutex is used for
* serializing object freeing, and the code is shared among other OSes without
* this issue. Thus, we need the spinlock to force the serialization on
* mutex_exit().
*
* See http://lwn.net/Articles/575477/ for the information about the race.
*/
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
#define mutex_exit(mp) \
{ \
Add new lock types MUTEX_NOLOCKDEP, and RW_NOLOCKDEP When running a kernel with CONFIG_LOCKDEP=y, lockdep reports possible recursive locking in some cases and possible circular locking dependency in others, within the SPL and ZFS modules. When lockdep detects these conditions, it disables further lock analysis for all locks. This causes /proc/lock_stats not to reflect full information about lock contention, even in locks without dependency issues. This commit creates a new type of mutex, MUTEX_NOLOCKDEP. This mutex type causes subsequent attempts to take or release those locks to be wrapped in lockdep_off() and lockdep_on(). This commit also creates an RW_NOLOCKDEP type analagous to MUTEX_NOLOCKDEP. MUTEX_NOLOCKDEP and RW_NOLOCKDEP are also defined in zfs, in a commit to that repo, for userspace builds. Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #480
2015-10-06 21:01:46 +00:00
spl_mutex_lockdep_off_maybe(mp); \
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
spin_lock(&(mp)->m_lock); \
Don't allow shrinking a PF_FSTRANS context Avoid deadlocks when entering the shrinker from a PF_FSTRANS context. This patch also reverts commit d0d5dd7 which added MUTEX_FSTRANS. Its use has been deprecated within ZFS as it was an ineffective mechanism to eliminate deadlocks. Among other things, it introduced the need for strict ordering of mutex locking and unlocking in order that the PF_FSTRANS flag wouldn't set incorrectly. Signed-off-by: Tim Chase <tim@chase2k.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #446
2015-03-31 12:49:15 +00:00
spl_mutex_clear_owner(mp); \
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
mutex_unlock(MUTEX(mp)); \
spin_unlock(&(mp)->m_lock); \
Add new lock types MUTEX_NOLOCKDEP, and RW_NOLOCKDEP When running a kernel with CONFIG_LOCKDEP=y, lockdep reports possible recursive locking in some cases and possible circular locking dependency in others, within the SPL and ZFS modules. When lockdep detects these conditions, it disables further lock analysis for all locks. This causes /proc/lock_stats not to reflect full information about lock contention, even in locks without dependency issues. This commit creates a new type of mutex, MUTEX_NOLOCKDEP. This mutex type causes subsequent attempts to take or release those locks to be wrapped in lockdep_off() and lockdep_on(). This commit also creates an RW_NOLOCKDEP type analagous to MUTEX_NOLOCKDEP. MUTEX_NOLOCKDEP and RW_NOLOCKDEP are also defined in zfs, in a commit to that repo, for userspace builds. Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #480
2015-10-06 21:01:46 +00:00
spl_mutex_lockdep_on_maybe(mp); \
Fix cstyle issue in mutex.h This patch only addresses the issues identified by the style checker in mutex.h. It contains no functional changes. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tim Chase <tim@chase2k.com> Issue #435
2015-02-25 17:20:38 +00:00
}
Reimplement mutexs for Linux lock profiling/analysis For a generic explanation of why mutexs needed to be reimplemented to work with the kernel lock profiling see commits: e811949a57044d60d12953c5c3b808a79a7d36ef and d28db80fd0fd4fd63aec09037c44408e51a222d6 The specific changes made to the mutex implemetation are as follows. The Linux mutex structure is now directly embedded in the kmutex_t. This allows a kmutex_t to be directly case to a mutex struct and passed directly to the Linux primative. Just like with the rwlocks it is critical that these functions be implemented as '#defines to ensure the location information is preserved. The preprocessor can then do a direct replacement of the Solaris primative with the linux primative. Just as with the rwlocks we need to track the lock owner. Here things get a little more interesting because depending on your kernel version, and how you've built your kernel Linux may already do this for you. If your running a 2.6.29 or newer kernel on a SMP system the lock owner will be tracked. This was added to Linux to support adaptive mutexs, more on that shortly. Alternately, your kernel might track the lock owner if you've set CONFIG_DEBUG_MUTEXES in the kernel build. If neither of the above things is true for your kernel the kmutex_t type will include and track the lock owner to ensure correct behavior. This is all handled by a new autoconf check called SPL_AC_MUTEX_OWNER. Concerning adaptive mutexs these are a very recent development and they did not make it in to either the latest FC11 of SLES11 kernels. Ideally, I'd love to see this kernel change appear in one of these distros because it does help performance. From Linux kernel commit: 0d66bf6d3514b35eb6897629059443132992dbd7 "Testing with Ingo's test-mutex application... gave a 345% boost for VFS scalability on my testbox" However, if you don't want to backport this change yourself you can still simply export the task_curr() symbol. The kmutex_t implementation will use this symbol when it's available to provide it's own adaptive mutexs. Finally, DEBUG_MUTEX support was removed including the proc handlers. This was done because now that we are cleanly integrated with the kernel profiling all this information and much much more is available in debug kernel builds. This code was now redundant. Update mutexs validated on: - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-25 21:47:01 +00:00
int spl_mutex_init(void);
void spl_mutex_fini(void);
#endif /* _SPL_MUTEX_H */