Archive-Team
/
zfs
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 103 Wiki Activity
zfs/include/sys/rwlock.h

254 lines
9.9 KiB
C
Raw Normal View History

Public Release Prep Updated AUTHORS, COPYING, DISCLAIMER, and INSTALL files. Added standardized headers to all source file to clearly indicate the copyright, license, and to give credit where credit is due.
2010-05-17 22:18:00 +00:00
/*****************************************************************************\
* Copyright (C) 2007-2010 Lawrence Livermore National Security, LLC.
* Copyright (C) 2007 The Regents of the University of California.
* Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER).
* Written by Brian Behlendorf <behlendorf1@llnl.gov>.
Go through and add a header with the proper UCRL number. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@114 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-26 04:38:26 +00:00
* UCRL-CODE-235197
*
Public Release Prep Updated AUTHORS, COPYING, DISCLAIMER, and INSTALL files. Added standardized headers to all source file to clearly indicate the copyright, license, and to give credit where credit is due.
2010-05-17 22:18:00 +00:00
* This file is part of the SPL, Solaris Porting Layer.
* For details, see <http://github.com/behlendorf/spl/>.
*
* The SPL is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
* Free Software Foundation; either version 2 of the License, or (at your
* option) any later version.
Go through and add a header with the proper UCRL number. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@114 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-26 04:38:26 +00:00
*
Public Release Prep Updated AUTHORS, COPYING, DISCLAIMER, and INSTALL files. Added standardized headers to all source file to clearly indicate the copyright, license, and to give credit where credit is due.
2010-05-17 22:18:00 +00:00
* The SPL is distributed in the hope that it will be useful, but WITHOUT
Go through and add a header with the proper UCRL number. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@114 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-26 04:38:26 +00:00
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
* for more details.
*
* You should have received a copy of the GNU General Public License along
Public Release Prep Updated AUTHORS, COPYING, DISCLAIMER, and INSTALL files. Added standardized headers to all source file to clearly indicate the copyright, license, and to give credit where credit is due.
2010-05-17 22:18:00 +00:00
* with the SPL. If not, see <http://www.gnu.org/licenses/>.
\*****************************************************************************/
Go through and add a header with the proper UCRL number. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@114 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-26 04:38:26 +00:00
Minor nit, SOLARIS should be SPL git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@17 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-28 00:52:31 +00:00
#ifndef _SPL_RWLOCK_H
- Updated rwlock's to reside in a .c file instead of a static inline - Updated rwlock's so they can be safely initialized in ctors. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@96 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-06 23:00:49 +00:00
#define _SPL_RWLOCK_H
Initial commit. All spl source written up to this point wrapped in an initial reasonable autoconf style build system. This does not yet build but the configure system does appear to work properly and integrate with the kernel. Hopefully the next commit gets us back to a buildable version we can run the test suite against. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@1 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-26 20:36:04 +00:00
Reorganize /include/ to add a /sys/, this way we don't need to muck with #includes in existing Solaris style source to get it to find the right stuff. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@18 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-01 00:45:59 +00:00
#include <sys/types.h>
Update rwlocks to track owner to ensure correct semantics The behavior of RW_*_HELD was updated because it was not quite right. It is not sufficient to return non-zero when the lock is help, we must only do this when the current task in the holder. This means we need to track the lock owner which is not something tracked in a Linux semaphore. After some experimentation the solution I settled on was to embed the Linux semaphore at the start of a larger krwlock_t structure which includes the owner field. This maintains good performance and allows us to cleanly intergrate with the kernel lock analysis tools. My reasons: 1) By placing the Linux semaphore at the start of krwlock_t we can then simply cast krwlock_t to a rw_semaphore and pass that on to the linux kernel. This allows us to use '#defines so the preprocessor can do direct replacement of the Solaris primative with the linux equivilant. This is important because it then maintains the location information for each rw_* call point. 2) Additionally, by adding the owner to krwlock_t we can keep this needed extra information adjacent to the lock itself. This removes the need for a fancy lookup to get the owner which is optimal for performance. We can also leverage the existing spin lock in the semaphore to ensure owner is updated correctly. 3) All helper functions which do not need to strictly be implemented as a define to preserve location information can be done as a static inline function. 4) Adding the owner to krwlock_t allows us to remove all memory allocations done during lock initialization. This is good for all the obvious reasons, we do give up the ability to specific the lock name. The Linux profiling tools will stringify the lock name used in the code via the preprocessor and use that. Update rwlocks validated on: - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-25 21:14:35 +00:00
#include <linux/rwsem.h>
Initial commit. All spl source written up to this point wrapped in an initial reasonable autoconf style build system. This does not yet build but the configure system does appear to work properly and integrate with the kernel. Hopefully the next commit gets us back to a buildable version we can run the test suite against. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@1 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-26 20:36:04 +00:00
typedef enum {
Reimplement rwlocks for Linux lock profiling/analysis. It turns out that the previous rwlock implementation worked well but did not integrate properly with the upstream kernel lock profiling/ analysis tools. This is a major problem since it would be awfully nice to be able to use the automatic lock checker and profiler. The problem is that the upstream lock tools use the pre-processor to create a lock class for each uniquely named locked. Since the rwsem was embedded in a wrapper structure the name was always the same. The effect was that we only ended up with one lock class for the entire SPL which caused the lock dependency checker to flag nearly everything as a possible deadlock. The solution was to directly map a krwlock to a Linux rwsem using a typedef there by eliminating the wrapper structure. This was not done initially because the rwsem implementation is specific to the arch. To fully implement the Solaris krwlock API using only the provided rwsem API is not possible. It can only be done by directly accessing some of the internal data member of the rwsem structure. For example, the Linux API provides a different function for dropping a reader vs writer lock. Whereas the Solaris API uses the same function and the caller does not pass in what type of lock it is. This means to properly drop the lock we need to determine if the lock is currently a reader or writer lock. Then we need to call the proper Linux API function. Unfortunately, there is no provided API for this so we must extracted this information directly from arch specific lock implementation. This is all do able, and what I did, but it does complicate things considerably. The good news is that in addition to the profiling benefits of this change. We may see performance improvements due to slightly reduced overhead when creating rwlocks and manipulating them. The only function I was forced to sacrafice was rw_owner() because this information is simply not stored anywhere in the rwsem. Luckily this appears not to be a commonly used function on Solaris, and it is my understanding it is mainly used for debugging anyway. In addition to the core rwlock changes, extensive updates were made to the rwlock regression tests. Each class of test was extended to provide more API coverage and to be more rigerous in checking for misbehavior. This is a pretty significant change and with that in mind I have been careful to validate it on several platforms before committing. The full SPLAT regression test suite was run numberous times on all of the following platforms. This includes various kernels ranging from 2.6.16 to 2.6.29. - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-18 23:09:47 +00:00
RW_DRIVER = 2,
RW_DEFAULT = 4
Initial commit. All spl source written up to this point wrapped in an initial reasonable autoconf style build system. This does not yet build but the configure system does appear to work properly and integrate with the kernel. Hopefully the next commit gets us back to a buildable version we can run the test suite against. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@1 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-26 20:36:04 +00:00
} krw_type_t;
typedef enum {
Update rwlocks to track owner to ensure correct semantics The behavior of RW_*_HELD was updated because it was not quite right. It is not sufficient to return non-zero when the lock is help, we must only do this when the current task in the holder. This means we need to track the lock owner which is not something tracked in a Linux semaphore. After some experimentation the solution I settled on was to embed the Linux semaphore at the start of a larger krwlock_t structure which includes the owner field. This maintains good performance and allows us to cleanly intergrate with the kernel lock analysis tools. My reasons: 1) By placing the Linux semaphore at the start of krwlock_t we can then simply cast krwlock_t to a rw_semaphore and pass that on to the linux kernel. This allows us to use '#defines so the preprocessor can do direct replacement of the Solaris primative with the linux equivilant. This is important because it then maintains the location information for each rw_* call point. 2) Additionally, by adding the owner to krwlock_t we can keep this needed extra information adjacent to the lock itself. This removes the need for a fancy lookup to get the owner which is optimal for performance. We can also leverage the existing spin lock in the semaphore to ensure owner is updated correctly. 3) All helper functions which do not need to strictly be implemented as a define to preserve location information can be done as a static inline function. 4) Adding the owner to krwlock_t allows us to remove all memory allocations done during lock initialization. This is good for all the obvious reasons, we do give up the ability to specific the lock name. The Linux profiling tools will stringify the lock name used in the code via the preprocessor and use that. Update rwlocks validated on: - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-25 21:14:35 +00:00
RW_NONE = 0,
Reimplement rwlocks for Linux lock profiling/analysis. It turns out that the previous rwlock implementation worked well but did not integrate properly with the upstream kernel lock profiling/ analysis tools. This is a major problem since it would be awfully nice to be able to use the automatic lock checker and profiler. The problem is that the upstream lock tools use the pre-processor to create a lock class for each uniquely named locked. Since the rwsem was embedded in a wrapper structure the name was always the same. The effect was that we only ended up with one lock class for the entire SPL which caused the lock dependency checker to flag nearly everything as a possible deadlock. The solution was to directly map a krwlock to a Linux rwsem using a typedef there by eliminating the wrapper structure. This was not done initially because the rwsem implementation is specific to the arch. To fully implement the Solaris krwlock API using only the provided rwsem API is not possible. It can only be done by directly accessing some of the internal data member of the rwsem structure. For example, the Linux API provides a different function for dropping a reader vs writer lock. Whereas the Solaris API uses the same function and the caller does not pass in what type of lock it is. This means to properly drop the lock we need to determine if the lock is currently a reader or writer lock. Then we need to call the proper Linux API function. Unfortunately, there is no provided API for this so we must extracted this information directly from arch specific lock implementation. This is all do able, and what I did, but it does complicate things considerably. The good news is that in addition to the profiling benefits of this change. We may see performance improvements due to slightly reduced overhead when creating rwlocks and manipulating them. The only function I was forced to sacrafice was rw_owner() because this information is simply not stored anywhere in the rwsem. Luckily this appears not to be a commonly used function on Solaris, and it is my understanding it is mainly used for debugging anyway. In addition to the core rwlock changes, extensive updates were made to the rwlock regression tests. Each class of test was extended to provide more API coverage and to be more rigerous in checking for misbehavior. This is a pretty significant change and with that in mind I have been careful to validate it on several platforms before committing. The full SPLAT regression test suite was run numberous times on all of the following platforms. This includes various kernels ranging from 2.6.16 to 2.6.29. - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-18 23:09:47 +00:00
RW_WRITER = 1,
RW_READER = 2
Initial commit. All spl source written up to this point wrapped in an initial reasonable autoconf style build system. This does not yet build but the configure system does appear to work properly and integrate with the kernel. Hopefully the next commit gets us back to a buildable version we can run the test suite against. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@1 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-26 20:36:04 +00:00
} krw_t;
Update rwlocks to track owner to ensure correct semantics The behavior of RW_*_HELD was updated because it was not quite right. It is not sufficient to return non-zero when the lock is help, we must only do this when the current task in the holder. This means we need to track the lock owner which is not something tracked in a Linux semaphore. After some experimentation the solution I settled on was to embed the Linux semaphore at the start of a larger krwlock_t structure which includes the owner field. This maintains good performance and allows us to cleanly intergrate with the kernel lock analysis tools. My reasons: 1) By placing the Linux semaphore at the start of krwlock_t we can then simply cast krwlock_t to a rw_semaphore and pass that on to the linux kernel. This allows us to use '#defines so the preprocessor can do direct replacement of the Solaris primative with the linux equivilant. This is important because it then maintains the location information for each rw_* call point. 2) Additionally, by adding the owner to krwlock_t we can keep this needed extra information adjacent to the lock itself. This removes the need for a fancy lookup to get the owner which is optimal for performance. We can also leverage the existing spin lock in the semaphore to ensure owner is updated correctly. 3) All helper functions which do not need to strictly be implemented as a define to preserve location information can be done as a static inline function. 4) Adding the owner to krwlock_t allows us to remove all memory allocations done during lock initialization. This is good for all the obvious reasons, we do give up the ability to specific the lock name. The Linux profiling tools will stringify the lock name used in the code via the preprocessor and use that. Update rwlocks validated on: - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-25 21:14:35 +00:00
typedef struct {
struct rw_semaphore rw_rwlock;
kthread_t *rw_owner;
} krwlock_t;
Initial commit. All spl source written up to this point wrapped in an initial reasonable autoconf style build system. This does not yet build but the configure system does appear to work properly and integrate with the kernel. Hopefully the next commit gets us back to a buildable version we can run the test suite against. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@1 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-26 20:36:04 +00:00
Reimplement rwlocks for Linux lock profiling/analysis. It turns out that the previous rwlock implementation worked well but did not integrate properly with the upstream kernel lock profiling/ analysis tools. This is a major problem since it would be awfully nice to be able to use the automatic lock checker and profiler. The problem is that the upstream lock tools use the pre-processor to create a lock class for each uniquely named locked. Since the rwsem was embedded in a wrapper structure the name was always the same. The effect was that we only ended up with one lock class for the entire SPL which caused the lock dependency checker to flag nearly everything as a possible deadlock. The solution was to directly map a krwlock to a Linux rwsem using a typedef there by eliminating the wrapper structure. This was not done initially because the rwsem implementation is specific to the arch. To fully implement the Solaris krwlock API using only the provided rwsem API is not possible. It can only be done by directly accessing some of the internal data member of the rwsem structure. For example, the Linux API provides a different function for dropping a reader vs writer lock. Whereas the Solaris API uses the same function and the caller does not pass in what type of lock it is. This means to properly drop the lock we need to determine if the lock is currently a reader or writer lock. Then we need to call the proper Linux API function. Unfortunately, there is no provided API for this so we must extracted this information directly from arch specific lock implementation. This is all do able, and what I did, but it does complicate things considerably. The good news is that in addition to the profiling benefits of this change. We may see performance improvements due to slightly reduced overhead when creating rwlocks and manipulating them. The only function I was forced to sacrafice was rw_owner() because this information is simply not stored anywhere in the rwsem. Luckily this appears not to be a commonly used function on Solaris, and it is my understanding it is mainly used for debugging anyway. In addition to the core rwlock changes, extensive updates were made to the rwlock regression tests. Each class of test was extended to provide more API coverage and to be more rigerous in checking for misbehavior. This is a pretty significant change and with that in mind I have been careful to validate it on several platforms before committing. The full SPLAT regression test suite was run numberous times on all of the following platforms. This includes various kernels ranging from 2.6.16 to 2.6.29. - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-18 23:09:47 +00:00
/*
When using x86 specific rwsem correctly intepret rwsem->count.
2009-12-01 23:47:27 +00:00
* For the generic implementations of rw-semaphores the following is
* true. If your semaphore implementation internally represents the
* semaphore state differently. Then special case handling will be
* required so RW_COUNT() provides these semantics:
Reimplement rwlocks for Linux lock profiling/analysis. It turns out that the previous rwlock implementation worked well but did not integrate properly with the upstream kernel lock profiling/ analysis tools. This is a major problem since it would be awfully nice to be able to use the automatic lock checker and profiler. The problem is that the upstream lock tools use the pre-processor to create a lock class for each uniquely named locked. Since the rwsem was embedded in a wrapper structure the name was always the same. The effect was that we only ended up with one lock class for the entire SPL which caused the lock dependency checker to flag nearly everything as a possible deadlock. The solution was to directly map a krwlock to a Linux rwsem using a typedef there by eliminating the wrapper structure. This was not done initially because the rwsem implementation is specific to the arch. To fully implement the Solaris krwlock API using only the provided rwsem API is not possible. It can only be done by directly accessing some of the internal data member of the rwsem structure. For example, the Linux API provides a different function for dropping a reader vs writer lock. Whereas the Solaris API uses the same function and the caller does not pass in what type of lock it is. This means to properly drop the lock we need to determine if the lock is currently a reader or writer lock. Then we need to call the proper Linux API function. Unfortunately, there is no provided API for this so we must extracted this information directly from arch specific lock implementation. This is all do able, and what I did, but it does complicate things considerably. The good news is that in addition to the profiling benefits of this change. We may see performance improvements due to slightly reduced overhead when creating rwlocks and manipulating them. The only function I was forced to sacrafice was rw_owner() because this information is simply not stored anywhere in the rwsem. Luckily this appears not to be a commonly used function on Solaris, and it is my understanding it is mainly used for debugging anyway. In addition to the core rwlock changes, extensive updates were made to the rwlock regression tests. Each class of test was extended to provide more API coverage and to be more rigerous in checking for misbehavior. This is a pretty significant change and with that in mind I have been careful to validate it on several platforms before committing. The full SPLAT regression test suite was run numberous times on all of the following platforms. This includes various kernels ranging from 2.6.16 to 2.6.29. - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-18 23:09:47 +00:00
* - if activity/count is 0 then there are no active readers or writers
* - if activity/count is +ve then that is the number of active readers
* - if activity/count is -1 then there is one active writer
*/
Update rwlocks to track owner to ensure correct semantics The behavior of RW_*_HELD was updated because it was not quite right. It is not sufficient to return non-zero when the lock is help, we must only do this when the current task in the holder. This means we need to track the lock owner which is not something tracked in a Linux semaphore. After some experimentation the solution I settled on was to embed the Linux semaphore at the start of a larger krwlock_t structure which includes the owner field. This maintains good performance and allows us to cleanly intergrate with the kernel lock analysis tools. My reasons: 1) By placing the Linux semaphore at the start of krwlock_t we can then simply cast krwlock_t to a rw_semaphore and pass that on to the linux kernel. This allows us to use '#defines so the preprocessor can do direct replacement of the Solaris primative with the linux equivilant. This is important because it then maintains the location information for each rw_* call point. 2) Additionally, by adding the owner to krwlock_t we can keep this needed extra information adjacent to the lock itself. This removes the need for a fancy lookup to get the owner which is optimal for performance. We can also leverage the existing spin lock in the semaphore to ensure owner is updated correctly. 3) All helper functions which do not need to strictly be implemented as a define to preserve location information can be done as a static inline function. 4) Adding the owner to krwlock_t allows us to remove all memory allocations done during lock initialization. This is good for all the obvious reasons, we do give up the ability to specific the lock name. The Linux profiling tools will stringify the lock name used in the code via the preprocessor and use that. Update rwlocks validated on: - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-25 21:14:35 +00:00
#define SEM(rwp) ((struct rw_semaphore *)(rwp))
Reimplement rwlocks for Linux lock profiling/analysis. It turns out that the previous rwlock implementation worked well but did not integrate properly with the upstream kernel lock profiling/ analysis tools. This is a major problem since it would be awfully nice to be able to use the automatic lock checker and profiler. The problem is that the upstream lock tools use the pre-processor to create a lock class for each uniquely named locked. Since the rwsem was embedded in a wrapper structure the name was always the same. The effect was that we only ended up with one lock class for the entire SPL which caused the lock dependency checker to flag nearly everything as a possible deadlock. The solution was to directly map a krwlock to a Linux rwsem using a typedef there by eliminating the wrapper structure. This was not done initially because the rwsem implementation is specific to the arch. To fully implement the Solaris krwlock API using only the provided rwsem API is not possible. It can only be done by directly accessing some of the internal data member of the rwsem structure. For example, the Linux API provides a different function for dropping a reader vs writer lock. Whereas the Solaris API uses the same function and the caller does not pass in what type of lock it is. This means to properly drop the lock we need to determine if the lock is currently a reader or writer lock. Then we need to call the proper Linux API function. Unfortunately, there is no provided API for this so we must extracted this information directly from arch specific lock implementation. This is all do able, and what I did, but it does complicate things considerably. The good news is that in addition to the profiling benefits of this change. We may see performance improvements due to slightly reduced overhead when creating rwlocks and manipulating them. The only function I was forced to sacrafice was rw_owner() because this information is simply not stored anywhere in the rwsem. Luckily this appears not to be a commonly used function on Solaris, and it is my understanding it is mainly used for debugging anyway. In addition to the core rwlock changes, extensive updates were made to the rwlock regression tests. Each class of test was extended to provide more API coverage and to be more rigerous in checking for misbehavior. This is a pretty significant change and with that in mind I have been careful to validate it on several platforms before committing. The full SPLAT regression test suite was run numberous times on all of the following platforms. This includes various kernels ranging from 2.6.16 to 2.6.29. - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-18 23:09:47 +00:00
#if defined(CONFIG_RWSEM_GENERIC_SPINLOCK)
Update rwlocks to track owner to ensure correct semantics The behavior of RW_*_HELD was updated because it was not quite right. It is not sufficient to return non-zero when the lock is help, we must only do this when the current task in the holder. This means we need to track the lock owner which is not something tracked in a Linux semaphore. After some experimentation the solution I settled on was to embed the Linux semaphore at the start of a larger krwlock_t structure which includes the owner field. This maintains good performance and allows us to cleanly intergrate with the kernel lock analysis tools. My reasons: 1) By placing the Linux semaphore at the start of krwlock_t we can then simply cast krwlock_t to a rw_semaphore and pass that on to the linux kernel. This allows us to use '#defines so the preprocessor can do direct replacement of the Solaris primative with the linux equivilant. This is important because it then maintains the location information for each rw_* call point. 2) Additionally, by adding the owner to krwlock_t we can keep this needed extra information adjacent to the lock itself. This removes the need for a fancy lookup to get the owner which is optimal for performance. We can also leverage the existing spin lock in the semaphore to ensure owner is updated correctly. 3) All helper functions which do not need to strictly be implemented as a define to preserve location information can be done as a static inline function. 4) Adding the owner to krwlock_t allows us to remove all memory allocations done during lock initialization. This is good for all the obvious reasons, we do give up the ability to specific the lock name. The Linux profiling tools will stringify the lock name used in the code via the preprocessor and use that. Update rwlocks validated on: - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-25 21:14:35 +00:00
# define RW_COUNT(rwp) (SEM(rwp)->activity)
# define rw_exit_locked(rwp) __up_read_locked(rwp)
# define rw_tryenter_locked(rwp) __down_write_trylock_locked(rwp)
extern void __up_read_locked(struct rw_semaphore *);
extern int __down_write_trylock_locked(struct rw_semaphore *);
Reimplement rwlocks for Linux lock profiling/analysis. It turns out that the previous rwlock implementation worked well but did not integrate properly with the upstream kernel lock profiling/ analysis tools. This is a major problem since it would be awfully nice to be able to use the automatic lock checker and profiler. The problem is that the upstream lock tools use the pre-processor to create a lock class for each uniquely named locked. Since the rwsem was embedded in a wrapper structure the name was always the same. The effect was that we only ended up with one lock class for the entire SPL which caused the lock dependency checker to flag nearly everything as a possible deadlock. The solution was to directly map a krwlock to a Linux rwsem using a typedef there by eliminating the wrapper structure. This was not done initially because the rwsem implementation is specific to the arch. To fully implement the Solaris krwlock API using only the provided rwsem API is not possible. It can only be done by directly accessing some of the internal data member of the rwsem structure. For example, the Linux API provides a different function for dropping a reader vs writer lock. Whereas the Solaris API uses the same function and the caller does not pass in what type of lock it is. This means to properly drop the lock we need to determine if the lock is currently a reader or writer lock. Then we need to call the proper Linux API function. Unfortunately, there is no provided API for this so we must extracted this information directly from arch specific lock implementation. This is all do able, and what I did, but it does complicate things considerably. The good news is that in addition to the profiling benefits of this change. We may see performance improvements due to slightly reduced overhead when creating rwlocks and manipulating them. The only function I was forced to sacrafice was rw_owner() because this information is simply not stored anywhere in the rwsem. Luckily this appears not to be a commonly used function on Solaris, and it is my understanding it is mainly used for debugging anyway. In addition to the core rwlock changes, extensive updates were made to the rwlock regression tests. Each class of test was extended to provide more API coverage and to be more rigerous in checking for misbehavior. This is a pretty significant change and with that in mind I have been careful to validate it on several platforms before committing. The full SPLAT regression test suite was run numberous times on all of the following platforms. This includes various kernels ranging from 2.6.16 to 2.6.29. - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-18 23:09:47 +00:00
#else
Check for changed gaurd macro in 2.6.28+ for rwsem implementation. As part of the 2.6.28 cleanup which moved all the linux/include/asm/ headers in to linux/arch, the guard headers for many header files changed. The i386 rwsem implementation keys off this header to ensure the internal members of the rwsem structure are interpreted correctly. This change checks for the new guard macro in addition to the only one, the implementation of the rwsem has not changed for i386 so this is safe and correct.
2009-12-17 19:57:44 +00:00
/*
* 2.6.x - 2.6.27 use guard macro _I386_RWSEM_H
* 2.6.28 - 2.6.32+ use guard macro _ASM_X86_RWSEM_H
*/
# if defined(_I386_RWSEM_H) || defined(_ASM_X86_RWSEM_H)
When using x86 specific rwsem correctly intepret rwsem->count.
2009-12-01 23:47:27 +00:00
# define RW_COUNT(rwp) ((SEM(rwp)->count < 0) ? (-1) : \
Disable rw_tryupgrade() for newer kernels For kernels using the CONFIG_RWSEM_GENERIC_SPINLOCK implementation nothing has changed. But if your kernel is building with arch specific rwsems rw_tryupgrade() has been disabled until it can be implemented correctly. In particular, the x86 implementation now leverages atomic primatives for serialization rather than spinlocks. So to get this working again it will need to be implemented as a cmpxchg for x86 and likely something similiar for other arches we are interested in. For now it's safest to simply disable it.
2010-04-20 22:16:27 +00:00
(SEM(rwp)->count & RWSEM_ACTIVE_MASK))
When using x86 specific rwsem correctly intepret rwsem->count.
2009-12-01 23:47:27 +00:00
# else
# define RW_COUNT(rwp) (SEM(rwp)->count & RWSEM_ACTIVE_MASK)
# endif
# define rw_exit_locked(rwp) up_read(rwp)
# define rw_tryenter_locked(rwp) down_write_trylock(rwp)
Reimplement rwlocks for Linux lock profiling/analysis. It turns out that the previous rwlock implementation worked well but did not integrate properly with the upstream kernel lock profiling/ analysis tools. This is a major problem since it would be awfully nice to be able to use the automatic lock checker and profiler. The problem is that the upstream lock tools use the pre-processor to create a lock class for each uniquely named locked. Since the rwsem was embedded in a wrapper structure the name was always the same. The effect was that we only ended up with one lock class for the entire SPL which caused the lock dependency checker to flag nearly everything as a possible deadlock. The solution was to directly map a krwlock to a Linux rwsem using a typedef there by eliminating the wrapper structure. This was not done initially because the rwsem implementation is specific to the arch. To fully implement the Solaris krwlock API using only the provided rwsem API is not possible. It can only be done by directly accessing some of the internal data member of the rwsem structure. For example, the Linux API provides a different function for dropping a reader vs writer lock. Whereas the Solaris API uses the same function and the caller does not pass in what type of lock it is. This means to properly drop the lock we need to determine if the lock is currently a reader or writer lock. Then we need to call the proper Linux API function. Unfortunately, there is no provided API for this so we must extracted this information directly from arch specific lock implementation. This is all do able, and what I did, but it does complicate things considerably. The good news is that in addition to the profiling benefits of this change. We may see performance improvements due to slightly reduced overhead when creating rwlocks and manipulating them. The only function I was forced to sacrafice was rw_owner() because this information is simply not stored anywhere in the rwsem. Luckily this appears not to be a commonly used function on Solaris, and it is my understanding it is mainly used for debugging anyway. In addition to the core rwlock changes, extensive updates were made to the rwlock regression tests. Each class of test was extended to provide more API coverage and to be more rigerous in checking for misbehavior. This is a pretty significant change and with that in mind I have been careful to validate it on several platforms before committing. The full SPLAT regression test suite was run numberous times on all of the following platforms. This includes various kernels ranging from 2.6.16 to 2.6.29. - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-18 23:09:47 +00:00
#endif
Initial commit. All spl source written up to this point wrapped in an initial reasonable autoconf style build system. This does not yet build but the configure system does appear to work properly and integrate with the kernel. Hopefully the next commit gets us back to a buildable version we can run the test suite against. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@1 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-26 20:36:04 +00:00
Update rwlocks to track owner to ensure correct semantics The behavior of RW_*_HELD was updated because it was not quite right. It is not sufficient to return non-zero when the lock is help, we must only do this when the current task in the holder. This means we need to track the lock owner which is not something tracked in a Linux semaphore. After some experimentation the solution I settled on was to embed the Linux semaphore at the start of a larger krwlock_t structure which includes the owner field. This maintains good performance and allows us to cleanly intergrate with the kernel lock analysis tools. My reasons: 1) By placing the Linux semaphore at the start of krwlock_t we can then simply cast krwlock_t to a rw_semaphore and pass that on to the linux kernel. This allows us to use '#defines so the preprocessor can do direct replacement of the Solaris primative with the linux equivilant. This is important because it then maintains the location information for each rw_* call point. 2) Additionally, by adding the owner to krwlock_t we can keep this needed extra information adjacent to the lock itself. This removes the need for a fancy lookup to get the owner which is optimal for performance. We can also leverage the existing spin lock in the semaphore to ensure owner is updated correctly. 3) All helper functions which do not need to strictly be implemented as a define to preserve location information can be done as a static inline function. 4) Adding the owner to krwlock_t allows us to remove all memory allocations done during lock initialization. This is good for all the obvious reasons, we do give up the ability to specific the lock name. The Linux profiling tools will stringify the lock name used in the code via the preprocessor and use that. Update rwlocks validated on: - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-25 21:14:35 +00:00
static inline kthread_t *
spl_rw_get_owner(krwlock_t *rwp)
{
return rwp->rw_owner;
}
static inline void
spl_rw_set_owner(krwlock_t *rwp)
{
unsigned long flags;
spin_lock_irqsave(&SEM(rwp)->wait_lock, flags);
rwp->rw_owner = current;
spin_unlock_irqrestore(&SEM(rwp)->wait_lock, flags);
}
static inline void
spl_rw_clear_owner(krwlock_t *rwp)
{
unsigned long flags;
spin_lock_irqsave(&SEM(rwp)->wait_lock, flags);
rwp->rw_owner = NULL;
spin_unlock_irqrestore(&SEM(rwp)->wait_lock, flags);
}
static inline kthread_t *
rw_owner(krwlock_t *rwp)
{
unsigned long flags;
kthread_t *owner;
spin_lock_irqsave(&SEM(rwp)->wait_lock, flags);
owner = spl_rw_get_owner(rwp);
spin_unlock_irqrestore(&SEM(rwp)->wait_lock, flags);
return owner;
}
static inline int
RW_READ_HELD(krwlock_t *rwp)
{
unsigned long flags;
int rc;
spin_lock_irqsave(&SEM(rwp)->wait_lock, flags);
rc = ((RW_COUNT(rwp) > 0) && (spl_rw_get_owner(rwp) == NULL));
spin_unlock_irqrestore(&SEM(rwp)->wait_lock, flags);
return rc;
}
static inline int
RW_WRITE_HELD(krwlock_t *rwp)
{
unsigned long flags;
int rc;
spin_lock_irqsave(&SEM(rwp)->wait_lock, flags);
rc = ((RW_COUNT(rwp) < 0) && (spl_rw_get_owner(rwp) == current));
spin_unlock_irqrestore(&SEM(rwp)->wait_lock, flags);
return rc;
}
static inline int
RW_LOCK_HELD(krwlock_t *rwp)
{
unsigned long flags;
int rc;
spin_lock_irqsave(&SEM(rwp)->wait_lock, flags);
rc = (RW_COUNT(rwp) != 0);
spin_unlock_irqrestore(&SEM(rwp)->wait_lock, flags);
return rc;
}
/*
* The following functions must be a #define and not static inline.
* This ensures that the native linux semaphore functions (down/up)
* will be correctly located in the users code which is important
* for the built in kernel lock analysis tools
*/
#define rw_init(rwp, name, type, arg) \
({ \
Expand SEM() outside init_rwsem and directly call __init_rwsem(). We need to directly call __init_rwsem() or the name gets expanded to SEM(lock-name). This is safe and correct for the support arches x86/x86_64/ppc/ppc64.
2009-09-29 10:19:09 +00:00
static struct lock_class_key __key; \
\
__init_rwsem(SEM(rwp), #rwp, &__key); \
Update rwlocks to track owner to ensure correct semantics The behavior of RW_*_HELD was updated because it was not quite right. It is not sufficient to return non-zero when the lock is help, we must only do this when the current task in the holder. This means we need to track the lock owner which is not something tracked in a Linux semaphore. After some experimentation the solution I settled on was to embed the Linux semaphore at the start of a larger krwlock_t structure which includes the owner field. This maintains good performance and allows us to cleanly intergrate with the kernel lock analysis tools. My reasons: 1) By placing the Linux semaphore at the start of krwlock_t we can then simply cast krwlock_t to a rw_semaphore and pass that on to the linux kernel. This allows us to use '#defines so the preprocessor can do direct replacement of the Solaris primative with the linux equivilant. This is important because it then maintains the location information for each rw_* call point. 2) Additionally, by adding the owner to krwlock_t we can keep this needed extra information adjacent to the lock itself. This removes the need for a fancy lookup to get the owner which is optimal for performance. We can also leverage the existing spin lock in the semaphore to ensure owner is updated correctly. 3) All helper functions which do not need to strictly be implemented as a define to preserve location information can be done as a static inline function. 4) Adding the owner to krwlock_t allows us to remove all memory allocations done during lock initialization. This is good for all the obvious reasons, we do give up the ability to specific the lock name. The Linux profiling tools will stringify the lock name used in the code via the preprocessor and use that. Update rwlocks validated on: - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-25 21:14:35 +00:00
spl_rw_clear_owner(rwp); \
Reimplement rwlocks for Linux lock profiling/analysis. It turns out that the previous rwlock implementation worked well but did not integrate properly with the upstream kernel lock profiling/ analysis tools. This is a major problem since it would be awfully nice to be able to use the automatic lock checker and profiler. The problem is that the upstream lock tools use the pre-processor to create a lock class for each uniquely named locked. Since the rwsem was embedded in a wrapper structure the name was always the same. The effect was that we only ended up with one lock class for the entire SPL which caused the lock dependency checker to flag nearly everything as a possible deadlock. The solution was to directly map a krwlock to a Linux rwsem using a typedef there by eliminating the wrapper structure. This was not done initially because the rwsem implementation is specific to the arch. To fully implement the Solaris krwlock API using only the provided rwsem API is not possible. It can only be done by directly accessing some of the internal data member of the rwsem structure. For example, the Linux API provides a different function for dropping a reader vs writer lock. Whereas the Solaris API uses the same function and the caller does not pass in what type of lock it is. This means to properly drop the lock we need to determine if the lock is currently a reader or writer lock. Then we need to call the proper Linux API function. Unfortunately, there is no provided API for this so we must extracted this information directly from arch specific lock implementation. This is all do able, and what I did, but it does complicate things considerably. The good news is that in addition to the profiling benefits of this change. We may see performance improvements due to slightly reduced overhead when creating rwlocks and manipulating them. The only function I was forced to sacrafice was rw_owner() because this information is simply not stored anywhere in the rwsem. Luckily this appears not to be a commonly used function on Solaris, and it is my understanding it is mainly used for debugging anyway. In addition to the core rwlock changes, extensive updates were made to the rwlock regression tests. Each class of test was extended to provide more API coverage and to be more rigerous in checking for misbehavior. This is a pretty significant change and with that in mind I have been careful to validate it on several platforms before committing. The full SPLAT regression test suite was run numberous times on all of the following platforms. This includes various kernels ranging from 2.6.16 to 2.6.29. - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-18 23:09:47 +00:00
})
Initial commit. All spl source written up to this point wrapped in an initial reasonable autoconf style build system. This does not yet build but the configure system does appear to work properly and integrate with the kernel. Hopefully the next commit gets us back to a buildable version we can run the test suite against. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@1 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-26 20:36:04 +00:00
Update rwlocks to track owner to ensure correct semantics The behavior of RW_*_HELD was updated because it was not quite right. It is not sufficient to return non-zero when the lock is help, we must only do this when the current task in the holder. This means we need to track the lock owner which is not something tracked in a Linux semaphore. After some experimentation the solution I settled on was to embed the Linux semaphore at the start of a larger krwlock_t structure which includes the owner field. This maintains good performance and allows us to cleanly intergrate with the kernel lock analysis tools. My reasons: 1) By placing the Linux semaphore at the start of krwlock_t we can then simply cast krwlock_t to a rw_semaphore and pass that on to the linux kernel. This allows us to use '#defines so the preprocessor can do direct replacement of the Solaris primative with the linux equivilant. This is important because it then maintains the location information for each rw_* call point. 2) Additionally, by adding the owner to krwlock_t we can keep this needed extra information adjacent to the lock itself. This removes the need for a fancy lookup to get the owner which is optimal for performance. We can also leverage the existing spin lock in the semaphore to ensure owner is updated correctly. 3) All helper functions which do not need to strictly be implemented as a define to preserve location information can be done as a static inline function. 4) Adding the owner to krwlock_t allows us to remove all memory allocations done during lock initialization. This is good for all the obvious reasons, we do give up the ability to specific the lock name. The Linux profiling tools will stringify the lock name used in the code via the preprocessor and use that. Update rwlocks validated on: - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-25 21:14:35 +00:00
#define rw_destroy(rwp) \
({ \
VERIFY(!RW_LOCK_HELD(rwp)); \
Reimplement rwlocks for Linux lock profiling/analysis. It turns out that the previous rwlock implementation worked well but did not integrate properly with the upstream kernel lock profiling/ analysis tools. This is a major problem since it would be awfully nice to be able to use the automatic lock checker and profiler. The problem is that the upstream lock tools use the pre-processor to create a lock class for each uniquely named locked. Since the rwsem was embedded in a wrapper structure the name was always the same. The effect was that we only ended up with one lock class for the entire SPL which caused the lock dependency checker to flag nearly everything as a possible deadlock. The solution was to directly map a krwlock to a Linux rwsem using a typedef there by eliminating the wrapper structure. This was not done initially because the rwsem implementation is specific to the arch. To fully implement the Solaris krwlock API using only the provided rwsem API is not possible. It can only be done by directly accessing some of the internal data member of the rwsem structure. For example, the Linux API provides a different function for dropping a reader vs writer lock. Whereas the Solaris API uses the same function and the caller does not pass in what type of lock it is. This means to properly drop the lock we need to determine if the lock is currently a reader or writer lock. Then we need to call the proper Linux API function. Unfortunately, there is no provided API for this so we must extracted this information directly from arch specific lock implementation. This is all do able, and what I did, but it does complicate things considerably. The good news is that in addition to the profiling benefits of this change. We may see performance improvements due to slightly reduced overhead when creating rwlocks and manipulating them. The only function I was forced to sacrafice was rw_owner() because this information is simply not stored anywhere in the rwsem. Luckily this appears not to be a commonly used function on Solaris, and it is my understanding it is mainly used for debugging anyway. In addition to the core rwlock changes, extensive updates were made to the rwlock regression tests. Each class of test was extended to provide more API coverage and to be more rigerous in checking for misbehavior. This is a pretty significant change and with that in mind I have been careful to validate it on several platforms before committing. The full SPLAT regression test suite was run numberous times on all of the following platforms. This includes various kernels ranging from 2.6.16 to 2.6.29. - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-18 23:09:47 +00:00
})
Add some missing rw_lock symbols git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@27 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-06 23:42:37 +00:00
Update rwlocks to track owner to ensure correct semantics The behavior of RW_*_HELD was updated because it was not quite right. It is not sufficient to return non-zero when the lock is help, we must only do this when the current task in the holder. This means we need to track the lock owner which is not something tracked in a Linux semaphore. After some experimentation the solution I settled on was to embed the Linux semaphore at the start of a larger krwlock_t structure which includes the owner field. This maintains good performance and allows us to cleanly intergrate with the kernel lock analysis tools. My reasons: 1) By placing the Linux semaphore at the start of krwlock_t we can then simply cast krwlock_t to a rw_semaphore and pass that on to the linux kernel. This allows us to use '#defines so the preprocessor can do direct replacement of the Solaris primative with the linux equivilant. This is important because it then maintains the location information for each rw_* call point. 2) Additionally, by adding the owner to krwlock_t we can keep this needed extra information adjacent to the lock itself. This removes the need for a fancy lookup to get the owner which is optimal for performance. We can also leverage the existing spin lock in the semaphore to ensure owner is updated correctly. 3) All helper functions which do not need to strictly be implemented as a define to preserve location information can be done as a static inline function. 4) Adding the owner to krwlock_t allows us to remove all memory allocations done during lock initialization. This is good for all the obvious reasons, we do give up the ability to specific the lock name. The Linux profiling tools will stringify the lock name used in the code via the preprocessor and use that. Update rwlocks validated on: - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-25 21:14:35 +00:00
#define rw_tryenter(rwp, rw) \
({ \
int _rc_ = 0; \
\
switch (rw) { \
case RW_READER: \
_rc_ = down_read_trylock(SEM(rwp)); \
break; \
case RW_WRITER: \
if ((_rc_ = down_write_trylock(SEM(rwp)))) \
spl_rw_set_owner(rwp); \
break; \
default: \
SBUG(); \
} \
_rc_; \
- Updated rwlock's to reside in a .c file instead of a static inline - Updated rwlock's so they can be safely initialized in ctors. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@96 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-06 23:00:49 +00:00
})
Update rwlocks to track owner to ensure correct semantics The behavior of RW_*_HELD was updated because it was not quite right. It is not sufficient to return non-zero when the lock is help, we must only do this when the current task in the holder. This means we need to track the lock owner which is not something tracked in a Linux semaphore. After some experimentation the solution I settled on was to embed the Linux semaphore at the start of a larger krwlock_t structure which includes the owner field. This maintains good performance and allows us to cleanly intergrate with the kernel lock analysis tools. My reasons: 1) By placing the Linux semaphore at the start of krwlock_t we can then simply cast krwlock_t to a rw_semaphore and pass that on to the linux kernel. This allows us to use '#defines so the preprocessor can do direct replacement of the Solaris primative with the linux equivilant. This is important because it then maintains the location information for each rw_* call point. 2) Additionally, by adding the owner to krwlock_t we can keep this needed extra information adjacent to the lock itself. This removes the need for a fancy lookup to get the owner which is optimal for performance. We can also leverage the existing spin lock in the semaphore to ensure owner is updated correctly. 3) All helper functions which do not need to strictly be implemented as a define to preserve location information can be done as a static inline function. 4) Adding the owner to krwlock_t allows us to remove all memory allocations done during lock initialization. This is good for all the obvious reasons, we do give up the ability to specific the lock name. The Linux profiling tools will stringify the lock name used in the code via the preprocessor and use that. Update rwlocks validated on: - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-25 21:14:35 +00:00
#define rw_enter(rwp, rw) \
({ \
switch (rw) { \
case RW_READER: \
down_read(SEM(rwp)); \
break; \
case RW_WRITER: \
down_write(SEM(rwp)); \
spl_rw_set_owner(rwp); \
break; \
default: \
SBUG(); \
} \
Reimplement rwlocks for Linux lock profiling/analysis. It turns out that the previous rwlock implementation worked well but did not integrate properly with the upstream kernel lock profiling/ analysis tools. This is a major problem since it would be awfully nice to be able to use the automatic lock checker and profiler. The problem is that the upstream lock tools use the pre-processor to create a lock class for each uniquely named locked. Since the rwsem was embedded in a wrapper structure the name was always the same. The effect was that we only ended up with one lock class for the entire SPL which caused the lock dependency checker to flag nearly everything as a possible deadlock. The solution was to directly map a krwlock to a Linux rwsem using a typedef there by eliminating the wrapper structure. This was not done initially because the rwsem implementation is specific to the arch. To fully implement the Solaris krwlock API using only the provided rwsem API is not possible. It can only be done by directly accessing some of the internal data member of the rwsem structure. For example, the Linux API provides a different function for dropping a reader vs writer lock. Whereas the Solaris API uses the same function and the caller does not pass in what type of lock it is. This means to properly drop the lock we need to determine if the lock is currently a reader or writer lock. Then we need to call the proper Linux API function. Unfortunately, there is no provided API for this so we must extracted this information directly from arch specific lock implementation. This is all do able, and what I did, but it does complicate things considerably. The good news is that in addition to the profiling benefits of this change. We may see performance improvements due to slightly reduced overhead when creating rwlocks and manipulating them. The only function I was forced to sacrafice was rw_owner() because this information is simply not stored anywhere in the rwsem. Luckily this appears not to be a commonly used function on Solaris, and it is my understanding it is mainly used for debugging anyway. In addition to the core rwlock changes, extensive updates were made to the rwlock regression tests. Each class of test was extended to provide more API coverage and to be more rigerous in checking for misbehavior. This is a pretty significant change and with that in mind I have been careful to validate it on several platforms before committing. The full SPLAT regression test suite was run numberous times on all of the following platforms. This includes various kernels ranging from 2.6.16 to 2.6.29. - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-18 23:09:47 +00:00
})
Initial commit. All spl source written up to this point wrapped in an initial reasonable autoconf style build system. This does not yet build but the configure system does appear to work properly and integrate with the kernel. Hopefully the next commit gets us back to a buildable version we can run the test suite against. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@1 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-26 20:36:04 +00:00
Update rwlocks to track owner to ensure correct semantics The behavior of RW_*_HELD was updated because it was not quite right. It is not sufficient to return non-zero when the lock is help, we must only do this when the current task in the holder. This means we need to track the lock owner which is not something tracked in a Linux semaphore. After some experimentation the solution I settled on was to embed the Linux semaphore at the start of a larger krwlock_t structure which includes the owner field. This maintains good performance and allows us to cleanly intergrate with the kernel lock analysis tools. My reasons: 1) By placing the Linux semaphore at the start of krwlock_t we can then simply cast krwlock_t to a rw_semaphore and pass that on to the linux kernel. This allows us to use '#defines so the preprocessor can do direct replacement of the Solaris primative with the linux equivilant. This is important because it then maintains the location information for each rw_* call point. 2) Additionally, by adding the owner to krwlock_t we can keep this needed extra information adjacent to the lock itself. This removes the need for a fancy lookup to get the owner which is optimal for performance. We can also leverage the existing spin lock in the semaphore to ensure owner is updated correctly. 3) All helper functions which do not need to strictly be implemented as a define to preserve location information can be done as a static inline function. 4) Adding the owner to krwlock_t allows us to remove all memory allocations done during lock initialization. This is good for all the obvious reasons, we do give up the ability to specific the lock name. The Linux profiling tools will stringify the lock name used in the code via the preprocessor and use that. Update rwlocks validated on: - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-25 21:14:35 +00:00
#define rw_exit(rwp) \
({ \
if (RW_WRITE_HELD(rwp)) { \
spl_rw_clear_owner(rwp); \
up_write(SEM(rwp)); \
} else { \
ASSERT(RW_READ_HELD(rwp)); \
up_read(SEM(rwp)); \
} \
})
#define rw_downgrade(rwp) \
({ \
spl_rw_clear_owner(rwp); \
downgrade_write(SEM(rwp)); \
})
Disable rw_tryupgrade() for newer kernels For kernels using the CONFIG_RWSEM_GENERIC_SPINLOCK implementation nothing has changed. But if your kernel is building with arch specific rwsems rw_tryupgrade() has been disabled until it can be implemented correctly. In particular, the x86 implementation now leverages atomic primatives for serialization rather than spinlocks. So to get this working again it will need to be implemented as a cmpxchg for x86 and likely something similiar for other arches we are interested in. For now it's safest to simply disable it.
2010-04-20 22:16:27 +00:00
#if defined(CONFIG_RWSEM_GENERIC_SPINLOCK)
Update rwlocks to track owner to ensure correct semantics The behavior of RW_*_HELD was updated because it was not quite right. It is not sufficient to return non-zero when the lock is help, we must only do this when the current task in the holder. This means we need to track the lock owner which is not something tracked in a Linux semaphore. After some experimentation the solution I settled on was to embed the Linux semaphore at the start of a larger krwlock_t structure which includes the owner field. This maintains good performance and allows us to cleanly intergrate with the kernel lock analysis tools. My reasons: 1) By placing the Linux semaphore at the start of krwlock_t we can then simply cast krwlock_t to a rw_semaphore and pass that on to the linux kernel. This allows us to use '#defines so the preprocessor can do direct replacement of the Solaris primative with the linux equivilant. This is important because it then maintains the location information for each rw_* call point. 2) Additionally, by adding the owner to krwlock_t we can keep this needed extra information adjacent to the lock itself. This removes the need for a fancy lookup to get the owner which is optimal for performance. We can also leverage the existing spin lock in the semaphore to ensure owner is updated correctly. 3) All helper functions which do not need to strictly be implemented as a define to preserve location information can be done as a static inline function. 4) Adding the owner to krwlock_t allows us to remove all memory allocations done during lock initialization. This is good for all the obvious reasons, we do give up the ability to specific the lock name. The Linux profiling tools will stringify the lock name used in the code via the preprocessor and use that. Update rwlocks validated on: - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-25 21:14:35 +00:00
#define rw_tryupgrade(rwp) \
({ \
unsigned long _flags_; \
int _rc_ = 0; \
\
spin_lock_irqsave(&SEM(rwp)->wait_lock, _flags_); \
if (list_empty(&SEM(rwp)->wait_list) && (RW_COUNT(rwp) == 1)) { \
rw_exit_locked(SEM(rwp)); \
VERIFY(_rc_ = rw_tryenter_locked(SEM(rwp))); \
(rwp)->rw_owner = current; \
} \
spin_unlock_irqrestore(&SEM(rwp)->wait_lock, _flags_); \
_rc_; \
})
Disable rw_tryupgrade() for newer kernels For kernels using the CONFIG_RWSEM_GENERIC_SPINLOCK implementation nothing has changed. But if your kernel is building with arch specific rwsems rw_tryupgrade() has been disabled until it can be implemented correctly. In particular, the x86 implementation now leverages atomic primatives for serialization rather than spinlocks. So to get this working again it will need to be implemented as a cmpxchg for x86 and likely something similiar for other arches we are interested in. For now it's safest to simply disable it.
2010-04-20 22:16:27 +00:00
#else
/*
* This can be done correctly but for each supported arch we will need
* a custom cmpxchg() to atomically check and promote the rwsem. That's
* not worth the trouble for now so rw_tryupgrade() will always fail.
*/
#define rw_tryupgrade(rwp) ({ 0; })
#endif
Update rwlocks to track owner to ensure correct semantics The behavior of RW_*_HELD was updated because it was not quite right. It is not sufficient to return non-zero when the lock is help, we must only do this when the current task in the holder. This means we need to track the lock owner which is not something tracked in a Linux semaphore. After some experimentation the solution I settled on was to embed the Linux semaphore at the start of a larger krwlock_t structure which includes the owner field. This maintains good performance and allows us to cleanly intergrate with the kernel lock analysis tools. My reasons: 1) By placing the Linux semaphore at the start of krwlock_t we can then simply cast krwlock_t to a rw_semaphore and pass that on to the linux kernel. This allows us to use '#defines so the preprocessor can do direct replacement of the Solaris primative with the linux equivilant. This is important because it then maintains the location information for each rw_* call point. 2) Additionally, by adding the owner to krwlock_t we can keep this needed extra information adjacent to the lock itself. This removes the need for a fancy lookup to get the owner which is optimal for performance. We can also leverage the existing spin lock in the semaphore to ensure owner is updated correctly. 3) All helper functions which do not need to strictly be implemented as a define to preserve location information can be done as a static inline function. 4) Adding the owner to krwlock_t allows us to remove all memory allocations done during lock initialization. This is good for all the obvious reasons, we do give up the ability to specific the lock name. The Linux profiling tools will stringify the lock name used in the code via the preprocessor and use that. Update rwlocks validated on: - SLES10 (ppc64) - SLES11 (x86_64) - CHAOS4.2 (x86_64) - RHEL5.3 (x86_64) - RHEL6 (x86_64) - FC11 (x86_64)
2009-09-25 21:14:35 +00:00
int spl_rw_init(void);
void spl_rw_fini(void);
- Updated rwlock's to reside in a .c file instead of a static inline - Updated rwlock's so they can be safely initialized in ctors. git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@96 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-06 23:00:49 +00:00
#endif /* _SPL_RWLOCK_H */