Commit Graph

6623 Commits

Author SHA1 Message Date
Brian Behlendorf c89fdee4d3 Remove __GFP_NOFAIL in kmem and retry internally.
As of 2.6.31 it's clear __GFP_NOFAIL should no longer be used and it
may disappear from the kernel at any time.  To handle this I have simply
added *_nofail wrappers in the kmem implementation which perform the
retry for non-atomic allocations.

From linux-2.6.31 mm/page_alloc.c:1166
/*
 * __GFP_NOFAIL is not to be used in new code.
 *
 * All __GFP_NOFAIL callers should be fixed so that they
 * properly detect and handle allocation failures.
 *
 * We most definitely don't want callers attempting to
 * allocate greater than order-1 page units with
 * __GFP_NOFAIL.
 */
WARN_ON_ONCE(order > 1);
2009-11-12 15:11:24 -08:00
Brian Behlendorf baf2979ed3 Linux 2.6.31 Compatibility Updates
SPL_AC_2ARGS_SET_FS_PWD macro updated to explicitly include
linux/fs_struct.h which was dropped from linux/sched.h.

min_wmark_pages, low_wmark_pages, high_wmark_pages macros
introduced in newer kernels.  For older kernels mm_compat.h
was introduced to define them as needed as direct mappings
to per zone min_pages, low_pages, max_pages.
2009-11-10 14:06:57 -08:00
Brian Behlendorf f97cd5fd87 Prep for 0.4.6 tag, updated META and ChangeLog. 2009-11-02 10:24:12 -08:00
Brian Behlendorf 055ffd98cf Autoconf --enable-debug-* cleanup
Cleanup the --enable-debug-* configure options, this has been pending
for quite some time and I am glad I finally got to it.  To summerize:

1) All SPL_AC_DEBUG_* macros were updated to be a more autoconf
friendly.  This mainly involved shift to the GNU approved usage of
AC_ARG_ENABLE and ensuring AS_IF is used rather than directly using
an if [ test ] construct.

2) --enable-debug-kmem=yes by default.  This simply enabled keeping
a running tally of total memory allocated and freed and reporting a
memory leak if there was one at module unload.  Additionally, it
ensure /proc/spl/kmem/slab will exist by default which is handy.
The overhead is low for this and it should not impact performance.

3) --enable-debug-kmem-tracking=no by default.  This option was added
to provide a configure option to enable to detailed memory allocation
tracking.  This support was always there but you had to know where to
turn it on.  By default this support is disabled because it is known
to badly hurt performence, however it is invaluable when chasing a
memory leak.

4) --enable-debug-kstat removed.  After further reflection I can't see
why you would ever really want to turn this support off.  It is now
always on which had the nice side effect of simplifying the proc handling
code in spl-proc.c.  We can now always assume the top level directory
will be there.

5) --enable-debug-callb removed.  This never really did anything, it was
put in provisionally because it might have been needed.  It turns out
it was not so I am just removing it to prevent confusion.
2009-10-30 13:58:51 -07:00
Brian Behlendorf 302b88e6ab Add autoconf checks for atomic64_cmpxchg + atomic64_xchg
These functions didn't exist for all archs prior to 2.6.24.  This
patch addes an autoconf test to detect this and add them when needed.
The autoconf check is needed instead of just an #ifndef because in
the most modern kernels atomic64_{cmp}xchg are implemented as in
inline function and not a #define.
2009-10-30 13:53:17 -07:00
Brian Behlendorf 5e9b5d832b Use Linux atomic primitives by default.
Previously Solaris style atomic primitives were implemented simply by
wrapping the desired operation in a global spinlock.  This was easy to
implement at the time when I wasn't 100% sure I could safely layer the
Solaris atomic primatives on the Linux counterparts.  It however was
likely not good for performance.

After more investigation however it does appear the Solaris primitives
can be layered on Linux's fairly safely.  The Linux atomic_t type really
just wraps a long so we can simply cast the Solaris unsigned value to
either a atomic_t or atomic64_t.  The only lingering problem for both
implementations is that Solaris provides no atomic read function.  This
means reading a 64-bit value on a 32-bit arch can (and will) result in
word breaking.  I was very concerned about this initially, but upon
further reflection it is a limitation of the Solaris API.  So really
we are just being bug-for-bug compatible here.

With this change the default implementation is layered on top of Linux
atomic types.  However, because we're assuming a lot about the internal
implementation of those types I've made it easy to fall-back to the
generic approach.  Simply build with --enable-atomic_spinlocks if
issues are encountered with the new implementation.
2009-10-30 10:55:25 -07:00
Brian Behlendorf 2b5adaf18f I should not have removed these, they are important. 2009-10-27 16:17:06 -07:00
Brian Behlendorf 4bd577d069 Rebase cmn_err on vcmn_err and don't warn about missing \n
The cmn_err/vcmn_err functions are layered on top of the debug
system which usually expects a newline at the end.  However, there
really doesn't need to be a newline there and there in fact should
not be for the CE_CONT case so let's just drop the warning.

Also we make a half-hearted attempt to handle a leading ! which
means only send it to the syslog not the console.  In this case
we just send to the the debug logs and not the console.
2009-10-27 16:13:35 -07:00
Brian Behlendorf 5c36312909 Script update-zfs.sh updated to include libefi library 2009-10-09 15:37:29 -07:00
Brian Behlendorf f44078fad5 Remove usage of the __id_u macro for portability.
This macro was removed from the default RPM macro file.  Interestly,
some of the arch specific macro's add it back it based on your distro
but it should not be counted on.  However, __id still exists and its
command line args have historically been fairly stable so we will
directly use %{__id} -un to get the user name.
2009-10-05 12:51:58 -07:00
Brian Behlendorf 39ab544079 Use kobject_set_name() for increased portability.
As of 2.6.25 kobj->k_name was replaced with kobj->name.  Some distros
such as RHEL5 (2.6.18) add a patch to prevent this from being a problem
but other older distros such as SLES10 (2.6.16) have not.  To avoid
the whole issue I'm updating the code to use kobject_set_name() which
does what I want and has existed all the way back to 2.6.11.
2009-10-02 16:21:59 -07:00
Brian Behlendorf 51a727e90f Set cwd to '/' for the process executing insmod.
Ricardo has pointed out that under Solaris the cwd is set to '/'
during module load, while under Linux it is set to the callers cwd.
To handle this cleanly I've reworked the module *_init()/_exit()
macros so they call a *_setup()/_cleanup() function when any SPL
dependent module is loaded or unloaded.  This gives us a chance to
perform any needed modification of the process, in this case changing
the cwd.  It also handily provides a way to avoid creating wrapper
init()/exit() functions because the Solaris and Linux prototypes
differ slightly.  All dependent modules should now call the spl
helper macros spl_module_{init,exit}() instead of the native linux
versions.

Unfortunately, it appears that under Linux there has been no consistent
API in the kernel to set the cwd in a module.  Because of this I have
had to add more autoconf magic than I'd like.  However, what I have
done is correct and has been tested on RHEL5, SLES11, FC11, and CHAOS
kernels.

In addition, I have change the rootdir type from a 'void *' to the
correct 'vnode_t *' type.  And I've set rootdir to a non-NULL value.
2009-10-01 16:06:15 -07:00
Brian Behlendorf 0e77fc118e Expand SEM() outside init_rwsem and directly call __init_rwsem().
We need to directly call __init_rwsem() or the name gets expanded
to SEM(lock-name).  This is safe and correct for the support arches
x86/x86_64/ppc/ppc64.
2009-09-29 03:19:09 -07:00
Brian Behlendorf 4d54fdee1d Reimplement mutexs for Linux lock profiling/analysis
For a generic explanation of why mutexs needed to be reimplemented
to work with the kernel lock profiling see commits:
  e811949a57 and
  d28db80fd0

The specific changes made to the mutex implemetation are as follows.
The Linux mutex structure is now directly embedded in the kmutex_t.
This allows a kmutex_t to be directly case to a mutex struct and
passed directly to the Linux primative.

Just like with the rwlocks it is critical that these functions be
implemented as '#defines to ensure the location information is
preserved.  The preprocessor can then do a direct replacement of
the Solaris primative with the linux primative.

Just as with the rwlocks we need to track the lock owner.  Here
things get a little more interesting because depending on your
kernel version, and how you've built your kernel Linux may already
do this for you.  If your running a 2.6.29 or newer kernel on a
SMP system the lock owner will be tracked.  This was added to Linux
to support adaptive mutexs, more on that shortly.  Alternately, your
kernel might track the lock owner if you've set CONFIG_DEBUG_MUTEXES
in the kernel build.  If neither of the above things is true for
your kernel the kmutex_t type will include and track the lock owner
to ensure correct behavior.  This is all handled by a new autoconf
check called SPL_AC_MUTEX_OWNER.

Concerning adaptive mutexs these are a very recent development and
they did not make it in to either the latest FC11 of SLES11 kernels.
Ideally, I'd love to see this kernel change appear in one of these
distros because it does help performance.  From Linux kernel commit:
  0d66bf6d3514b35eb6897629059443132992dbd7
  "Testing with Ingo's test-mutex application...
  gave a 345% boost for VFS scalability on my testbox"
However, if you don't want to backport this change yourself you
can still simply export the task_curr() symbol.  The kmutex_t
implementation will use this symbol when it's available to
provide it's own adaptive mutexs.

Finally, DEBUG_MUTEX support was removed including the proc handlers.
This was done because now that we are cleanly integrated with the
kernel profiling all this information and much much more is available
in debug kernel builds.  This code was now redundant.

Update mutexs validated on:
    - SLES10   (ppc64)
    - SLES11   (x86_64)
    - CHAOS4.2 (x86_64)
    - RHEL5.3  (x86_64)
    - RHEL6    (x86_64)
    - FC11     (x86_64)
2009-09-25 14:47:01 -07:00
Brian Behlendorf d28db80fd0 Update rwlocks to track owner to ensure correct semantics
The behavior of RW_*_HELD was updated because it was not quite right.
It is not sufficient to return non-zero when the lock is help, we must
only do this when the current task in the holder.

This means we need to track the lock owner which is not something
tracked in a Linux semaphore.  After some experimentation the
solution I settled on was to embed the Linux semaphore at the start
of a larger krwlock_t structure which includes the owner field.
This maintains good performance and allows us to cleanly intergrate
with the kernel lock analysis tools.  My reasons:

1) By placing the Linux semaphore at the start of krwlock_t we can
then simply cast krwlock_t to a rw_semaphore and pass that on to
the linux kernel.  This allows us to use '#defines so the preprocessor
can do direct replacement of the Solaris primative with the linux
equivilant.  This is important because it then maintains the location
information for each rw_* call point.

2) Additionally, by adding the owner to krwlock_t we can keep this
needed extra information adjacent to the lock itself.  This removes
the need for a fancy lookup to get the owner which is optimal for
performance.  We can also leverage the existing spin lock in the
semaphore to ensure owner is updated correctly.

3) All helper functions which do not need to strictly be implemented
as a define to preserve location information can be done as a static
inline function.

4) Adding the owner to krwlock_t allows us to remove all memory
allocations done during lock initialization.  This is good for all
the obvious reasons, we do give up the ability to specific the lock
name.  The Linux profiling tools will stringify the lock name used
in the code via the preprocessor and use that.

Update rwlocks validated on:
- SLES10   (ppc64)
- SLES11   (x86_64)
- CHAOS4.2 (x86_64)
- RHEL5.3  (x86_64)
- RHEL6    (x86_64)
- FC11     (x86_64)
2009-09-25 14:14:35 -07:00
Brian Behlendorf e811949a57 Reimplement rwlocks for Linux lock profiling/analysis.
It turns out that the previous rwlock implementation worked well but
did not integrate properly with the upstream kernel lock profiling/
analysis tools.  This is a major problem since it would be awfully
nice to be able to use the automatic lock checker and profiler.

The problem is that the upstream lock tools use the pre-processor
to create a lock class for each uniquely named locked.  Since the
rwsem was embedded in a wrapper structure the name was always the
same.  The effect was that we only ended up with one lock class for
the entire SPL which caused the lock dependency checker to flag
nearly everything as a possible deadlock.

The solution was to directly map a krwlock to a Linux rwsem using
a typedef there by eliminating the wrapper structure.  This was not
done initially because the rwsem implementation is specific to the arch.
To fully implement the Solaris krwlock API using only the provided rwsem
API is not possible.  It can only be done by directly accessing some of
the internal data member of the rwsem structure.

For example, the Linux API provides a different function for dropping
a reader vs writer lock.  Whereas the Solaris API uses the same function
and the caller does not pass in what type of lock it is.  This means to
properly drop the lock we need to determine if the lock is currently a
reader or writer lock.  Then we need to call the proper Linux API function.
Unfortunately, there is no provided API for this so we must extracted this
information directly from arch specific lock implementation.  This is
all do able, and what I did, but it does complicate things considerably.

The good news is that in addition to the profiling benefits of this
change.  We may see performance improvements due to slightly reduced
overhead when creating rwlocks and manipulating them.

The only function I was forced to sacrafice was rw_owner() because this
information is simply not stored anywhere in the rwsem.  Luckily this
appears not to be a commonly used function on Solaris, and it is my
understanding it is mainly used for debugging anyway.

In addition to the core rwlock changes, extensive updates were made to
the rwlock regression tests.  Each class of test was extended to provide
more API coverage and to be more rigerous in checking for misbehavior.

This is a pretty significant change and with that in mind I have been
careful to validate it on several platforms before committing.  The full
SPLAT regression test suite was run numberous times on all of the following
platforms.  This includes various kernels ranging from 2.6.16 to 2.6.29.

- SLES10   (ppc64)
- SLES11   (x86_64)
- CHAOS4.2 (x86_64)
- RHEL5.3  (x86_64)
- RHEL6    (x86_64)
- FC11     (x86_64)
2009-09-18 16:09:47 -07:00
Brian Behlendorf 45d1cae3b8 Rebase master to b121 2009-08-18 11:43:27 -07:00
Brian Behlendorf 73358d5a1d Various spec file tweaks to handle rpm building of several distros.
Supported and tested distros now include SLES10, SLES11, Chaos 4.x,
RHEL5, and Fedora 11.  This update was mainly to address rebuildable
kernel module rpms, and correct rpm dependencies for each distro.
2009-08-14 14:09:16 -07:00
Brian Behlendorf 26d77c4493 Explicit check for requires_* rpm defines
Due to different distros and/or versions of rpm mishandling the shorthand
syntax simply use the longer version which get interpreted correctly.
2009-08-13 15:02:34 -07:00
Brian Behlendorf 68ada11e5c Tag spl-0.4.5.
Update the ChangeLog with a summary of the changes since the last release
and update the META file to reflect the new version number.
2009-08-04 12:22:33 -07:00
Brian Behlendorf 16f4a92c10 Required missing symbols for FC11 kernels (2.6.29.4-167.fc11.x86_64) 2009-07-31 12:44:34 -07:00
Brian Behlendorf c65d62d8bf Disable stack overflow checking by default.
The run time stack overflow checking is being disabled by default
because it is not safe for use with 2.6.29 and latter kernels.  These
kernels do now have their own stack overflow checking so this support
has become redundant anyway.  It can be re-enabled for older kernels or
arches without stack overflow checking by redefining CHECK_STACK().
2009-07-30 13:52:11 -07:00
Brian Behlendorf 6ae7fef5b9 Update global_page_state() support for 2.6.29 kernels.
Basically everything we need to monitor the global memory state of
the system is now cleanly available via global_page_state().  The
problem is that this interface is still fairly recent, and there
has been one change in the page state enum which we need to handle.
These changes basically boil down to the following:
- If global_page_state() is available we should use it.  Several
  autoconf checks have been added to detect the correct enum names.
- If global_page_state() is not available check to see if
  get_zone_counts() symbol is available and use that.
- If the get_zone_counts() symbol is not exported we have no choice
  be to dynamically aquire it at load time.  This is an absolute
  last resort for old kernel which we don't want to patch to
  cleanly export the symbol.
2009-07-28 15:06:42 -07:00
Brian Behlendorf 6b09f73939 Remove get/put_task_struct as they are not available for SLES11
This interface is going away, and it's not as if most callers actually
use crhold/crfree when working with credentials.  So it'll be okay
they we're not taking a reference on the task structure the odds of
it going away while working with a credential and pretty small.
2009-07-28 15:04:21 -07:00
Brian Behlendorf ec7d53e99a Add basic credential support and splat tests.
The previous credential implementation simply provided the needed types and
a couple of dummy functions needed.  This update correctly ties the basic
Solaris credential API in to one of two Linux kernel APIs.

Prior to 2.6.29 the linux kernel embeded all credentials in the task
structure.  For these kernels, we pass around the entire task struct as if
it were the credential, then we use the helper functions to extract the
credential related bits.

As of 2.6.29 a new credential type was added which we can and do fairly
cleanly layer on top of.  Once again the helper functions nicely hide
the implementation details from all callers.

Three tests were added to the splat test framework to verify basic
correctness.  They should be extended as needed when need credential
functions are added.
2009-07-27 17:18:59 -07:00
Brian Behlendorf 3d0cb2d31d Remove LINUXINCLUDE from autoconf wrapper, breaks 2.6.28+ kernels.
Modern kernel build systems at least post 2.6.16 will set this properly
so we should not.  In fact post 2.6.28 the include headers have moved
under arch so the guess we make here is completely wrong.  Letting
the kernel build system set this ensure it will be correct.
2009-07-27 09:52:01 -07:00
Brian Behlendorf 7064b767c2 Positive Solaris ioctl return codes need to be negated for use by libc 2009-07-23 16:14:52 -07:00
Brian Behlendorf 3c9ce2bf69 Allow kmem or vmem based slab for slab_lock and slab_overcommit tests.
The slab_overcommit test case could hang on a system with fragmented
memory because it was creating a kmem based slab with 256K objects.
To avoid this I've removed the KMC_KMEM flag which allows the slab
to decide if it should be kmem or vmem backed based on the object
side.  The slab_lock test shares this code and will also be effected.
But the point of these two tests is to stress cache locking and
memory overcommit, the type of slab is not critical.  In fact, allowing
the slab to do the default smart thing is preferable.
2009-07-23 13:50:53 -07:00
Brian Behlendorf 2141116167 The HAVE_PATH_IN_NAMEIDATA compat macros should have been used here. 2009-07-22 14:28:19 -07:00
Brian Behlendorf 749e5eb1ed Check arch/default/ path when detecting kernel objects on SLES
We still preferentially use arch/arch looking for a native version
but if that fails it is acceptable to use default.
2009-07-22 06:59:28 -07:00
Brian Behlendorf 78d6de97bd Register a basic compat ioctl handler (32 vs 64 bit compat)
Simply pass the ioctl on to the normal handler.  If the ioctl
helper macros are used correctly this should be safe as they
will handle the packing/unpacking of the data encoded in the
ioctl command.  And actually, if the caller does not use the
IO* macros at all, and just passes small values, it will probably
be OK as well.  We only get in to trouble if they try and use
the upper 32-bits.  Endianness is not really a concern here, we
we are pretty much assumed they user and kernel will match.
2009-07-21 10:13:58 -07:00
Ricardo M. Correia ac95d0974b Fixed NULL dereference by tcd_for_each() when the kmalloc() call in module/spl/spl-debug.c:1163 returns NULL.
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2009-07-14 15:24:59 -07:00
Ricardo M. Correia e004f04c8b Prevent integer overflow after ~164 days of uptime.
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2009-07-14 15:23:25 -07:00
Brian Behlendorf b11b08ed64 Add a little paranoia here to ensure endianess is set correctly. 2009-07-14 14:28:04 -07:00
Brian Behlendorf 06dea10380 Add basic groupmember() function, not sup groups. 2009-07-10 10:58:06 -07:00
Brian Behlendorf d3126abe75 Add ddi_copyin/ddi_copyout support for fake kernel originated ioctls. 2009-07-10 10:56:32 -07:00
Brian Behlendorf 2a734e9c26 Define ACE_ALL_PERMS for use by ACLs 2009-07-09 15:00:25 -07:00
Brian Behlendorf c18cbcfe66 Define FKIOCTL which is used on Solaris to mark an in-kernel ioctl. 2009-07-09 14:59:41 -07:00
Brian Behlendorf 3a68dc5374 Add ASSERTV macro to simplify removing variables (the V in ASSERTV)
which are only used in ASSERT().
2009-07-09 12:15:23 -07:00
Brian Behlendorf 915404bd50 Add basic support for TASKQ_THREADS_CPU_PCT taskq flag which is
used to scale the number of threads based on the number of online
CPUs.  As CPUs are added/removed we should rescale the thread
count appropriately, but currently this is only done at create.
2009-07-09 10:07:52 -07:00
Brian Behlendorf 9babb37438 Rebase master to b117 2009-07-02 15:44:48 -07:00
Brian Behlendorf aaad2f7226 Update ChangeLog 2009-07-02 14:19:11 -07:00
Brian Behlendorf bb339d0670 Cleanly handle --with-linux=NONE option when used to generate source
rpms.  These should not be fatal because we actually don't need them
until we build the source rpm.  When doing mock builds this is
important because these dependent rpms will only be installed if
they are specificed in the source rpms spec file.
2009-07-02 10:47:28 -07:00
Brian Behlendorf 86933a6e51 Simplify rpm build rules, added config/rpm.am.
Distro friendly changes such that the kernel modules are packaged seperately.
2009-07-01 14:37:44 -07:00
Brian Behlendorf 5c3c70adec Add spl.release to spl-devel to simply dependent package version check. 2009-06-29 16:41:21 -07:00
Brian Behlendorf f4f9cd75a1 Install spl-devel products in /usr/src/spl-SPL_VERSION/LINUX_VERSION/
Remove the spl symlink, it's just confusing
2009-06-26 16:30:44 -07:00
Brian Behlendorf c0517c35d2 Use do_div on older kernel where do_div64 doesn't exist. 2009-06-26 13:10:52 -07:00
Brian Behlendorf 155189d4a7 Additional tuning to get the BuildRequires right for all cases.
pl.spec~
2009-06-26 12:43:27 -07:00
Brian Behlendorf ac12b26284 Simplify the kernel depenency logic 2009-06-26 11:37:06 -07:00
Brian Behlendorf af971a8594 Spec file update, for some reason the following shorthand syntax
was failing so it was replaced with the longer %if version.

%{!?foo: %define foo bar}

changed to

%if %{undefined foo}
 %define foo bar
%endif
2009-06-26 10:34:40 -07:00