Commit Graph

1765 Commits

Author SHA1 Message Date
Brian Behlendorf bb01879ebe Coverity 9654, 9654: Use After Free
Because vmem_free() was implemented as a macro using the ','
operator to evaluate both arguments and we performed the free
before evaluating size we would deference the free'd pointer.
To resolve the problem we just invert the ordering and evaluate
size first just as if it was evaluated by the caller when being
passed to this function.  This ensure that if the caller is
doing something reckless like performing an assignment as
part of the size argument we still perform it and it simply
doesn't get removed by the macro.  Oh course nobody should
be doing this sort of thing, but just in case.
2009-02-17 16:51:19 -08:00
Brian Behlendorf 15dc8b072e Coverity 9652, 9653: No Effect
Removed 2 ASSERT()s which had no effect because by definition
size_t is always an unsigned type thus is always >= 0.
2009-02-17 16:30:58 -08:00
Brian Behlendorf 014b1d6f54 Coverity 9641: Buffer Size
When SPLAT_TEST_INIT() initialized SPLAT_KMEM_TEST11_NAME the short
short test name overran the static length buffer of SPLAT_NAME_SIZE.
This was fixed by increasing the buffer length from 16 to 20 bytes.
2009-02-17 16:24:26 -08:00
Brian Behlendorf 9b1b8e4c24 kmem slab magazine ageing deadlock
- The previous magazine ageing sceme relied on the on_each_cpu()
  function to call spl_magazine_age() on each cpu.  It turns out
  this could deadlock with do_flush_tlb_all() which also relies
  on the IPI based on_each_cpu().  To avoid this problem a per-
  magazine delayed work item is created and indepentantly
  scheduled to the correct cpu removing the need for on_each_cpu().
- Additionally two unused fields were removed from the type
  spl_kmem_cache_t, they were hold overs from previous cleanup.
    - struct work_struct work
    - struct timer_list timer
2009-02-17 15:52:18 -08:00
Brian Behlendorf f6c5d4ff88 Build system update
- Added default build flags:
  -Wall -Wstrict-prototypes -Werror -Wshadow
- Added missing Makefile's for include/ subdirectories.
2009-02-12 14:45:22 -08:00
Brian Behlendorf 37db7d8cf9 kmem slab fixes
- Default SPL_KMEM_CACHE_DELAY changed to 15 to match Solaris.
- Aged out slab checking occurs every SPL_KMEM_CACHE_DELAY / 3.
- skc->skc_reap tunable added whichs allows callers of
  spl_slab_reclaim() to cap the number of slabs reclaimed.
  On Solaris all eligible slabs are always reclaimed, and this
  is still the default behavior.  However, I suspect that is
  not always wise for reasons such as in the next comment.
- spl_slab_reclaim() added cond_resched() while walking the
  slab/object free lists.  Soft lockups were observed when
  freeing large numbers of vmalloc'd slabs/objets.
- spl_slab_reclaim() 'sks->sks_ref > 0' check changes from
  incorrect 'break' to 'continue' to ensure all slabs are
  checked.
- spl_cache_age() reworked to avoid a deadlock with
  do_flush_tlb_all() which occured because we slept waiting
  for completion in spl_cache_age().  To waiting for magazine
  reclamation to finish is not required so we no longer wait.
- spl_magazine_create() and spl_magazine_destroy() shifted
  back to using for_each_online_cpu() instead of the
  spl_on_each_cpu() approach which was of course a bad idea
  due to memory allocations which Ricardo pointed out.
2009-02-12 13:32:10 -08:00
Ricardo M. Correia f500ccff35 Minor bug fix due to MAXOFFSET_T constant being too large on 32-bit systems. 2009-02-07 00:53:39 +00:00
Brian Behlendorf 4ab13d3b5c Additional Linux VM integration
Added support for Solaris swapfs_minfree, and swapfs_reserve tunables.
In additional availrmem is now available and return a reasonable value
which is reasonably analogous to the Solaris meaning.  On linux we
return the sun of free and inactive pages since these are all easily
reclaimable.

All tunables are available in /proc/sys/kernel/spl/vm/* and they may
need a little adjusting once we observe the real behavior.  Some of
the defaults are mapped to similar linux counterparts, others are
straight from the OpenSolaris defaults.
2009-02-05 12:26:34 -08:00
Brian Behlendorf 36b313dacf Linux VM integration / device special files
Support added to provide reasonable values for the global Solaris
VM variables: minfree, desfree, lotsfree, needfree.  These values
are set to the sum of their per-zone linux counterparts which
should be close enough for Solaris consumers.

When a non-GPL app links against the SPL we cannot use the udev
interfaces, which means non of the device special files are created.
Because of this I had added a poor mans udev which cause the SPL
to invoke an upcall and create the basic devices when a minor
is registered.  When a minor is unregistered we use the vnode
interface to unlink the special file.
2009-02-04 15:15:41 -08:00
Brian Behlendorf 31a033ecd4 2.6.27+ portability changes
- Added SPL_AC_3ARGS_ON_EACH_CPU configure check to determine
  if the older 4 argument version of on_each_cpu() should be
  used or the new 3 argument version.  The retry argument was
  dropped in the new API which was never used anyway.
- Updated work queue compatibility wrappers.  The old way this
  worked was to pass a data point when initialized the workqueue.
  The new API assumed the work item is embedding in a structure
  and we us container_of() to find that data pointer.
- Updated skc->skc_flags to be an unsigned long which is now
  type checked in the bit operations.  This silences the warnings.
- Updated autogen products and splat tests accordingly
2009-02-02 15:12:30 -08:00
Brian Behlendorf 416bae036b Add new workqueue header 2009-01-30 21:11:42 -08:00
Brian Behlendorf ea3e6ca9e5 kmem_cache hardening and performance improvements
- Added slab work queue task which gradually ages and free's slabs
  from the cache which have not been used recently.
- Optimized slab packing algorithm to ensure each slab contains the
  maximum number of objects without create to large a slab.
- Fix deadlock, we can never call kv_free() under the skc_lock.  We
  now unlink the objects and slabs from the cache itself and attach
  them to a private work list.  The contents of the list are then
  subsequently freed outside the spin lock.
- Move magazine create/destroy operation on to local cpu.
- Further performace optimizations by minimize the usage of the large
  per-cache skc_lock.  This includes the addition of KMC_BIT_REAPING
  bit mask which is used to prevent concurrent reaping, and to defer
  new slab creation when reaping is occuring.
- Add KMC_BIT_DESTROYING bit mask which is set when the cache is being
  destroyed, this is used to catch any task accessing the cache while
  it is being destroyed.
- Add comments to all the functions and additional comments to try
  and make everything as clear as possible.
- Major cleanup and additions to the SPLAT kmem tests to more
  rigerously stress the cache implementation and look for any problems.
  This includes correctness and performance tests.
- Updated portable work queue interfaces
2009-01-30 20:54:49 -08:00
Brian Behlendorf 0f233eac33 Pull the blkdev header in to the sunldi for some useful structure definitions and helper functions 2009-01-26 16:47:49 -08:00
Brian Behlendorf 48e0606a52 Implement kmem cache alignment argument 2009-01-26 09:02:04 -08:00
Brian Behlendorf e4f3ea278e Remove stray ` from macro 2009-01-23 08:59:11 -08:00
Brian Behlendorf 511176398c Update debug.h to standardize VERIFY3_IMPL error messages in debug and non-debug mode 2009-01-22 09:41:47 -08:00
Brian Behlendorf 1e4ed6c990 Add missing stub headers 2009-01-09 16:04:44 -08:00
Brian Behlendorf 121d48c97d Add basic ksid_lookupdomain and ksiddomain_rele support, just allocations 2009-01-09 15:30:53 -08:00
Brian Behlendorf 0e41414946 Add two new stub headers 2009-01-09 14:04:13 -08:00
Brian Behlendorf 97735c39e3 Add VOP_SEEK 2009-01-09 13:59:39 -08:00
Brian Behlendorf d83ba26e18 Add missing policy includes, add missing sun ddi bits 2009-01-09 10:49:47 -08:00
Brian Behlendorf 70997fb4b1 Add share.h stub 2009-01-09 10:06:18 -08:00
Brian Behlendorf 71c8ab9c68 Drat fix missing ; 2009-01-09 10:05:03 -08:00
Brian Behlendorf 23f5c4c281 Add missing callback_context_t and fid_t types 2009-01-09 10:03:37 -08:00
Brian Behlendorf 703e7a3cf4 Add stubs for three more includes 2009-01-09 09:47:27 -08:00
Brian Behlendorf d702c04ff1 Add 5 splat tests for list handling 2009-01-07 12:54:03 -08:00
Brian Behlendorf 4c18c39ecb Add include/sys/compress.h header 2009-01-06 09:47:00 -08:00
Brian Behlendorf 160c63ab76 Add P2BOUNDARY macro 2009-01-06 09:23:13 -08:00
Brian Behlendorf 7adbea4141 Pull in some default page typedefs 2009-01-05 16:14:38 -08:00
Brian Behlendorf 0f37204417 Add DTRACE_PROBE(a) 2009-01-05 16:09:21 -08:00
Brian Behlendorf b53c565e65 Stub u8_textprep.h for inclusion purposes 2009-01-05 15:37:07 -08:00
Brian Behlendorf e9cb2b4f64 Add system taskq support 2009-01-05 15:08:03 -08:00
Brian Behlendorf 8a2b328b18 Remove u8_textprep, we will not be implementing this nightmare yet 2009-01-05 11:32:08 -08:00
Brian Behlendorf f3fc90c249 Include the header 2008-12-23 16:48:15 -08:00
Brian Behlendorf 925ca8cc01 Add sys/thread.h 2008-12-23 16:27:36 -08:00
Brian Behlendorf bb9cfc6cc3 Define needfree 2008-12-23 15:59:36 -08:00
Brian Behlendorf 2b88beb74f Add timer.h header 2008-12-23 15:40:20 -08:00
Brian Behlendorf bbdec3be06 Add u8 stub 2008-12-23 15:38:15 -08:00
Brian Behlendorf de79fdd3a8 Move sunddi include 2008-12-23 13:32:07 -08:00
Brian Behlendorf 9d457afd1b Add sunddi to uio 2008-12-23 13:30:04 -08:00
Brian Behlendorf dc0f920710 Minor updates 2008-12-23 13:25:52 -08:00
Brian Behlendorf 926e2b6058 Pull in lock types 2008-12-23 13:18:39 -08:00
Brian Behlendorf c1d42c2f1d Add header 2008-12-23 13:05:50 -08:00
Brian Behlendorf f5b92a66ad Add a few more missing header which the upstream stock kernel context expects 2008-12-23 13:03:09 -08:00
Brian Behlendorf 2ee63a549a Add struct ddi_strtox functions 2008-12-05 16:23:57 -08:00
Brian Behlendorf 72e7de6026 Prefix META_ALIAS with SPL_ 2008-11-26 13:26:05 -08:00
Brian Behlendorf abc3ca149d Prefix all META_* #defines with SPL to prevent colisions which include our spl_config.h. Dependent packages may do this to leverage the autoconf check we have already run aganst the kernel. 2008-11-26 13:09:37 -08:00
behlendo 7212e2cd27 Add missing autogen products
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@182 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-11-26 17:07:59 +00:00
behlendo 6a1c3d418a * include/sys/sunddi.h, modules/spl/spl-module.c : Removed default
udev support from sunddi implementation because it uses GPL-only
symbols.  This support is optionally available for SPL consumers
if they define HAVE_GPL_ONLY_SYMBOLS and license their module as
GPL using the MODULE_LICENSE("GPL") macro.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@179 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-11-13 21:43:30 +00:00
behlendo 36833ea4e4 Slightly increase SPLAT_NAME_SIZE to ensure string is always
NULL terminated.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@174 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-11-05 21:27:31 +00:00
behlendo 0498e6c585 Removed useless check
Fix forward NULL in splat kmem_cache test ctors/dtors



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@171 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-11-04 23:18:31 +00:00
behlendo c8e60837b7 * spl-09-fix-kmem-track-oops.patch
This fixes an oops when unloading the modules, in the case where memory
tracking was enabled and there were memory leaks. The comment in the
code explains what was the problem.

* spl-10-fix-assert-verify-ndebug.patch
This fixes ASSERT*() and VERIFY*() macros in non-debug builds. VERIFY*()
macros are supposed to check the condition and panic even in production
builds, and ASSERT*() macros don't need to evaluate the arguments.
Also some 32-bit fixes.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@165 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-11-03 22:02:15 +00:00
behlendo c22e7a427b Under Solaris KM_SLEEP ensures success (or at least you hang forever).
That said when working with a finite resource like memory failure really
is always a possibility.  It would be far better longer term if the ZFS
code could be weened off this assumption and properly handle the cases
where an allocation fails.  Still I've applied the patch to spl-0.3.4
since this layer is supposed to emulate Solaris as closely as possible.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@164 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-11-03 21:51:33 +00:00
behlendo a0f6da3d95 Add a SPL_AC_TYPE_ATOMIC64_T test to configure for systems which do
already supprt atomic64_t types.

* spl-07-kmem-cleanup.patch
This moves all the debugging code from sys/kmem.h to spl-kmem.c, because
the huge macros were hard to debug and were bloating functions that
allocated memory. I also fixed some other minor problems, including
32-bit fixes and a reported memory leak which was just due to using the
wrong free function.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@163 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-11-03 21:06:04 +00:00
behlendo 550f170525 Apply two nice improvements caught by Ricardo,
spl-05-div64.patch
This is a much less intrusive fix for undefined 64-bit division symbols
when compiling the DMU in 32-bit kernels.

* spl-06-atomic64.patch
This is a workaround for 32-bit kernels that don't have atomic64_t.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@162 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-11-03 20:34:17 +00:00
behlendo 749045bbfa Apply a nice fix caught by Ricardo,
* spl-04-fix-taskq-spinlock-lockup.patch
Fixes a deadlock in the BIO completion handler, due to the taskq code
prematurely re-enabling interrupts when another spinlock had disabled
them in the IDE IRQ handler.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@161 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-11-03 20:21:08 +00:00
behlendo f6c81c5ea7 Reviewed and applied spl-01-rm-gpl-symbol-set_cpus_allowed.patch
from Ricardo which removes a dependency on the GPL-only symbol
set_cpus_allowed().  Using this symbol is simpler but in the name
of portability we are adopting a spinlock based solution here
to remove this dependency.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@160 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-11-03 20:07:20 +00:00
behlendo 25557fd884 Sigh more compat fixes, this is almost right for 2.6.9 - 2.6.26 kernels.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@157 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-08-11 23:47:44 +00:00
behlendo b61a6e8bdc Pull in initial 32-bit support patches.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@156 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-08-11 22:42:04 +00:00
behlendo 3d061e9d10 Commit bulk of remaining 2.6.9 and 2.6.26 compat changes.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@155 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-08-11 22:13:47 +00:00
behlendo 322640b7b5 Include linux/uaccess.h compat changes.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@154 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-08-11 19:10:14 +00:00
behlendo 86de8532a9 More 2.6.26 compat changes
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@153 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-08-11 17:56:40 +00:00
behlendo 6a6cafbe8d Pull in timespec, list, and type compat changes to support
building against a wider range of kernels.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@152 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-08-11 17:20:11 +00:00
behlendo 46c685d0c4 Add class / device portability code. Two autoconf tests
were added to cover the 3 possible APIs from 2.6.9 to
2.6.26.  We attempt to use the newest interfaces and if
not available fallback to the oldest.  This a rework of
some changes proposed by Ricardo for RHEL4.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@150 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-08-10 03:50:36 +00:00
behlendo 877a32e91e Pull in fls64 compat changes from spl-00-rhel4-compat.patch,
to allow greater compatibility with kernels pre 2.6.16.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@149 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-08-06 04:52:39 +00:00
behlendo 7afde631f6 Start bringing in Ricardo's spl-00-rhel4-compat.patch, a few chunks
at a time as I audit it.  This chunk finishes moving the SPL entirely
off the linux slab on to the SPL implementation.  It differs slightly
from the proposed version in that the spl continues to export to
all the Solaris types and functions.  These do conflict with the
Linux slab so a module usings these interfaces must not include the
SPL slab if they also intend to use the linux slab.  Or they must
explcitly #undef the macros which remap the functioin to their
spl_* equivilants.

A nice side of effect of dropping the entire linux slab is we
don't need to autoconf checks anymore.  They kept messing with
the slab API endlessly!



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@148 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-08-05 04:16:09 +00:00
behlendo a1502d76ae - Remove hash functionality from slab in favor of direct lookups
based of the spl_kmem_obj_t tacked on the end of each object.
  This actually isn't so back because we are now allocing large
  chunks for the slab and partitioning it ourselves.  So there's
  not a ton of wasted space.  We may suffer a performance hit
  however due to alignment issues.

- Remove remaining depenancies on the linux slab implementation.
  We're standing on our own now for better or worse.

- Rework slabs to be either kmem or vmem based.  If neither
  KMC_VMEM of KMC_KMEM are specified we make a decent guess
  about what will work best for their based on the object 
  size.  Additionally we provide a kmem_virt() function caller
  can use to see if they have a virtual or physical address.

- Minor fixups in the test suite.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@141 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-07-01 03:28:54 +00:00
behlendo 1c3832576d Remove stray call to spl_cache_free() and remove all the
cycle count which was costing me overhead.  It was hurting
performance pretty badly for heavily used caches.  I'm also
thinking the hash may be hurting me as well and it might
be worth sticking a pointer in to a little space after the
alloced object.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@140 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-28 20:03:11 +00:00
behlendo fece7c99bf Victory! I've reworked caches with large objects which are
based by vmalloc()'ed memory.  I now alloc a slab which is
roughly 32*spl_obj_size and in this block of memory I place
the slab descriptor, slab object descriptors, and objects
themselves.  This greatly reduces vmalloc lock contention.

Still some minor cleanup remains and fine tuning but
it's working pretty well.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@139 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-28 05:04:46 +00:00
behlendo ff449ac406 Further slab improvements, I'm getting close to something which works
well for the expected workloads.  Improvement in this commit include:

- Added DEBUG_KMEM_TRACKING #define which can optionally be set
  when DEBUG_KMEM is defined to do per allocation tracking.  This
  allows us to get all the lightweight kmem debugging enabled by
  default which is pretty light weight, and only when looking 
  for a memory leak we can briefly enable the per alloc tracking.

- Added set_normalized_timespec() in to SPL to simply using
  the timespec() primatives from within a module.

- Added per-spinlock cycle counters to the slab in an attempt
  to run down a lock contention issue.  The contended lock 
  was in vmalloc() but I'm going to leave the cycle counters
  in place for a little while until I'm convinced there arn't
  other locking improvement possible in the slab.

- Added a proc interface to the slab to export per slab
  cache statistics to /proc/spl/kmem/slab for analysis.

- Reworked spl_slab_alloc() function to allocate from kmem for
  small allocation and vmem for large allocations.  This improved
  things considerably but futher work is needed.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@138 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-27 21:40:11 +00:00
behlendo e9d7a2bef5 Fix for memory corruption caused by overruning the magazine
when repopulating it.  Plus I fixed a few more suble races in
that part of the code which were catching me.  Finally I fixed
a small race in kmem_test8.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@137 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-26 19:49:42 +00:00
behlendo 4afaaefa05 Implement per-cpu local caches. This seems to have bough me another
factor of 10x improvement on SMP system due to reduced lock contention.
This may put me in the ballpark of what is needed.  We can still further
improve things on NUMA systems by creating an additional L3 cache per 
memory node instead of the current global pool.  With luck this won't
be needed.  I should also take another look at the locking now that
everything is working.  There's a good chance I can tighten it up a
little bit and improve things a little more.

   kmem_lock: time (sec)        slabs           objs            hash
   kmem_lock:                   tot/max/calc    tot/max/calc    size/depth
   kmem_lock:  0.000999926      6/6/1           192/192/32      32768/0
   kmem_lock:  0.000999926      4/4/2           128/128/64      32768/0
   kmem_lock:  0.000999926      4/4/4           128/128/128     32768/0
   kmem_lock:  0.000999926      4/4/8           128/128/256     32768/0
   kmem_lock:  0.000999926      4/4/16          128/128/512     32768/0
   kmem_lock:  0.000999926      4/4/32          128/128/1024    32768/0
   kmem_lock:  0.000999926      4/4/64          128/128/2048    32768/0
   kmem_lock:  0.000999926      8/8/128         256/256/4096    32768/0
   kmem_lock:  0.003999704      24/23/256       768/736/8192    32768/1
   kmem_lock:  0.012999038      44/41/512       1408/1312/16384 32768/1
   kmem_lock:  0.051996153      96/93/1024      3072/2976/32768 32768/2
   kmem_lock:  0.181986536      187/184/2048    5984/5888/65536 32768/3
   kmem_lock:  0.655951469      342/339/4096    10944/10848/131072 32768/4



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@136 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-25 20:57:45 +00:00
behlendo d46630e0f3 The first locking issue was due to the semaphore I used. I was trying
to be overly clever and the context switch when the semaphore was busy
was destroying performance.  Converting to a simple spin lock bough me
a factor of 50 or so.  That said it's still not good enough.  Tests
show bad performance and we are still CPU bound.  The logical fix is
I need to implement per-cpu hot caches to minimize the SMP contention.
Linux and Solaris both have this, I was hoping to do without but it
looks like that's not to be.

   kmem_lock: time (sec)        slabs           objs            hash
   kmem_lock:                   tot/max/calc    tot/max/calc    size/depth
   kmem_lock:  0.022000000      7/6/64  224/177/2048    32768/1
   kmem_lock:  0.039000000      13/13/128       416/404/4096    32768/1
   kmem_lock:  0.079000000      23/21/256       736/672/8192    32768/1
   kmem_lock:  0.158000000      48/47/512       1536/1504/16384 32768/1
   kmem_lock:  0.345000000      105/105/1024    3360/3358/32768 32768/2
   kmem_lock:  0.760000000      202/200/2048    6464/6400/65536 32768/3



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@135 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-24 17:18:15 +00:00
behlendo 5cbd57fa91 Fix minor chaos/fc9 kernel discrepencies in build
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@133 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-13 23:56:26 +00:00
behlendo 2fb9b26a85 * : modules/sys/kmem-slab.c : Re-implemented the slab to no
longer be based on the linux slab but to be its own complete
implementation.  The new slab behaves much more like the
Solaris slab than the Linux slab.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@132 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-13 23:41:06 +00:00
behlendo 41cf38df92 Add missing () to quiet warnings in NDEBUG case
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@128 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-04 22:52:13 +00:00
behlendo 475cdc788e Just use CONFIG_SLUB to detect SLUB use
Add ASSERTF to the NDEBUG build
Fix minor issue with various debug build flags



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@126 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-04 21:09:25 +00:00
behlendo c30df9c863 Fixes:
1) Ensure mutex_init() never fails in the case of ENOMEM by retrying
   forever.  I don't think I've ever seen this happen but it was clear
   after code inspection that if it did we would immediately crash.

2) Enable full debugging in check.sh for sanity tests.  Might as well
   get as much debug as we can in the case of a failure.

3) Reworked list of kmem caches tracked by SPL in to a hash with the
   key based on the address of the kmem_cache_t.  This should speed
   up the constructor/destructor/shrinker lookup needed now for newer
   kernel which removed the destructor support.

4) Updated kmem_cache_create to handle the case where CONFIG_SLUB
   is defined.  The slub would occasionally merge slab caches which
   resulted in non-unique keys for our hash lookup in 3).  To fix this
   we detect if the slub is enabled and then set the needed flag
   to prevent this merging from ever occuring.

5) New kernels removed the proc_dir_entry pointer from items
   registered by sysctl.  This means we can no long be sneaky and
   manually insert things in to the sysctl tree simply by walking
   the proc tree.  So I'm forced to create a seperate tree for
   all the things I can't easily support via sysctl interface.
   I don't like it but it will do for now.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@124 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-04 06:00:46 +00:00
behlendo 691d2bd733 Update utsname to use proper compatible interface to avoid API issues.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@123 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-03 21:20:18 +00:00
behlendo 57d862349b Breaking the world for a little bit. If anyone is going to continue
working on this branch for the next few days I suggested you work
off of the 0.3.1 tag.  The following changes are fairly extensive
and are designed to make the SPL compatible with all kernels in
the range of 2.6.18-2.6.25.  There were 13 relevant API changes
between these releases and I have added the needed autoconf tests
to check for them.  However, this has not all been tested extensively.
I'll sort of the breakage on Fedora Core 9 and RHEL5 this week.

SPL_AC_TYPE_UINTPTR_T
SPL_AC_TYPE_KMEM_CACHE_T
SPL_AC_KMEM_CACHE_DESTROY_INT
SPL_AC_ATOMIC_PANIC_NOTIFIER
SPL_AC_3ARGS_INIT_WORK
SPL_AC_2ARGS_REGISTER_SYSCTL
SPL_AC_KMEM_CACHE_T
SPL_AC_KMEM_CACHE_CREATE_DTOR
SPL_AC_3ARG_KMEM_CACHE_CREATE_CTOR
SPL_AC_SET_SHRINKER
SPL_AC_PATH_IN_NAMEIDATA
SPL_AC_TASK_CURR
SPL_AC_CTL_UNNUMBERED



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@119 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-06-02 17:28:49 +00:00
behlendo 715f625146 Go through and add a header with the proper UCRL number.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@114 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-26 04:38:26 +00:00
behlendo cc7449ccd6 - Properly fix the debug support for all the ASSERT's, VERIFIES, etc can be
compiled out when doing performance runs.
- Bite the bullet and fully autoconfize the debug options in the configure
  time parameters.  By default all the debug support is disable in the core
  SPL build, but available to modules which enable it when building against
  the SPL.  To enable particular SPL debug support use the follow configure
  options:

  --enable-debug		Internal ASSERTs
  --enable-debug-kmem		Detailed memory accounting
  --enable-debug-mutex		Detailed mutex tracking
  --enable-debug_kstat          Kstat info exported to /proc
  --enable-debug-callb		Additional callb debug



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@111 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-19 02:49:12 +00:00
behlendo 6ab69573ff SPL additions to increase support for updated ZFS build
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@110 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-15 23:39:19 +00:00
behlendo 4efd41189a Rework condition variable implementation to be consistent with
other primitive implementations.  Additionally ensure that GFP_ATOMIC
is use for allocations when in interrupt context.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@108 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-15 17:10:30 +00:00
behlendo 8464443f8d Add a comment so I remember to fix this.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@106 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-12 16:53:41 +00:00
behlendo c6dc93d6a8 By default disable extra KMEM and MUTEX debugging to aid performance.
They can easily be re-enabled when new stability issues are uncovered.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@105 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-09 22:53:20 +00:00
behlendo 5c2bb9b2c3 Stability hack. Under Solaris when KM_SLEEP is set kmem_cache_alloc()
may not fail.  To get this behavior I'd added a retry to the shim layer
even though it is abusive to the VM, at least it should prevent the crash.
Additionally I added a proc counter so I can easily check how often this
is happening.  It should be fairly rare, but likely will get worse and
worse the longer the machine has been up.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@104 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-09 21:21:33 +00:00
behlendo 04a479f706 Add an almost feature complete implemenation of kstat. I chose
not to support a few flags (we assert if they are used), and I
did not add the libkstat interface and instead exported everything
to proc for easy access.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@103 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-08 23:21:47 +00:00
behlendo 427a782d7d Decrease of kmem warnign threshold back to 2 pages, no worse than a stack.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@100 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-07 19:33:01 +00:00
behlendo 13cdca65ec Add vmem memory accounting
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@99 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-07 18:54:32 +00:00
behlendo 404992e31a - Relocate 'stats_per' in to proper /proc/sys/spl/mutex/ directory
- Shift to spinlock for mutex list addition and removal



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@98 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-07 17:58:22 +00:00
behlendo 4f86a887d8 Remaining issues fixed after reenabled mutex debugging.
- Ensure the mutex_stats_sem and mutex_stats_list are initialized
- Only spin if you have to in mutex_init



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@97 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-06 23:19:27 +00:00
behlendo e8b31e8482 - Updated rwlock's to reside in a .c file instead of a static inline
- Updated rwlock's so they can be safely initialized in ctors.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@96 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-06 23:00:49 +00:00
behlendo d6a26c6a32 Lots of fixes here:
- Detailed kmem memory allocation tracking.  We can now get on
  spl module unload a list of all memory allocations which were
  not free'd and where the original alloc was.  E.g.

SPL: 15554:632:(spl-kmem.c:442:kmem_fini()) kmem leaked 90/319332 bytes
SPL: 15554:648:(spl-kmem.c:451:kmem_fini()) address          size  data             func:line
SPL: 15554:648:(spl-kmem.c:457:kmem_fini()) ffff8100734b68b8 32    0100000001005a5a __spl_mutex_init:70
SPL: 15554:648:(spl-kmem.c:457:kmem_fini()) ffff8100734b6148 13    &tl->tl_lock     __spl_mutex_init:74
SPL: 15554:648:(spl-kmem.c:457:kmem_fini()) ffff81007ac43730 32    0100000001005a5a __spl_mutex_init:70
SPL: 15554:648:(spl-kmem.c:457:kmem_fini()) ffff81007ac437d8 13    &tl->tl_lock     __spl_mutex_init:74

- Shift to using rwsems in kmem implmentation, to simply locking and
  improve concurency.

- Shift to using rwsems in mutex implementation, additionally ensure we
  never sleep in the init function if non-zero preempt_count or 
  interrupts are disabled as can happen in a slab cache ctor/dtor.

- Other minor formating fixes and such.

TODO:

- Finish the vmem memory allocation tracking

- Vet all other SPL primatives for potential sleeping during *_init.  I
suspect the rwlock implemenation does this and should be fixes just
like the mutex implemenation.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@95 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-06 20:38:28 +00:00
behlendo 9ab1ac14ad Commit adaptive mutexes. This seems to have introduced some new
crashes but it's not clear to me yet if these are a problem with
the mutex implementation or ZFSs usage of it.

Minor taskq fixes to add new tasks to the end of the pending list.

Minor enhansements to the debug infrastructure.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@94 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-05 20:18:49 +00:00
behlendo bcd68186d8 New an improved taskq implementation for the SPL. It allows a
configurable number of threads like the Solaris version and almost
all of the options are supported.  Unfortunately, it appears to have
made absolutely no difference to our performance numbers.  I need
to keep looking for where we are bottle necking.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@93 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-25 22:10:47 +00:00
behlendo 839d8b438e Update kmem.h to properly use new debug subsystem.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@92 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-24 20:21:07 +00:00
behlendo 3561541c24 Prep for 0.2.1 tag
Minor fixes to headers to use debug macros
Added /proc/sys/spl/version



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@90 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-24 17:41:23 +00:00
wartens2 8100fe56f1 Make sure that when calling __vmem_alloc that we
do not have __GFP_ZERO set.  Once the memory is allocated
then zero out the memory if __GFP_ZERO is passed to
__vmem_alloc.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@88 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-24 17:07:56 +00:00
behlendo 6e605b6e58 Minor improvement to taskq handling. This is a small step towards
dynamic taskqs which still need to be fully implemented.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@87 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-23 21:19:47 +00:00
behlendo b831734a43 Stack usage is my enemy. Trade cpu cycles in the debug code to
ensure I never add anything to the stack I don't absolutely need.
All this debug code could be removed from a production build
anyway so I'm not so worried about the performance impact.  We
may also consider revisting the mutex and condvar implementation
to ensure no additional stack is used there.

Initial indications are I have reduced the worst case stack
usage to 9080 bytes.  Still to large for the default 8k stacks
so I have been forced to run with 16k stacks until I can
reduce the worst offenders.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@83 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-22 16:55:26 +00:00
behlendo 7fea96c04f More fixes to ensure we get good debug logs even if we're in the
process of destroying the stacks.  Threshhold set fairly aggressively
top 80% of stack usage.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@82 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-21 22:44:11 +00:00
behlendo 892d51061e Handful of minor stack checking fixes
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@79 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-21 18:08:33 +00:00
behlendo 937879f11d Update SPL to use new debug infrastructure. This means:
- Replacing all BUG_ON()'s with proper ASSERT()'s
- Using ENTRY,EXIT,GOTO, and RETURN macro to instument call paths



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@78 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-21 17:29:47 +00:00
behlendo 2fae1b3d0a Frist minor batch of fixes. Catch a dropped ;, and use SBUG instead of BUG.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@77 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-19 00:02:11 +00:00
behlendo ce86265693 Whoops need this!
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@76 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-18 23:42:45 +00:00
behlendo 57d1b18858 First commit of lustre style internal debug support. These
changes bring over everything lustre had for debugging with
two exceptions.  I dropped by the debug daemon and upcalls
just because it made things a little easier.  They can be
readded easily enough if we feel they are needed.

Everything compiles and seems to work on first inspection
but I suspect there are a handful of issues still lingering
which I'll be sorting out right away.  I just wanted to get
all these changes commited and safe.  I'm getting a little
paranoid about losing them.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@75 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-18 23:39:58 +00:00
behlendo d61e12af5a - Add some spinlocks to cover all the private data in the mutex. I don't
think this should fix anything but it's a good idea regardless.

- Drop the lock before calling the construct/destructor for the slab
otherwise we can't sleep in a constructor/destructor and for long running
functions we may NMI.

- Do something braindead, but safe for the console debug logs for now.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@73 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-15 20:53:36 +00:00
behlendo c5fd77fcbf Just cleanup up an error case to avoid overspamming the console.
We get the stack once from the BUG() no reason to dump it twice.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@72 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-14 18:37:20 +00:00
behlendo 12ea923056 Adjust the condition variables to simply sleep uninteruptibly.
This way we don't have to contend with superious wakeups which
it appears ZFS is not so careful to handle anyway.  So this is
probably for the best.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@70 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-11 22:49:48 +00:00
behlendo 115aed0dd8 - Add more strict in_atomic() checking to the mutex entry
function just to be extra safety and paranoid.

- Rewrite the thread shim to take full advantage of the
new kernel kthread API.  This greatly simplifies things.

- Add a new regression test for thread_exit() to ensure
it properly terminates a thread immediately without
allowing futher execution of the thread.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@69 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-11 17:03:57 +00:00
behlendo 79f92663e3 Fix race in rwlock implementation which can occur when
your task is rescheduled to a different cpu after you've
taken the lock but before calling RW_LOCK_HELD is called.
We need the spinlock to ensure there is a wmb() there.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@68 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-07 23:54:34 +00:00
behlendo 968eccd1d1 Update the thread shim to use the current kernel threading API.
We need to use kthread_create() here for a few reasons.  First
off to old kernel_thread() API functioin will be going away.
Secondly, and more importantly if I use kthread_create() we can
then properly implement a thread_exit() function which terminates
the kernel thread at any point with do_exit().  This fixes our
cleanup bug which was caused by dropping a mutex twice after
thread_exit() didn't really exit.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@66 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-04 04:44:16 +00:00
behlendo 996faa6869 Correctly implement atomic_cas_ptr() function. Ideally all of these
atomic operations will be rewritten anyway with the correct arch
specific assembly.  But not today.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@65 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-03 21:48:57 +00:00
behlendo 0a6fd143fd - Remapped ldi_handle_t to struct block_device * which is much more useful
- Added liunx block device headers to sunldi.h
- Made __taskq_dispatch safe for interrupt context where it turns out we
  need to be useing it.
- Fixed NULL const/dest bug for kmem slab caches
- Places debug __dprintf debugging messages under a spin_lock_irqsave
  so it's safe to use then in interrupt handlers.  For debugging only!



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@64 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-03 16:33:31 +00:00
behlendo 0998fdd6db Apparently it's OK for done to be NULL, which was not clear in the
Solaris man page.  Anyway, since apparently this usage is accectable
I've updated the function to handle it.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@63 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-01 17:00:06 +00:00
behlendo 6a585c61de Double large kmalloc warning size to 4 pages. It was 2 pages, and ideally
it should be dropped to one page but in the short term we should be able
to easily live with 4 page allocations.

Fix the nvlist bug, it turns out the user space side of things were
packing the nvlists correctly as little endian, and the kernel space
side of things due to a missing #define were unpacking them as big endian.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@62 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-01 16:09:18 +00:00
behlendo 4fd2f7eea5 Add vmem_zalloc support.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@60 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-31 23:04:07 +00:00
behlendo 8d0f1ee907 Add some crude debugging support. It leaves alot to be
desired, but it should allow more easy kernel debugging for now.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@59 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-31 20:42:36 +00:00
behlendo 9f4c835a0e Correctly functioning 64-bit atomic shim layer. It's not
what I would call effecient but it does have the advantage
of being correct which is all I need right now.  I added
a regression test as well.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@57 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-28 18:21:09 +00:00
behlendo 4a4295b267 Remove minor lingering CDDL tait of copied headers. Required
headers rewritten to include minimally what we need.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@56 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-27 23:40:09 +00:00
behlendo d429b03d85 - Thinko fix to the SPL module interface
- Enhanse the VERIFY() support to output the values which
  failed to compare as expected before crashing.  This make
  debugging much much much easier.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@55 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-27 22:06:59 +00:00
behlendo 8ac547ec4c Relocated to zfs repo
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@54 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-27 17:58:10 +00:00
behlendo 336bb0c0c1 Two fixes to the module interface. Could be worse!
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@53 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-21 19:16:25 +00:00
behlendo 4e62fd4104 OK, a first reasonable attempt at a solaris module/chdev shim layer.
This should handle the absolute minimum I need for ZFS.  It will 
register the chdev with the right callbacks.  Then the generic 
registered linux callback will find the right registered solaris
callback for the function and munge the args just right before
passing it on.  Should work, but untested (just compiled), so I
expect bugs.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@52 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-20 23:30:15 +00:00
behlendo e4f1d29f89 OK, some pretty substantial rework here. I've merged the spl-file
stuff which only inclused the getf()/releasef() in to the vnode area
where it will only really be used.  These calls allow a user to
grab an open file struct given only the known open fd for a particular
user context.  ZFS makes use of these, but they're a bit tricky to
test from within the kernel since you already need the file open
and know the fd.  So we basically spook the system calls to setup
the environment we need for the splat test case and verify given
just the know fd we can get the file, create the needed vnode, and
then use the vnode interface as usual to read and write from it.

While I was hacking away I also noticed a NULL termination issue
in the second kobj test case so I fixed that too.  In fact, I fixed
a few other things as well but all for the best!



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@51 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-18 23:20:30 +00:00
behlendo 5d86345d37 Initial pass at a file API getf/releasef hooks
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@50 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-18 04:56:43 +00:00
behlendo 1ec74a114c Minimal signal handling interface.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@49 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-17 18:29:57 +00:00
behlendo 2bdb28fbe0 Missing headers, more minor fixes
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@48 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-15 00:05:38 +00:00
behlendo c19c06f3b0 Fix kmem memory accounting
Adjust kmem slab interface to make a copy of the slab name before
passing it on to the linux slab (we free it latter too)



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@47 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-14 20:56:26 +00:00
behlendo 79b31f3601 Fix KMEM_DEBUG support (enable by default)
Add vmem_alloc/vmem_free support (and test case)
Add missing time functions



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@46 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-14 19:04:41 +00:00
behlendo af828292e5 Add missing headers
Rework vnodes to be based on the slab cache, just like on Solaris.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@45 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-14 00:04:01 +00:00
behlendo ea19fbed05 Add missing headers
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@44 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-13 22:52:23 +00:00
behlendo 8ddd0ee415 Add two more missing headers
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@43 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-13 20:41:29 +00:00
behlendo 73e540a0d1 Drop unicode support, provided in ZFS tree libport
Update uio support


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@42 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-13 19:49:09 +00:00
behlendo 36e6f86146 - Add some more missing headers
- Map the LE/BE_* byteorder macros to the linux versions
- More minor vnodes fixes


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@41 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-12 23:48:28 +00:00
behlendo 2f5d55aac5 Add copyin/copyout mapping
Fix some vnode types



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@40 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-12 21:33:28 +00:00
behlendo 4b17158506 - Implemented vnode interfaces and 6 test cases to the test suite.
- Re-implmented kobj support based on the vnode support.
- Add TESTS option to check.sh, and removed delay after module load.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@39 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-12 20:52:46 +00:00
behlendo 9490c14835 Apply fix from bug239 for rwlock deadlock.
Update check.sh script to take V=1 env var so you can run it verbosely as
follows if your chasing something: sudo make check V=1

Add new kobj api and needed regression tests to allow reading of files from
within the kernel.  Normally thats not something I support but the spa layer
needs the support for its config file.

Add some more missing stub headers



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@38 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-11 20:54:40 +00:00
behlendo b123971fc2 Two more GPL only symbols moved to helper functions in the spl module.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@37 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-11 02:08:57 +00:00
behlendo ee4766827a Remap gethrestime() with #define to new symbol and export that new
symbol to avoid direct use of GPL only symbol.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@36 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-10 21:38:39 +00:00
behlendo 51f443a074 Add some typedefs to make it clearer when we passing a function,
Add fm_panic define
Add another bad atomic hack (need to do this right)


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@35 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-10 19:25:20 +00:00
behlendo 4098c921b6 Fix systemic naming mistake
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@34 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-10 19:04:14 +00:00
behlendo 6adf99e7d6 Add missing headers
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@33 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-10 17:05:34 +00:00
behlendo 12472b242d Just filling in more of the env.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@32 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-08 00:58:32 +00:00
behlendo 05ae387b50 Add somre debugging support
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@31 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-08 00:18:21 +00:00
behlendo 0b3cf046cb Add the initial vestigates of vnode support
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@30 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-07 23:07:02 +00:00
behlendo 3b3ba48fe9 Add missing cred.h functions
Resolve compiler warning with kmem_free (unused len)
Add stub for byteorder.h
Add zlib shim for compress2 and uncompress functions



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@29 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-07 20:48:44 +00:00
behlendo b0dd3380aa Minor atomic cleanup, this needs to be done right.
Fixed a bug in the timer code
Added missing macros



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@28 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-07 00:28:32 +00:00
behlendo ed61a7d05e Add some missing rw_lock symbols
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@27 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-06 23:42:37 +00:00
behlendo 77b1fe8fa8 Add highbit func,
Add sloopy atomic declaration which will need to be fixed (eventually)
Fill out more of the Solaris VM hooks
Adjust the create_thread function



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@26 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-06 23:12:55 +00:00
behlendo a713518f5d Checkpoint for the night,
added a few more stub headers,
fleshed out a few stub headers,
added a FIXME file,
added various compatibility macros


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@25 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-05 00:58:54 +00:00
behlendo 23f28c4f75 Remove spl.h, just include the headers you need.
Add a few more stubs.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@24 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-04 20:06:29 +00:00
behlendo 48f940b943 Fix type
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@23 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-04 19:38:27 +00:00
behlendo 14c5326ccd More stub headers,
moved generic to sysmacros,
added some more macros for kernel compatibility


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@22 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-04 18:22:31 +00:00
behlendo dbb484ec60 Stub out some missing headers which are expected. I'll fill
in what the contents need to be as I encounter the warnings
about missing prototypes, symbols, and such.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@21 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-01 18:30:12 +00:00
behlendo ea70970ff5 Almost dropped this!
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@19 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-01 00:47:23 +00:00
behlendo f4b377415b Reorganize /include/ to add a /sys/, this way we don't need to
muck with #includes in existing Solaris style source to get it
to find the right stuff.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@18 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-01 00:45:59 +00:00
behlendo 09b414e880 Minor nit, SOLARIS should be SPL
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@17 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-28 00:52:31 +00:00
behlendo 596e65b4e8 OK, I think this is the last of major cleanup and restructuring.
We've dropped all the linux- prefixes on the file in favor of spl-
which makes more sense.  And we've cleaned up some of the includes
so everybody should be including their own dependencies properly.
All a module which wants to use the spl support needs to do in
include spl.h and ensure it has access to Module.symvers.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@16 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-28 00:48:31 +00:00
behlendo 7c50328b40 More cleanup.
- Removed all references to kzt and replaced with splat
- Moved portions of include files which do not need to be
  available to all source files in to local.h files in 
  proper source subdirs.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@14 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-27 23:42:31 +00:00
behlendo f1b59d2620 Lots of build fixes. This is turning out to be a very good
idea since it forcefully codifing the ABI.  Since the shim
layer is no longer linked at build time in to the test suite
we can;'t cut any corners and get away with it.

Everything is working now with the exception of sorting
setting Module.symvers properly.  This may take a little
Makefile reorg.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@5 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-27 19:09:51 +00:00
behlendo 3d4ea0ced6 More build fixes, I have the kernel module almost building and its
feeling a lot more sane, cleaner, and linuxy.  I may finish this tonight.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@4 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-27 00:59:48 +00:00
behlendo 564f6d1509 User space build fixes:
- Add list handling compatibility library
- Drop uu_* list handling in favor of local list implementation
- libtoolize
- generic makefile cleanup



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@3 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-26 23:20:41 +00:00
behlendo f1ca4da6f7 Initial commit. All spl source written up to this point wrapped
in an initial reasonable autoconf style build system.  This does
not yet build but the configure system does appear to work properly
and integrate with the kernel.  Hopefully the next commit gets
us back to a buildable version we can run the test suite against.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@1 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-02-26 20:36:04 +00:00