Commit Graph

5697 Commits

Author SHA1 Message Date
behlendo 13cdca65ec Add vmem memory accounting
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@99 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-07 18:54:32 +00:00
behlendo 404992e31a - Relocate 'stats_per' in to proper /proc/sys/spl/mutex/ directory
- Shift to spinlock for mutex list addition and removal



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@98 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-07 17:58:22 +00:00
behlendo 4f86a887d8 Remaining issues fixed after reenabled mutex debugging.
- Ensure the mutex_stats_sem and mutex_stats_list are initialized
- Only spin if you have to in mutex_init



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@97 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-06 23:19:27 +00:00
behlendo e8b31e8482 - Updated rwlock's to reside in a .c file instead of a static inline
- Updated rwlock's so they can be safely initialized in ctors.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@96 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-06 23:00:49 +00:00
behlendo d6a26c6a32 Lots of fixes here:
- Detailed kmem memory allocation tracking.  We can now get on
  spl module unload a list of all memory allocations which were
  not free'd and where the original alloc was.  E.g.

SPL: 15554:632:(spl-kmem.c:442:kmem_fini()) kmem leaked 90/319332 bytes
SPL: 15554:648:(spl-kmem.c:451:kmem_fini()) address          size  data             func:line
SPL: 15554:648:(spl-kmem.c:457:kmem_fini()) ffff8100734b68b8 32    0100000001005a5a __spl_mutex_init:70
SPL: 15554:648:(spl-kmem.c:457:kmem_fini()) ffff8100734b6148 13    &tl->tl_lock     __spl_mutex_init:74
SPL: 15554:648:(spl-kmem.c:457:kmem_fini()) ffff81007ac43730 32    0100000001005a5a __spl_mutex_init:70
SPL: 15554:648:(spl-kmem.c:457:kmem_fini()) ffff81007ac437d8 13    &tl->tl_lock     __spl_mutex_init:74

- Shift to using rwsems in kmem implmentation, to simply locking and
  improve concurency.

- Shift to using rwsems in mutex implementation, additionally ensure we
  never sleep in the init function if non-zero preempt_count or 
  interrupts are disabled as can happen in a slab cache ctor/dtor.

- Other minor formating fixes and such.

TODO:

- Finish the vmem memory allocation tracking

- Vet all other SPL primatives for potential sleeping during *_init.  I
suspect the rwlock implemenation does this and should be fixes just
like the mutex implemenation.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@95 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-06 20:38:28 +00:00
behlendo 9ab1ac14ad Commit adaptive mutexes. This seems to have introduced some new
crashes but it's not clear to me yet if these are a problem with
the mutex implementation or ZFSs usage of it.

Minor taskq fixes to add new tasks to the end of the pending list.

Minor enhansements to the debug infrastructure.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@94 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-05-05 20:18:49 +00:00
behlendo bcd68186d8 New an improved taskq implementation for the SPL. It allows a
configurable number of threads like the Solaris version and almost
all of the options are supported.  Unfortunately, it appears to have
made absolutely no difference to our performance numbers.  I need
to keep looking for where we are bottle necking.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@93 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-25 22:10:47 +00:00
behlendo 839d8b438e Update kmem.h to properly use new debug subsystem.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@92 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-24 20:21:07 +00:00
behlendo 3561541c24 Prep for 0.2.1 tag
Minor fixes to headers to use debug macros
Added /proc/sys/spl/version



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@90 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-24 17:41:23 +00:00
wartens2 1bac409fa3 Forgot to update the ChangeLog.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@89 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-24 17:24:02 +00:00
wartens2 8100fe56f1 Make sure that when calling __vmem_alloc that we
do not have __GFP_ZERO set.  Once the memory is allocated
then zero out the memory if __GFP_ZERO is passed to
__vmem_alloc.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@88 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-24 17:07:56 +00:00
behlendo 6e605b6e58 Minor improvement to taskq handling. This is a small step towards
dynamic taskqs which still need to be fully implemented.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@87 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-23 21:19:47 +00:00
behlendo 18c9eadf97 Be careful to never use any of the debug infrastructure either
before the debug subsystem is fully set up, or after the debug
subsystem has been torn down.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@86 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-22 22:22:02 +00:00
behlendo 7e4e211333 Give it a real version for a tag
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@84 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-22 20:10:55 +00:00
behlendo b831734a43 Stack usage is my enemy. Trade cpu cycles in the debug code to
ensure I never add anything to the stack I don't absolutely need.
All this debug code could be removed from a production build
anyway so I'm not so worried about the performance impact.  We
may also consider revisting the mutex and condvar implementation
to ensure no additional stack is used there.

Initial indications are I have reduced the worst case stack
usage to 9080 bytes.  Still to large for the default 8k stacks
so I have been forced to run with 16k stacks until I can
reduce the worst offenders.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@83 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-22 16:55:26 +00:00
behlendo 7fea96c04f More fixes to ensure we get good debug logs even if we're in the
process of destroying the stacks.  Threshhold set fairly aggressively
top 80% of stack usage.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@82 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-21 22:44:11 +00:00
behlendo e5bbd245e3 Added 4 missing subsystem flags
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@81 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-21 18:43:02 +00:00
behlendo a8ac0b8966 Whoops, missed an instance where we could recursively stack check... bad.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@80 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-21 18:16:04 +00:00
behlendo 892d51061e Handful of minor stack checking fixes
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@79 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-21 18:08:33 +00:00
behlendo 937879f11d Update SPL to use new debug infrastructure. This means:
- Replacing all BUG_ON()'s with proper ASSERT()'s
- Using ENTRY,EXIT,GOTO, and RETURN macro to instument call paths



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@78 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-21 17:29:47 +00:00
behlendo 2fae1b3d0a Frist minor batch of fixes. Catch a dropped ;, and use SBUG instead of BUG.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@77 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-19 00:02:11 +00:00
behlendo ce86265693 Whoops need this!
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@76 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-18 23:42:45 +00:00
behlendo 57d1b18858 First commit of lustre style internal debug support. These
changes bring over everything lustre had for debugging with
two exceptions.  I dropped by the debug daemon and upcalls
just because it made things a little easier.  They can be
readded easily enough if we feel they are needed.

Everything compiles and seems to work on first inspection
but I suspect there are a handful of issues still lingering
which I'll be sorting out right away.  I just wanted to get
all these changes commited and safe.  I'm getting a little
paranoid about losing them.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@75 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-18 23:39:58 +00:00
wartens2 55152ebbb4 * modules/spl/spl-kmem.c : Make sure to disable interrupts
when necessary to avoid deadlocks.  We were seeing the deadlock
        when calling kmem_cache_generic_constructor() and then an interrupt
        forced us to end up calling kmem_cache_generic_destructor()
        which caused our deadlock.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@74 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-16 16:37:51 +00:00
behlendo d61e12af5a - Add some spinlocks to cover all the private data in the mutex. I don't
think this should fix anything but it's a good idea regardless.

- Drop the lock before calling the construct/destructor for the slab
otherwise we can't sleep in a constructor/destructor and for long running
functions we may NMI.

- Do something braindead, but safe for the console debug logs for now.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@73 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-15 20:53:36 +00:00
behlendo c5fd77fcbf Just cleanup up an error case to avoid overspamming the console.
We get the stack once from the BUG() no reason to dump it twice.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@72 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-14 18:37:20 +00:00
behlendo f23e92fabf Add hw_serial support based on a usermodehelper which runs
at spl module load time can calls hostid.  The resolved hostid
is then fed back in to a proc entry for latter use.  It's
not a pretty thing, but it will work for now.  The hw_serial
is required for things such as 'zpool status' to work.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@71 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-12 04:27:59 +00:00
behlendo 12ea923056 Adjust the condition variables to simply sleep uninteruptibly.
This way we don't have to contend with superious wakeups which
it appears ZFS is not so careful to handle anyway.  So this is
probably for the best.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@70 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-11 22:49:48 +00:00
behlendo 115aed0dd8 - Add more strict in_atomic() checking to the mutex entry
function just to be extra safety and paranoid.

- Rewrite the thread shim to take full advantage of the
new kernel kthread API.  This greatly simplifies things.

- Add a new regression test for thread_exit() to ensure
it properly terminates a thread immediately without
allowing futher execution of the thread.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@69 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-11 17:03:57 +00:00
behlendo 79f92663e3 Fix race in rwlock implementation which can occur when
your task is rescheduled to a different cpu after you've
taken the lock but before calling RW_LOCK_HELD is called.
We need the spinlock to ensure there is a wmb() there.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@68 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-07 23:54:34 +00:00
behlendo 728b9dd800 - Fix write-only behavior in vn-open()
- Ensure we have at least 1 write-only splat test
- Fix return codes for vn_* Solaris does not use negative return 
  codes in the kernel.  So linux errno's must be inverted.




git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@67 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-04 17:08:12 +00:00
behlendo 968eccd1d1 Update the thread shim to use the current kernel threading API.
We need to use kthread_create() here for a few reasons.  First
off to old kernel_thread() API functioin will be going away.
Secondly, and more importantly if I use kthread_create() we can
then properly implement a thread_exit() function which terminates
the kernel thread at any point with do_exit().  This fixes our
cleanup bug which was caused by dropping a mutex twice after
thread_exit() didn't really exit.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@66 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-04 04:44:16 +00:00
behlendo 996faa6869 Correctly implement atomic_cas_ptr() function. Ideally all of these
atomic operations will be rewritten anyway with the correct arch
specific assembly.  But not today.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@65 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-03 21:48:57 +00:00
behlendo 0a6fd143fd - Remapped ldi_handle_t to struct block_device * which is much more useful
- Added liunx block device headers to sunldi.h
- Made __taskq_dispatch safe for interrupt context where it turns out we
  need to be useing it.
- Fixed NULL const/dest bug for kmem slab caches
- Places debug __dprintf debugging messages under a spin_lock_irqsave
  so it's safe to use then in interrupt handlers.  For debugging only!



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@64 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-03 16:33:31 +00:00
behlendo 0998fdd6db Apparently it's OK for done to be NULL, which was not clear in the
Solaris man page.  Anyway, since apparently this usage is accectable
I've updated the function to handle it.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@63 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-01 17:00:06 +00:00
behlendo 6a585c61de Double large kmalloc warning size to 4 pages. It was 2 pages, and ideally
it should be dropped to one page but in the short term we should be able
to easily live with 4 page allocations.

Fix the nvlist bug, it turns out the user space side of things were
packing the nvlists correctly as little endian, and the kernel space
side of things due to a missing #define were unpacking them as big endian.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@62 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-01 16:09:18 +00:00
behlendo e966e04fd5 Ensure all file ops pointer are NULL or we may end up
calling garbage pointers on open/close etc and get
what look like random crashes.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@61 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-04-01 03:24:17 +00:00
behlendo 4fd2f7eea5 Add vmem_zalloc support.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@60 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-31 23:04:07 +00:00
behlendo 8d0f1ee907 Add some crude debugging support. It leaves alot to be
desired, but it should allow more easy kernel debugging for now.


git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@59 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-31 20:42:36 +00:00
behlendo e487ee08fb Fixed that.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@58 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-28 18:22:29 +00:00
behlendo 9f4c835a0e Correctly functioning 64-bit atomic shim layer. It's not
what I would call effecient but it does have the advantage
of being correct which is all I need right now.  I added
a regression test as well.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@57 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-28 18:21:09 +00:00
behlendo 4a4295b267 Remove minor lingering CDDL tait of copied headers. Required
headers rewritten to include minimally what we need.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@56 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-27 23:40:09 +00:00
behlendo d429b03d85 - Thinko fix to the SPL module interface
- Enhanse the VERIFY() support to output the values which
  failed to compare as expected before crashing.  This make
  debugging much much much easier.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@55 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-27 22:06:59 +00:00
behlendo 8ac547ec4c Relocated to zfs repo
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@54 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-27 17:58:10 +00:00
behlendo 336bb0c0c1 Two fixes to the module interface. Could be worse!
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@53 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-21 19:16:25 +00:00
behlendo 4e62fd4104 OK, a first reasonable attempt at a solaris module/chdev shim layer.
This should handle the absolute minimum I need for ZFS.  It will 
register the chdev with the right callbacks.  Then the generic 
registered linux callback will find the right registered solaris
callback for the function and munge the args just right before
passing it on.  Should work, but untested (just compiled), so I
expect bugs.



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@52 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-20 23:30:15 +00:00
behlendo e4f1d29f89 OK, some pretty substantial rework here. I've merged the spl-file
stuff which only inclused the getf()/releasef() in to the vnode area
where it will only really be used.  These calls allow a user to
grab an open file struct given only the known open fd for a particular
user context.  ZFS makes use of these, but they're a bit tricky to
test from within the kernel since you already need the file open
and know the fd.  So we basically spook the system calls to setup
the environment we need for the splat test case and verify given
just the know fd we can get the file, create the needed vnode, and
then use the vnode interface as usual to read and write from it.

While I was hacking away I also noticed a NULL termination issue
in the second kobj test case so I fixed that too.  In fact, I fixed
a few other things as well but all for the best!



git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@51 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-18 23:20:30 +00:00
behlendo 5d86345d37 Initial pass at a file API getf/releasef hooks
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@50 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-18 04:56:43 +00:00
behlendo 1ec74a114c Minimal signal handling interface.
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@49 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-17 18:29:57 +00:00
behlendo 2bdb28fbe0 Missing headers, more minor fixes
git-svn-id: https://outreach.scidac.gov/svn/spl/trunk@48 7e1ea52c-4ff2-0310-8f11-9dd32ca42a1c
2008-03-15 00:05:38 +00:00