Commit Graph

399 Commits

Author SHA1 Message Date
Brian Behlendorf a9accbcb57 Always open using O_EXCL to ensure the device is not in use.
Allow partition tables on md devices but not dm- devices.
2009-10-27 14:58:12 -07:00
Brian Behlendorf 37a519433c Add additional clarification for Linux systems to error message. 2009-10-26 10:50:43 -07:00
Brian Behlendorf 29c9a2518c Properly handle block devices other the IDE and SCSI disks.
Based on the block device type we can expect a specific naming
convention.  With this in mind update efi_get_info() to be more
aware of the type when parsing out the partition number.  In,
addition be aware that all block device types are not partitionable.
Finally, when attempting to lookup a device partition by appending
the partition number to the whole device take in to account the
kernel naming scheme.  If the last character of the device name
is a digit the partition will always be 'p#' instead of just '#'.
2009-10-23 16:25:16 -07:00
Brian Behlendorf 24f3d6e49e Misc fixed based on testing with the dragon config.
In check_disk() we should only check the entire device if it
not a whole disk.  It is a whole disk with an EFI label on it,
it is possible that libblkid will misidentify the device as a
filesystem.  I had a case yesterday where 2 bytes in the EFI
GUID happened we set to the right values such that libblkid
decided there was a minux filesystem there.  If it's a whole
device we look for a EFI label.

If we are able to read the backup EFI label from a device but
the primary is corrupt.  Then don't bother trying to stat
the partitions in /dev/ the kernel will not create devices
using the backup label when the primary is damaged.

Add code to determine if we have a udev path instead of a
normal device path.  In this case use the -part# partition
naming scheme instead of the /dev/disk# scheme.  This is
important because we always want to access devices using
the full path provided at configuration time.

Readded support for zpool_relabel_disk() now that we have
the full libefi library in place we do have access to this
functionality.

Lots of additional paranoia to ensure EFI label are written
correctly.  These changes include:

1) Removing the O_NDELAY flag when opening a file descriptor
for libefi.  This flag should really only be used when you
do not intend to do any file IO.  Under Solaris only ioctl()'s
were performed under linux we do perform reads and writes.

2) Use O_DIRECT to ensure any caching is bypassed while
writing or reading the EFI labels.  This change forces the
use of sector aligned memory buffers which are allocated
using posix_memalign().

3) Add additional efi_debug error messages to efi_ioctl().

4) While doing a fsync is good to ensure the EFI label is on
disk we can, and should go one step futher by issuing the
BLKFLSBUF ioctl().  This signals the kernel to instruct the
drive to flush it's on-disk cache.

5) Because of some initial strangeness I observed in testing
with some flakey drives be extra paranoid in zpool_label_disk().
After we've written the device without error, flushed the drive
caches, correctly detected the new partitions created by the
kernel.  Then additionally read back the EFI label from user
space to make sure it is intact and correct.  I don't think we
can ever be to careful here.

NOTE: The was recently some concern expressed that writing EFI
labels from user space on Linux was not the right way to do this.
That instead two kernel ioctl()s should be used to create and
remove partitions.  After some investigation it's clear to me
using those ioctl() would be a bad idea.  The in fact don't
actually write partition tables to the disk, they only create
the partition devices in the kernel.  So what you really want
to do is write the label out from user space, then prompt the
kernel to re-read the partition from disk to create the partitions.
This is in fact exactly what newer version of parted do.
2009-10-23 11:57:59 -07:00
Brian Behlendorf aec988734b Command 'zpool create' needs to wait on correct partition names.
When creating partition tables we always need to wait until not
only the /dev/<disk><part> device appears.  But just as importantly
if we were originally given a udev path we need to wait for the
/dev/disk/*/<name>-part<part> symlink to be created.  However,
since the partition naming convention differs between /dev/ and
/dev/disk we determine based on the path which convention to
expect and then wait (for a few seconds) for the device to be
created.  Based on my experience with udev on my test nodes it
takes about 300ms for the devices to be created after being
prompted by the kernel.  This time will vary somehwat based
on how complicated your udev rules are, so for safety I threw
in a factor of 10.  We wait 3 seconds for the devices to appears
before erroring out with a failure.

An additional minor fix includes checking the force flag in the
EFI_GPT_PRIMARY_CORRUPT case.  This allows you to force the
update even in the corrupt partition case.

Finally, since these are Linux only changes I've dropped the
devid code entirely here because I still can't think of why we
would need or want it on a Linux system.
2009-10-21 11:50:42 -07:00
Brian Behlendorf 5be28776fb Always preserve the passed path at creation time so udev may be used
After spending considerable time thinking about this I've come to the
conclusion that on Linux systems we don't need Solaris style devid
support.  Instead was can simply use udev if we are careful, there
are even some advantages.

The Solaris style devid's are designed to provide a mechanism by which
a device can be opened reliably regardless of it's location in the system.
This is exactly what udev provides us on Linux, a flexible mechanism for
consistently identifing the same devices regardless of probing order.
We just need to be careful to always open the device by the path provided
at creation time, this path must be stored in ZPOOL_CONFIG_PATH.  This
in fact has certain advantages.

For example, if in your system you always want the zpool to be able to
locate the disk regardless of physical location you can create the pool
using /dev/disk/by-id/.  This is perhaps what you'ld want on a desktop
system where the exact location is not that important.  It's more
critical that all the disks can be found.

However, in an enterprise setup there's a good chace that the physical
location of each drive is important.  You have like set things up such
that your raid groups span multiple hosts adapters, such that you can
lose an adapter without downtime.  In this case you would want to use
the /dev/disk/by-path/ path to ensure the path information is preserved
and you always open the disks at the right physical locations.  This
would ensure your system never gets accidently misconfigured and still
just works because the zpool was still able to locate the disk.

Finally, if you want to get really fancy you can always create your
own udev rules.  This way you could implement whatever lookup sceme
you wanted in user space for your drives.  This would include nice
cosmetic things like being able to control the device names in tools
like zpool status, since the name as just based of the device names.

I've yet to come up with a good reason to implement devid support on
Linux since we have udev.  But I've still just commented it out for now
because somebody might come up with a really good I forgot.
2009-10-19 13:46:48 -07:00
Brian Behlendorf 48d1b0c5ab Cleanly integrate ZFS tools with libblkid.
The majority of this this patch concerns itself with doing a direct
replacement of Solaris's libdiskmgt library with libblkid+libefi.
You'll notice that this patch removes all libdiskmgt code instead of
ifdef'ing it out.  This was done to minimize any confusion when reading
the code because it seems unlikely we will ever port libdiskmgt to Linux.

Despite the replacement the behavior of the tools should have remained
the same with one exception.  For the moment, we are unable to check
the partitions of devices which have an MBR style partition table when
creating a filesystem.  If a non-efi partition sceme is detected on a
whole disk device we prompt the user to explicity use the force option.
It would not be a ton of work to make the tool aware of MBR style
partitions if this becomes a problem.

I've done basic sanity checking for various configurations and all
the issues I'm aware of have been addressed.  Even things like blkid
misidentifing a disk as ext3 when it is added to a zfs pool.  I'm
careful to always zero out the first 4k of any new zfs partition.  That
all said this is all new code and while it looks like it's working right
for me we should keep an eye on it for any strange behavior.
2009-10-15 16:28:47 -07:00
Brian Behlendorf 992be351d5 Changes required to integrate libefi in to Linux.
The major change here is to fix up libefi to be linux aware.  For
the most part this wasn't too hard but there were a few major issues.

First off I needed to handle the DKIOCGMEDIAINFO and DKIOCINFO ioctls.
There is no direct equivilant for these ioctls under linux.  To handle
this I added wrapper functions which under Solaris simple call the ioctls.
But under Linux dig around the system a little bit getting the needed
info to fill in the requested structures.

Secondly the efi_ioctl() call was adapted such that under linux it directly
read or writes out the partition table.  Under Solaris this work was
handed off to the kernel via an ioctl.  In the efi_write() case we also
ensure we prompt the kernel via BLKRRPART to re-scan the new partition
table.  The libefi generated partition tables are correct but older
versions of ~parted-1.8.1 can not read them without a small patch.
The kernel and fdisk are able to read them just fine.

Thirdly efi_alloc_and_init() which is used by zpool to determine if a
device is a 'wholedisk' was updated to be linux aware.  This check is
performed by using the partition number for the device, which the
partition number is 0 on linux it is a 'wholedisk'.  However, certain
device type such as the loopback and ram disks needed to be excluded
because they do not support partitioning.

Forthly the zpool command was made symlink aware so it can correctly
resolve udev entries such as /dev/disk/by-*/*.  This symlinks are
fully expanded ensuring all block devices are recognized.  When a
when a 'wholedisk' block device is detected we now properly write
out an efi label and place zfs in the first partition (0th slice).
This partition is created 1MiB in to the disk to ensure it is aligned
nicely with all high end block devices I'm aware of.

This all works for me now but it did take quite a bit of work to get
it all sorted out.  It would not surprise me if certain special cases
were missed so we should keep any eye of for any odd behavior.
2009-10-14 16:07:48 -07:00
Brian Behlendorf 245e7692f7 Merge commit 'refs/top-bases/linux-user-disk' into linux-user-disk 2009-10-14 15:58:55 -07:00
Brian Behlendorf 085ed9fe82 Merge commit 'refs/top-bases/zfs-branch' into zfs-branch 2009-10-14 15:57:10 -07:00
Brian Behlendorf cb678c0b31 Merge branch 'gcc-branch' into refs/top-bases/zfs-branch 2009-10-14 15:57:09 -07:00
Brian Behlendorf a8adb61c00 Merge branch 'gcc-ident-pragmas' into refs/top-bases/gcc-branch 2009-10-14 15:57:08 -07:00
Brian Behlendorf 39d85a8b94 Remove pragmas from new libefi files. 2009-10-14 15:44:22 -07:00
Brian Behlendorf 81ec3f8951 Merge commit 'refs/top-bases/linux-user-disk' into linux-user-disk 2009-10-09 16:43:53 -07:00
Brian Behlendorf cb91bbe6ce Additional set of build system tweaks for libefi library. 2009-10-09 16:37:32 -07:00
Brian Behlendorf 7c073d0a7d Merge commit 'refs/top-bases/linux-user-disk' into linux-user-disk 2009-10-09 16:09:54 -07:00
Brian Behlendorf a0337cfcd8 Add libefi library in to the build system. 2009-10-09 15:58:45 -07:00
Brian Behlendorf 503c5eeb2d Merge commit 'refs/top-bases/linux-user-disk' into linux-user-disk 2009-10-09 15:45:22 -07:00
Brian Behlendorf e0e0f0ea84 Merge commit 'refs/top-bases/zfs-branch' into zfs-branch 2009-10-09 15:43:33 -07:00
Brian Behlendorf 9302f90662 Merge branch 'gcc-branch' into refs/top-bases/zfs-branch 2009-10-09 15:42:56 -07:00
Brian Behlendorf 926a2e3cc4 Merge commit 'refs/top-bases/gcc-ident-pragmas' into gcc-ident-pragmas 2009-10-09 15:42:48 -07:00
Brian Behlendorf 3728b6d9c1 Merge branch 'gcc-c90' into refs/top-bases/gcc-branch 2009-10-09 15:42:16 -07:00
Brian Behlendorf 1b0e5a1485 Merge commit 'refs/top-bases/gcc-c90' into gcc-c90 2009-10-09 15:42:15 -07:00
Brian Behlendorf 5c36312909 Script update-zfs.sh updated to include libefi library 2009-10-09 15:37:29 -07:00
Brian Behlendorf 060284b353 Merge commit 'refs/top-bases/linux-user-disk' into linux-user-disk 2009-08-18 14:21:35 -07:00
Brian Behlendorf e4b55cfbff Merge commit 'refs/top-bases/zfs-branch' into zfs-branch 2009-08-18 14:18:36 -07:00
Brian Behlendorf 5d83878f46 Merge branch 'gcc-branch' into refs/top-bases/zfs-branch 2009-08-18 14:18:34 -07:00
Brian Behlendorf e134b976b6 Merge branch 'gcc-missing-braces' into refs/top-bases/gcc-branch 2009-08-18 14:18:33 -07:00
Brian Behlendorf b29c1f87a0 Required build 121 updates for gcc-missing-braces 2009-08-18 14:08:46 -07:00
Brian Behlendorf 8b02e1a43f Merge commit 'refs/top-bases/linux-user-disk' into linux-user-disk 2009-08-18 12:35:12 -07:00
Brian Behlendorf 50c623ccef Merge commit 'refs/top-bases/zfs-branch' into zfs-branch 2009-08-18 12:10:55 -07:00
Brian Behlendorf b99b422796 Merge branch 'gcc-branch' into refs/top-bases/zfs-branch 2009-08-18 12:08:58 -07:00
Brian Behlendorf 41b297a27f Merge commit 'refs/top-bases/gcc-missing-braces' into gcc-missing-braces 2009-08-18 12:08:46 -07:00
Brian Behlendorf abd8fdb94f Merge commit 'refs/top-bases/gcc-ident-pragmas' into gcc-ident-pragmas
Conflicts:

	module/zfs/dsl_deleg.c
	module/zfs/rrwlock.c
2009-08-18 12:08:36 -07:00
Brian Behlendorf e5e4fadc34 Merge branch 'gcc-c90' into refs/top-bases/gcc-branch
Conflicts:

	module/zfs/dmu.c
	module/zfs/dsl_deleg.c
	module/zfs/rrwlock.c
	module/zfs/vdev_raidz.c
2009-08-18 12:04:53 -07:00
Brian Behlendorf 0f80e41c89 Merge commit 'refs/top-bases/gcc-c90' into gcc-c90
Conflicts:

	module/zfs/spa_misc.c
	module/zfs/vdev_label.c
2009-08-18 11:55:36 -07:00
Brian Behlendorf 45d1cae3b8 Rebase master to b121 2009-08-18 11:43:27 -07:00
Brian Behlendorf 4eae170dc5 Merge commit 'refs/top-bases/linux-user-disk' into linux-user-disk 2009-08-04 15:57:44 -07:00
Brian Behlendorf f88cbd6535 Merge commit 'refs/top-bases/zfs-branch' into zfs-branch 2009-08-04 15:54:21 -07:00
Brian Behlendorf 8c28ed9bff Merge branch 'feature-branch' into refs/top-bases/zfs-branch 2009-08-04 15:54:19 -07:00
Brian Behlendorf 7069d048a0 Revert kthreads based on pthread thread specific data due to failure.
With this patch applied I get the following failure 100% of the time,
I'd prefer to debug it and keep moving forward but I do not have the
time right now so I'm reverting the patch to the version which worked.
Ricardo please fix.

(gdb) bt
0  ztest_dmu_write_parallel (za=0x2aaaac898960) at
../../cmd/ztest/ztest.c:2566
1  0x0000000000405a79 in ztest_thread (arg=<value optimized out>)
    at ../../cmd/ztest/ztest.c:3862
2  0x00002b2e6a7a841d in zk_thread_helper (arg=<value optimized out>)
    at ../../lib/libzpool/kernel.c:131
3  0x000000379be06367 in start_thread (arg=<value optimized out>)
    at pthread_create.c:297
4  0x000000379b2d30ad in clone () from /lib64/libc.so.6
2009-08-04 15:42:24 -07:00
Brian Behlendorf 94fd6b5326 Merge commit 'refs/top-bases/linux-user-disk' into linux-user-disk 2009-08-04 14:42:33 -07:00
Brian Behlendorf 3e5388907b Merge commit 'refs/top-bases/zfs-branch' into zfs-branch 2009-08-04 14:39:07 -07:00
Brian Behlendorf 9aedfd95f6 Merge branch 'feature-branch' into refs/top-bases/zfs-branch 2009-08-04 14:39:06 -07:00
Ricardo M. Correia c0ce3b40f2 Reimplement user kthreads based on pthread thread specific data.
This resolves previous scalabily concerns about the cost of calling
curthread which previously required a list walk.  The kthread address
is now tracked as thread specific data which can be quickly returned.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2009-08-04 13:55:15 -07:00
Brian Behlendorf 21dee98331 Merge commit 'refs/top-bases/linux-user-disk' into linux-user-disk 2009-07-31 14:05:42 -07:00
Brian Behlendorf 9f5e45ef4d Merge commit 'refs/top-bases/zfs-branch' into zfs-branch 2009-07-31 14:02:49 -07:00
Brian Behlendorf cf44f97b96 Merge branch 'gcc-branch' into refs/top-bases/zfs-branch 2009-07-31 14:02:48 -07:00
Brian Behlendorf 252846a145 Merge branch 'gcc-shadow' into refs/top-bases/gcc-branch 2009-07-31 14:02:46 -07:00
Brian Behlendorf cfd9e9936f Fix an accidentally introduced formatting issue in zfs output.
Because the local 'index' variable shadows the index() function
it was replaced by 'i'.  Unfortunately when I made this change
I accidentally replaced one instance with 'j' resulting in the
short decimal values being printed incorrectly.
2009-07-31 13:50:21 -07:00