Commit Graph

335 Commits

Author SHA1 Message Date
Tim Chase 2e2ddc30b4 Dedup-related documentation additions for zpool and zdb.
Document the "-D" and "-T" options and the optional interval
and count or "zpool status".

Also for zpool's man page, use a consistent order for the
various "-T" options to match the program's help output.

Document the effect of additional "-D" options for zdb.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1786
2013-10-22 10:08:51 -07:00
Richard Yao 31fc19399e Generate libraries with correct DT_NEEDED entries
Libraries that depend on other libraries should list them in ELF's
DT_NEEDED field so that programs linking to them do not need to specify
those libraries unless they depend on them as well. This is not the case
in the current code and the consequence is that anything that needs a
library must know its dependencies. This is fragile and caused GRUB2's
configure script to break when a dependency was added on libblkid in
libzfs.

This resolves that problem by using LIBADD/LDADD to specify libraries in
Makefile.am instead of LDFLAGS. This ensures that proper DT_NEEDED
entries are generated and prevents GRUB2's configure script from
breaking in the presence of a libblkid dependency. This also removes
unneeded dependencies from various files.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1751
2013-10-10 16:56:51 -07:00
Richard Yao 3549721c9e Update drive database
Add Corsair Force GS drive (obtained from drive_id)
Add Kingston HyperX 3K (obtained from drive_id)
Add OCZ Vertex 4 drive (obtained from drive_id)
Add Samsung SM843T enterprise drive (obtained from drive_id)
Add entries for additional sizes of Intel 320/330/335/520 series
Add Cruical C400 (obtained from Illumos user's sd.conf)
Add Toshiba SSD (obtained from Illumos user's sd.conf)
Add Samsung's first SLC SSD (obtained from drive_id)
Add OCZ Core Series (obtained from drive_id)
Add Intel DC S3700 (obtained from drive_id)

Notes:

1. The drive identifer obtained for the Samsung SM843T was MZ7WD480. The
rest were extrapolated. The additional entries were checked with Google
to verify that such drives exist in the wild.

2. The additional entries for Intel drives were extrapolated from
existing entries. The additional entries were checked with Google to
verify that such drives exist in the wild.

3. The "ATA     C400-MTFDDAC512M" and "ATA     TOSHIBA THNSNH51" entries
are from the sd.conf of gcbirzan on freenode. Additional entries were
extrapolated from them and checked with Google.

4. I obtained the Samsung MCCOE64G entry from an actual drive. The
Samsung MCCOE32G entry was extrapolated from it and checked with
Google.

5. I obtained the SSDSC2BA10 from a 100GB Intel DC S3700 drive and
extrapolated the entries for the additional models.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1752
2013-10-09 09:16:23 -07:00
Matthew Ahrens 13fe019870 Illumos #3464
3464 zfs synctask code needs restructuring
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Approved by: Garrett D'Amore <garrett@damore.org>

References:
  https://www.illumos.org/issues/3464
  illumos/illumos-gate@3b2aab1880

Ported-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1495
2013-09-04 16:01:24 -07:00
Matthew Ahrens 6f1ffb0665 Illumos #2882, #2883, #2900
2882 implement libzfs_core
2883 changing "canmount" property to "on" should not always remount dataset
2900 "zfs snapshot" should be able to create multiple, arbitrary snapshots at once

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Chris Siden <christopher.siden@delphix.com>
Reviewed by: Garrett D'Amore <garrett@damore.org>
Reviewed by: Bill Pijewski <wdp@joyent.com>
Reviewed by: Dan Kruchinin <dan.kruchinin@gmail.com>
Approved by: Eric Schrock <Eric.Schrock@delphix.com>

References:
  https://www.illumos.org/issues/2882
  https://www.illumos.org/issues/2883
  https://www.illumos.org/issues/2900
  illumos/illumos-gate@4445fffbbb

Ported-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1293

Porting notes:

WARNING: This patch changes the user/kernel ABI.  That means that
the zfs/zpool utilities built from master are NOT compatible with
the 0.6.2 kernel modules.  Ensure you load the matching kernel
modules from master after updating the utilities.  Otherwise the
zfs/zpool commands will be unable to interact with your pool and
you will see errors similar to the following:

  $ zpool list
  failed to read pool configuration: bad address
  no pools available

  $ zfs list
  no datasets available

Add zvol minor device creation to the new zfs_snapshot_nvl function.

Remove the logging of the "release" operation in
dsl_dataset_user_release_sync().  The logging caused a null dereference
because ds->ds_dir is zeroed in dsl_dataset_destroy_sync() and the
logging functions try to get the ds name via the dsl_dataset_name()
function. I've got no idea why this particular code would have worked
in Illumos.  This code has subsequently been completely reworked in
Illumos commit 3b2aab1 (3464 zfs synctask code needs restructuring).

Squash some "may be used uninitialized" warning/erorrs.

Fix some printf format warnings for %lld and %llu.

Apply a few spa_writeable() changes that were made to Illumos in
illumos/illumos-gate.git@cd1c8b8 as part of the 3112, 3113, 3114 and
3115 fixes.

Add a missing call to fnvlist_free(nvl) in log_internal() that was added
in Illumos to fix issue 3085 but couldn't be ported to ZoL at the time
(zfsonlinux/zfs@9e11c73) because it depended on future work.
2013-09-04 15:49:00 -07:00
Richard Yao bff32e0972 Implement database to workaround misreported physical sector sizes
This implements vdev_bdev_database_check(). It alters the detected
sector size of any device listed in a database of drives known to lie
about their physical sector sizes.

This is based on "6931570 Add flash devices' VID/PID to disk table to
advertising 4K physical sector size" from Open Solaris and on
sg_simple4.c from sg3_utils. About two dozen lines are taken from
sg_simple4.c, which is GPLv2 licensed. However, sg_simple4.c is
analogous to a Hello World program and is safe for us to use. We
requested that Douglas Gilbert, the author of sg_simple4.c, confirm that
this is the case. A cutdown version of his response is as follows:

```
I would consider a SCSI INQUIRY example using the Linux sg
driver interface (also written by me) as the equivalent of an
"hello world" program in C.
```

The database was created with the help of the freenode and ZFSOnLinux
communities.

Some notes:

1. The following drives both were confirmed to lie via reports in IRC
and they contain capacity information in their identifiers:

INTEL SSDSA2M080
INTEL SSDSA2M160
M4-CT256M4SSD2
WDC WD15EARS-00S
WDC WD15EARS-00Z
WDC WD20EARS-00M

The identifiers for different capacity models were extrapolated and
added under the assumption that those models also lie. Google was used
to verify that the extrapolated drive identifiers existed prior to their
inclusion.

2. The OCZ-VERTEX2 3.5 identifer applies to two drives that differ
solely in page size (and slightly in capacity). One uses 4096-byte pages
and the other uses 8192-byte pages. Both are set to use 8192-byte pages.
We could detect the page size by checking the capacity, but that would
unnecessarily complicate the code.

3. It is possible for updated drive firmware to correctly report the
sector size. There were reports of a few advanced format drives doing
that. One report stated that the vendor changed the identification
string while another was unclear on this. Both reports involved WDC
models.

4. Google was used to determine the size of pages in the listed flash
devices. Reports of 8192-byte pages took precedence over reports of
4096-byte pages.

5. Devices behind USB adapters can have their identification strings
altered. Identification strings obtained across USB adapters are
omitted and no attempt is made to correct for alterations made by USB
adapters when doing comparisons against the database. Two entries in the
Open Solaris database that appear to have been altered by a USB
adapter were omitted.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1652
2013-08-22 11:24:06 -07:00
Turbo Fredriksson 0bc7a7a754 Don't specifically open /etc/mtab - it is done in libzfs_init()
a few lines further down and we can share the open file handle.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1498
2013-08-15 10:19:38 -07:00
Turbo Fredriksson f9e459d143 Use setmntent() OR fopen()
For the same reasons it's used in libzfs_init(), this was just
overlooked because zinject gets minimal use.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1498
2013-08-15 10:09:09 -07:00
Yuri Pankov 105afebb15 Illumos #3098 zfs userspace/groupspace fail
3098 zfs userspace/groupspace fail without saying why when run as non-root
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/3098
  illumos/illumos-gate@70f56fa693

Ported-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1596
2013-08-14 09:28:34 -07:00
Brian Behlendorf cd72af9c68 Fix 'zpool list -H' error code
Due to an uninitialized variable it was possible for the command
'zpool list -H' to return a non-zero error when there are no pools.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1605
2013-07-23 12:39:05 -07:00
Christer Ekholm da91c90154 Add missing -v to usage help for zpool list.
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-07-22 13:39:01 -07:00
Dmitry Khasanov 131cc95ca7 Add FreeBSD 'zpool labelclear' command
The FreeBSD implementation of zfs adds the 'zpool labelclear'
command.  Since this functionality is helpful and straight
forward to add it is being included in ZoL.

References:
  freebsd/freebsd@119a041dc9

Ported-by: Dmitry Khasanov <pik4ez@gmail.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1126
2013-07-09 15:58:05 -07:00
Tim Chase 5021058756 zdb: enhancement - Display SA xattrs.
If the znode has SA xattrs, display them following the other
standard attributes.  The format used is similar to that used
when listing the contents of a ZAP.  It is as follows:

	$ zdb -vvv <pool>/<dataset> <object>
	...
	SA xattrs: <size> bytes, <number> entries
		<name1> = <value1>
		<name2> = <value2>
		...

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1581
2013-07-09 13:52:28 -07:00
Mike Leddy 5d3dc3fb72 Avoid abort() in vn_rdwr(): libzpool/kernel.c
Make sure that buffer is aligned to 512 bytes on linux so that
pread call combined with O_DIRECT does not return EINVAL.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1570
2013-07-09 11:56:43 -07:00
Craig Loomis 50fe577d1f Explicitly flush output at end of each zevent
For "zpool events -f" flush stdout to ensure the last zevent
is always printed immediately.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1568
2013-07-08 17:01:16 -07:00
Brian Behlendorf c76955eaa5 Fix parse_dataset error handling
A mount failure was accidentally introduced by commit 0c1171d
which reworked the parse_dataset() function to read pool names
from devices.  The error case where a label is read from the
device but the pool name/value pair doesn't exist was not
handled properly.  In this case we should fall back to the
previous behavior.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1560
2013-07-03 09:20:52 -07:00
George Wilson 294f68063b Illumos #3498 panic in arc_read()
3498 panic in arc_read(): !refcount_is_zero(&pbuf->b_hdr->b_refcnt)
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  illumos/illumos-gate@1b912ec710
  https://www.illumos.org/issues/3498

Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1249
2013-07-02 13:34:31 -07:00
Richard Yao 3db3ff4a78 Use MAXPATHLEN instead of sizeof in snprintf
This silences a GCC 4.8.0 warning by fixing a programming error
caught by static analysis:

../../cmd/ztest/ztest.c: In function ‘ztest_vdev_aux_add_remove’:
../../cmd/ztest/ztest.c:2584:33: error: argument to ‘sizeof’
  in ‘snprintf’ call is the same expression as the destination;
  did you mean to provide an explicit length?
  [-Werror=sizeof-pointer-memaccess]
  (void) snprintf(path, sizeof (path), ztest_aux_template,
                                 ^

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1480
2013-07-02 10:39:24 -07:00
Cyril Plisko 64d7b6cf75 Override default SPA config location via environment
When using zdb with non-default SPA config file it is not convenient
to add -U <non-default-config-file-path> all the time. This commit
introduces support for setting/overriding SPA config location via
environment variable 'SPA_CONFIG_PATH'.

If -U flag is specified in the command line it will override any other
value as usual.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1545
2013-07-01 13:25:44 -07:00
Cyril Plisko 20c17b96c9 Add absent \n at the end of the help text line
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1545
2013-06-28 11:26:41 -07:00
Brian Behlendorf 0c1171dcb5 Allow fetching the pool from the device at mount
To simplify integration with the xfstests test suite the
mount.zfs helper has been extended.  When passed a block
device (/dev/sdX) to mount, instead of a pool/dataset,
the pool name will be read from any existing zfs label
and used.  This allows you to mount the root dataset of
a zfs filesystem by specifing any of the member vdevs.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-06-26 15:20:09 -07:00
George Wilson e51be06697 Illumos #3552, #3564
3552 condensing one space map burns 3 seconds of CPU in spa_sync() thread
3564 spa_sync() spends 5-10% of its time in metaslab_sync() (when not condensing)
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  illumos/illumos-gate@16a4a80742
  https://www.illumos.org/issues/3552
  https://www.illumos.org/issues/3564

Ported-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1513
2013-06-19 16:22:39 -07:00
Madhav Suresh c99c90015e Illumos #3006
3006 VERIFY[S,U,P] and ASSERT[S,U,P] frequently check if first
     argument is zero

Reviewed by Matt Ahrens <matthew.ahrens@delphix.com>
Reviewed by George Wilson <george.wilson@delphix.com>
Approved by Eric Schrock <eric.schrock@delphix.com>

References:
  illumos/illumos-gate@fb09f5aad4
  https://illumos.org/issues/3006

Requires:
  zfsonlinux/spl@1c6d149feb

Ported-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1509
2013-06-19 15:14:10 -07:00
Christ Schlacta fb02fabf9b Modified arcstat.py to run on linux
* Modified kstat_update() to read arcstats from proc.
* Fix shebang.
* Added Makefile.am entries for arcstat.py

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1506
2013-06-18 15:43:15 -07:00
Christ Schlacta 7634cd54db Added arcstat.py from FreeNAS
Original source:
  http://support.freenas.org/browser/nanobsd/Files/usr/local/bin/arcstat.py

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1506
2013-06-18 15:30:08 -07:00
Ned Bass da29fe63f0 Don't leak mount flags into kernel
When calling mount(), care must be taken to avoid passing in flags
that are used only by the user space utilities.  Otherwise we may
stomp on flags that are reserved for other purposes in the kernel.

In particular, openSUSE 12.3 kernels have added a new MS_RICHACL
super-block flag whose value conflicts with our MS_COMMENT flag. This
causes incorrect behavior such as the umask being ignored.  The
MS_COMMENT flag essentially serves as a placeholder in the option_map
data structure of zfs_mount.c, but its value is never used. Therefore
we can avoid the conflict by defining it to 0.

The MS_USERS, MS_OWNER, and MS_GROUP flags also conflict with reserved
flags in the kernel. While this is not known to have caused any
problems, it is nevertheless incorrect.  For the purposes of the
mount.zfs helper, the "users", "owner", and "group" options just serve
as hints to set additional implied options.  Therefore we now define
their associated mount flags in terms of the options that they imply
rather than giving them unique values.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1457
2013-06-18 15:30:08 -07:00
George Wilson 5853fe790d Illumos #3306, #3321
3306 zdb should be able to issue reads in parallel
3321 'zpool reopen' command should be documented in the man
     page and help

Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Matt Ahrens <matthew.ahrens@delphix.com>
Reviewed by: Christopher Siden <chris.siden@delphix.com>
Approved by: Garrett D'Amore <garrett@damore.org>

References:
  illumos/illumos-gate@31d7e8fa33
  https://www.illumos.org/issues/3306
  https://www.illumos.org/issues/3321

The vdev_file.c implementation in this patch diverges significantly
from the upstream version.  For consistenty with the vdev_disk.c
code the upstream version leverages the Illumos bio interfaces.
This makes sense for Illumos but not for ZoL for two reasons.

1) The vdev_disk.c code in ZoL has been rewritten to use the
   Linux block device interfaces which differ significantly
   from those in Illumos.  Therefore, updating the vdev_file.c
   to use the Illumos interfaces doesn't get you consistency
   with vdev_disk.c.

2) Using the upstream patch as is would requiring implementing
   compatibility code for those Solaris block device interfaces
   in user and kernel space.  That additional complexity could
   lead to confusion and doesn't buy us anything.

For these reasons I've opted to simply move the existing vn_rdwr()
as is in to the taskq function.  This has the advantage of being
low risk and easy to understand.  Moving the vn_rdwr() function
in to its own taskq thread also neatly avoids the possibility of
a stack overflow.

Finally, because of the additional work which is being handled by
the free taskq the number of threads has been increased.  The
thread count under Illumos defaults to 100 but was decreased to 2
in commit 08d08e due to contention.  We increase it to 8 until
the contention can be address by porting Illumos #3581.

Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1354
2013-05-03 16:53:52 -07:00
Brian Behlendorf 937210a54b Fix zinject list handlers
The zfs_fd must be opened before calling print_all_handlers() or
the ioctl() cannot be used to the zfs control device.  This brings
the zinject code back in sync with the Illumos implementation.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-05-01 17:05:58 -07:00
George.Wilson cc92e9d0c3 3246 ZFS I/O deadman thread
Reviewed by: Matt Ahrens <matthew.ahrens@delphix.com>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: Christopher Siden <chris.siden@delphix.com>
Approved by: Garrett D'Amore <garrett@damore.org>

NOTES: This patch has been reworked from the original in the
following ways to accomidate Linux ZFS implementation

*) Usage of the cyclic interface was replaced by the delayed taskq
   interface.  This avoids the need to implement new compatibility
   code and allows us to rely on the existing taskq implementation.

*) An extern for zfs_txg_synctime_ms was added to sys/dsl_pool.h
   because declaring externs in source files as was done in the
   original patch is just plain wrong.

*) Instead of panicing the system when the deadman triggers a
   zevent describing the blocked vdev and the first pending I/O
   is posted.  If the panic behavior is desired Linux provides
   other generic methods to panic the system when threads are
   observed to hang.

*) For reference, to delay zios by 30 seconds for testing you can
   use zinject as follows: 'zinject -d <vdev> -D30 <pool>'

References:
  illumos/illumos-gate@283b84606b
  https://www.illumos.org/issues/3246

Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1396
2013-05-01 17:05:52 -07:00
Chris Dunlop 254255f735 Trivial spelling fix
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1411
2013-04-19 15:43:16 -07:00
Ned Bass 92db59ca3b Refresh links to web site
A few files still refer to @behlendorf's private fork on
github.  Use the primary web site URL instead.  Two typos
are also corrected.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-03-06 15:46:41 -08:00
Richard Yao b01615d5ac Constify structures containing function pointers
The PaX team modified the kernel's modpost to report writeable function
pointers as section mismatches because they are potential exploit
targets. We could ignore the warnings, but their presence can obscure
actual issues. Proper const correctness can also catch programming
mistakes.

Building the kernel modules against a PaX/GrSecurity patched Linux 3.4.2
kernel reports 133 section mismatches prior to this patch. This patch
eliminates 130 of them. The quantity of writeable function pointers
eliminated by constifying each structure is as follows:

vdev_opts_t             52
zil_replay_func_t       24
zio_compress_info_t     24
zio_checksum_info_t     9
space_map_ops_t         7
arc_byteswap_func_t     5

The remaining 3 writeable function pointers cannot be addressed by this
patch. 2 of them are in zpl_fs_type. The kernel's sget function requires
that this be non-const. The final writeable function pointer is created
by SPL_SHRINKER_DECLARE. The kernel's set_shrinker() and
remove_shrinker() functions also require that this be non-const.

Signed-off-by: Richard Yao <ryao@cs.stonybrook.edu>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1300
2013-03-04 08:49:32 -08:00
Brian Behlendorf 8128bd89fb Fix hot spares
The issue with hot spares in ZoL is because it opens all leaf
vdevs exclusively (O_EXCL).  On Linux, exclusive opens cause
subsequent exclusive opens to fail with EBUSY.

This could be resolved by not opening any of the devices
exclusively, which is what Illumos does, but the additional
protection offered by exclusive opens is desirable.  It cleanly
prevents you from accidentally adding an in-use non-ZFS device
to your pool.

To fix this we very slightly relaxed the usage of O_EXCL in
the following ways.

1) Functions which open the device but only read had the
   O_EXCL flag removed and were updated to use O_RDONLY.

2) A common holder was added to the vdev disk code.  This
   allow the ZFS code to internally open the device multiple
   times but non-ZFS callers may not.

3) An exception was added to make_disks() for hot spare when
   creating partition tables.  For hot spare devices which
   are already opened exclusively we skip creating the partition
   table because this must already have been done when the disk
   was originally added as a hot spare.

Additional minor changes include fixing check_in_use() to use
a partition instead of a slice suffix.  And is_spare() was moved
above make_disks() to avoid adding a forward reference.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #250
2013-03-01 13:31:02 -08:00
Tim Connors c5b247f335 -x shouldn't warn about old on-disk format or unavailable features
`zpool status -x` should only flag errors or where the pool is
unavailable.  If it imported fine but isn't using the latest features
available in the code, that's not an error.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1319
2013-02-28 09:17:09 -08:00
Brian Behlendorf f52b31eaf0 Honor 80 character limit in 'zpool status'
This is a minor nit, but the second line of the 'action' message
when you need to upgrade your pool to support feature flags exceeds
the standard 80 character limit.  Fix it by moving the word
'feature' on to the third line.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-01-31 11:06:57 -08:00
Brian Behlendorf dbf763b39b Retire zpool_id infrastructure
In the interest of maintaining only one udev helper to give vdevs
user friendly names, the zpool_id and zpool_layout infrastructure
is being retired.  They are superseded by vdev_id which incorporates
all the previous functionality.

Documentation for the new vdev_id(8) helper and its configuration
file, vdev_id.conf(5), can be found in their respective man pages.
Several useful example files are installed under /etc/zfs/.

  /etc/zfs/vdev_id.conf.alias.example
  /etc/zfs/vdev_id.conf.multipath.example
  /etc/zfs/vdev_id.conf.sas_direct.example
  /etc/zfs/vdev_id.conf.sas_switch.example

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #981
2013-01-29 12:23:17 -08:00
Ned Bass ba43f4565a vdev_id: improve keyword parsing flexibility
The vdev_id udev helper strictly requires configuration file keywords
to always be anchored at the beginning of the line and to be followed
by a space character.  However, users may prefer to use indentation or
tab delimitation.  Improve flexibility by simply requiring a keyword
to be the first field on the line.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1239
2013-01-25 13:44:32 -08:00
Christopher Siden e6f7d01502 Illumos #3397, #3398
3397 zdb <pool> <objnum> output is too verbose
3398 zdb can't dump feature flags zap objects

Reviewed by: Matt Ahrens <matthew.ahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: Richard Lowe <richlowe@richlowe.net>
Approved by: Dan McDonald <danmcd@nexenta.com>

References:
  illumos/illumos-gate@e690fb27a7
  https://www.illumos.org/issues/3397
  https://www.illumos.org/issues/3398

Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-01-11 16:42:50 -08:00
Yuri Pankov 5990da81a7 Illumos #1884, #3028, #3048, #3049, #3060, #3061, #3093
1884 Empty "used" field for zfs *space commands
3028 zfs {group,user}space -n prints (null) instead of numeric GID/UID
3048 zfs {user,group}space [-s|-S] is broken
3049 zfs {user,group}space -t doesn't really filter the results
3060 zfs {user,group}space -H output isn't tab-delimited
3061 zfs {user,group}space -o doesn't use specified fields order
3093 zfs {user,group}space's -i is noop

Reviewed by: Garry Mills <gary_mills@fastmail.fm>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  illumos/illumos-gate@89f5d17b06
  illumos changeset: 13803:b5e49d71ff0e
  https://www.illumos.org/issues/1884
  https://www.illumos.org/issues/3028
  https://www.illumos.org/issues/3048
  https://www.illumos.org/issues/3049
  https://www.illumos.org/issues/3060
  https://www.illumos.org/issues/3061
  https://www.illumos.org/issues/3093

Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1194
2013-01-11 09:17:19 -08:00
Yuri Pankov 240245896a Illumos #1377 `zpool status -D' should tell if there are no DDT entries
1337 `zpool status -D' should tell if there are no DDT entries

Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: Igor Kozhukhov <ikozhukhov@gmail.com>
Reviewed by: George Wilson <gwilson@zfsmail.com>
Approved by: Albert Lee <trisk@nexenta.com>

References:
  illumos/illumos-gate@ce72e614c1
  illumos changeset: 13432:d1ad8d106d64
  https://www.illumos.org/issues/1337

Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-01-11 09:17:13 -08:00
Brian Behlendorf a1e147eef8 Add /sbin/fsck.zfs helper
A fsck helper to accomidate distributions that expect to be able
to execute a fsck on all filesystem types.  Currently this script
does nothing but it could be extended to act as a compatibility
wrapper for 'zpool scrub'.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #964
2013-01-09 16:54:58 -08:00
Brian Behlendorf 87bdc45ccb Report realpath() canonicalization error
Rather than just reporting the failure include the passed
mount point and error number.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1153
2013-01-09 16:54:58 -08:00
George Wilson 1eb5bfa3dc Illumos #3145, #3212
3145 single-copy arc
3212 ztest: race condition between vdev_online() and spa_vdev_remove()

Reviewed by: Matt Ahrens <matthew.ahrens@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: Justin T. Gibbs <gibbs@scsiguy.com>
Approved by: Eric Schrock <eric.schrock@delphix.com>

References:
  illumos-gate/commit/9253d63df408bb48584e0b1abfcc24ef2472382e
  illumos changeset: 13840:97fd5cdf328a
  https://www.illumos.org/issues/3145
  https://www.illumos.org/issues/3212

Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #989
Closes #1137
2013-01-08 10:35:44 -08:00
George Wilson ea0b2538cd Illumos #3349: zpool upgrade -V bumps the on disk version number
3349 zpool upgrade -V bumps the on disk version number, but leaves
the in core version
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Christopher Siden <chris.siden@delphix.com>
Reviewed by: Matt Ahrens <matthew.ahrens@delphix.com>
Reviewed by: Richard Lowe <richlowe@richlowe.net>
Approved by: Dan McDonald <danmcd@nexenta.com>

References:
  illumos/illumos-gate@25345e4666
  https://www.illumos.org/issues/3349

Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-01-08 10:35:43 -08:00
Matthew Ahrens 29809a6cba Illumos #3086: unnecessarily setting DS_FLAG_INCONSISTENT on async
3086 unnecessarily setting DS_FLAG_INCONSISTENT on async
destroyed datasets
Reviewed by: Christopher Siden <chris.siden@delphix.com>
Approved by: Eric Schrock <Eric.Schrock@delphix.com>

References:
  illumos/illumos-gate@ce636f8b38
  illumos changeset: 13776:cd512c80fd75
  https://www.illumos.org/issues/3086

Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-01-08 10:35:43 -08:00
Christopher Siden b9b24bb4ca Illumos #2762: zpool command should have better support for feature flags
2762 zpool command should have better support for feature flags
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Eric Schrock <Eric.Schrock@delphix.com>

References:
  illumos/illumos-gate@57221772c3
  https://www.illumos.org/issues/2762

Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-01-08 10:35:43 -08:00
George Wilson 3bc7e0fb0f Illumos #3090 and #3102
3090 vdev_reopen() during reguid causes vdev to be treated as corrupt
3102 vdev_uberblock_load() and vdev_validate() may read the wrong label

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Christopher Siden <chris.siden@delphix.com>
Reviewed by: Garrett D'Amore <garrett@damore.org>
Approved by: Eric Schrock <Eric.Schrock@delphix.com>

References:
  illumos/illumos-gate@dfbb943217
  illumos changeset: 13777:b1e53580146d
  https://www.illumos.org/issues/3090
  https://www.illumos.org/issues/3102

Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #939
2013-01-08 10:35:42 -08:00
Brian Behlendorf 5ac0c30a94 Revert "Temporarily disable the reguid test."
This reverts commit d135245791.
Since feature flags have now been merged we can apply the real
upstream fix from Illumos.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #997
2013-01-08 10:35:42 -08:00
Christopher Siden 9ae529ec5d Illumos #2619 and #2747
2619 asynchronous destruction of ZFS file systems
2747 SPA versioning with zfs feature flags
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <gwilson@delphix.com>
Reviewed by: Richard Lowe <richlowe@richlowe.net>
Reviewed by: Dan Kruchinin <dan.kruchinin@gmail.com>
Approved by: Eric Schrock <Eric.Schrock@delphix.com>

References:
  illumos/illumos-gate@53089ab7c8
  illumos/illumos-gate@ad135b5d64
  illumos changeset: 13700:2889e2596bd6
  https://www.illumos.org/issues/2619
  https://www.illumos.org/issues/2747

NOTE: The grub specific changes were not ported.  This change
must be made to the Linux grub packages.

Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
2013-01-08 10:35:35 -08:00
Will Rouesnel 462ee8e3f3 Allow fake mounts to succeed on non-legacy filesystems.
mountall in Debian depends on being able to pass the -f parameter to
mount, which specifies a fake mount and just updates the mtab. Currently
mount.zfs will fail such a request if it is not passed with -o zfsutil.

This patch allows a fake mount on a non-legacy filesystem to succeed in
the same manner as a -o remount does, thus enabling mountall to work
correctly.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1167
2013-01-07 11:30:27 -08:00
Ned Bass 2957f38d78 vdev_id support for device link aliases
Add a vdev_id feature to map device names based on already defined
udev device links.  To increase the odds that vdev_id will run after
the rules it depends on, increase the vdev.rules rule number from 60
to 69.  With this change, vdev_id now provides functionality analogous
to zpool_id and zpool_layout, paving the way to retire those tools.

A defined alias takes precedence over a topology-derived name, but the
two naming methods can otherwise coexist. For example, one might name
drives in a JBOD with the sas_direct topology while naming an internal
L2ARC device with an alias.

For example, the following lines in vdev_id.conf will result in the
creation of links /dev/disk/by-vdev/{d1,d2}, each pointing to the same
target as the device link specified in the third field.

  #     by-vdev
  #     name     fully qualified or base name of device link
  alias d1       /dev/disk/by-id/wwn-0x5000c5002de3b9ca
  alias d2       wwn-0x5000c5002def789e

Also perform some minor vdev_id cleanup, such as removal of the unused
-s command line option.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #981
2012-12-03 14:04:47 -08:00
Cyril Plisko 4588bf5701 Make zpool attach -o ashift=... actually work
Commit df83110856 missed update to
getopt() call, while delivering all the rest. This commit adds
"o" to getopt().

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #566
2012-11-30 13:50:26 -08:00
Cyril Plisko 38b344d22a vdev_id fails to handle complex device topologies
While expanding positional parameters shell requires non-single
digits to be enclosed in braces. When the SAS topology is
non-trivial the number of positional parameters generated internally
by vdev_id script (using set -- ...) easily crosses single digit limit
and vdev_id fails to generate links.

Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1119
2012-11-29 13:07:47 -08:00
Ned Bass a6ef9522ea Make vdev_id POSIX sh compatible
Full bash may not be available in all environments where udev helpers
run, such as in an initial ramdisk.  To avoid breakage in this case,
remove use of bash-specific features such as variable arrays and the
`declare' keyword from the vdev_id script.

Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #870
2012-11-27 14:23:22 -08:00
nordaux 33364b15d3 mount.zfs: canonicalize mount point for mtab
Canonicalize the mount point passed to the mount.zfs helper.
This way a clean path is always added to mtab which ensures
the umount can properly locate and remove the entry.

Test case:
$ mkdir /mnt/foo
$ mount -t zfs zpool/foo /mnt/../mnt/foo////
$ umount /mnt/foo
$ cat /etc/mtab | grep zpool/foo
zpool/foo /mnt/../mnt/foo//// zfs rw 0 0

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #573
2012-11-15 14:28:52 -08:00
Cyril Plisko df83110856 Add "-o ashift" to zpool add and zpool attach
When adding devices to an existing pool "ashift" property is
auto-detected.  However, if this property was overridden at
the pool creation time (i.e. zpool create -o ashift=12 tank ...)
this may not be what the user wants.  This commit lets the user
specify the value of "ashift" property to be used with newly
added drives. For example,

    zpool add -o ashift=12 tank disk1
    zpool attach -o ashift=12 tank disk1 disk2

Signed-off-by: Cyril Plisko <cyril.plisko@mountall.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #566
2012-11-15 11:06:14 -08:00
George Wilson 32a9872bba Illumos #2671: zpool import should not fail if vdev ashift has increased
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: Richard Elling <richard.elling@richardelling.com>
Reviewed by: Gordon Ross <gwr@nexenta.com>
Reviewed by: Garrett D'Amore <garrett@damore.org>
Approved by: Richard Lowe <richlowe@richlowe.net>

Refererces to Illumos issue:
      https://www.illumos.org/issues/2671

This patch has been slightly modified from the upstream Illumos
version.  In the upstream implementation a warning message is
logged to the console.  To prevent pointless console noise this
notification is now posted as a "ereport.fs.zfs.vdev.bad_ashift"
event.

The event indicates a non-optimial (but entirely safe) ashift
value was used to create the pool.  Depending on your workload
this may impact pool performance.  Unfortunately, the only way
to correct the issue is to recreate the pool with a new ashift.

NOTE: The unrelated fix to the comment in zpool_main.c appears
in the upstream commit and was preserved for consistnecy.

Ported-by: Cyril Plisko <cyril.plisko@mountall.com>
Reworked-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #955
2012-11-15 11:05:59 -08:00
Brian Behlendorf e8fd45a0f9 Add ddt_object_count() error handling
The interface for the ddt_zap_count() function assumes it can
never fail.  However, internally ddt_zap_count() is implemented
with zap_count() which can potentially fail.  Now because there
was no way to return the error to the caller a VERIFY was used
to ensure this case never happens.

Unfortunately, it has been observed that pools can be damaged in
such a way that zap_count() fails.  The result is that the pool can
not be imported without hitting the VERIFY and crashing the system.

This patch reworks ddt_object_count() so the error can be safely
caught and returned to the caller.  This allows a pool which has
be damaged in this way to be safely rewound for import.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #910
2012-10-29 08:57:45 -07:00
Brian Behlendorf eac4720465 Allow 'zpool replace' to use short device names
The 'zpool replace' command would fail when given a short name
because unlike on other platforms the short name cannot be
deterministically expanded to a single path.  Multiple path
prefixes must be checked and in addition the partition suffix
for whole disks is determined by the prefix.

To handle this complexity a zfs_strcmp_pathname() function was
added which takes either a short or fully qualified device name.
Short names will be expanded using the prefixes in the default
import search path, or the ZPOOL_IMPORT_PATH environment variable
if it's defined.  All posible expansions are then compared against
the comparison path.  Care is taken to strip redundant slashes to
ensure legitimate matches are not missed.

In the context of this work the existing zfs_resolve_shortname()
function was extended to consider the ZPOOL_IMPORT_PATH when set.
The zfs_append_partition() interface was also simplified to take
only a single buffer.

The vast majority of these changes rework existing Linux specific
code which was originally written to accomidate udev.  However,
there is some minimal cleanup which removes Illumos specific code.
This was done to improve readability but the basic flow and intent
of the upstream code was maintained.

These changes are the logical conclusion of the previos work to
adjust the 'zpool import' search behavior, see commit 44867b6a.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #544
Closes #976
2012-10-22 08:45:58 -07:00
Brian Behlendorf 26099167e6 Disable ztest deadman timer
The ztest deadman timer has been causing false positives in the
testing VMs.  To make it easier to spot possible regressions
I'm disabling this timer.  The buildbot test infrastructure
will still mark ztest instances which take to long to complete
as failures.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1018
2012-10-14 19:35:09 -07:00
Brian Behlendorf ae380cfa76 Realpath arg 2 must be a minimum of PATH_MAX
The realpath(3) function expects that when a buffer is passed
for the 'resolved_path' that it be at least PATH_MAX in length.
If it's not a buffer overflow may occur.

Therefore the passed buffer size is changed from MAXNAMELEN to
MAXPATHLEN.  We also take this opertunity to dynamically allocate
the buffer to keep it off the stack.

  warning: call to '__realpath_chk_warn' declared with attribute
  warning: second argument of realpath must be either NULL or at
  least PATH_MAX bytes long buffer [enabled by default]

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-10-04 13:19:10 -07:00
Brian Behlendorf 5be98cfe2f Verify the return value for warn_unused_result functions
Under Linux the following functions are flagged with the
attribute warn_unused_result, this triggers a warning when
ever they are used without checking the return value.

To handle this case we check the result VERIFY().  It's
better to detect this immediately on failure rather than
segfault farther down in the function.

  ../../cmd/ztest/ztest.c:6033:2: warning:
  ignoring return value of 'asprintf', declared with
  attribute warn_unused_result [-Wunused-result]
  ../../cmd/ztest/ztest.c:739:3: warning:
  ignoring return value of 'realpath', declared with
  attribute warn_unused_result [-Wunused-result]

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-10-04 13:19:10 -07:00
Brian Behlendorf facbbe4366 Replace tempnam() with mkstemp()
The use of tempnam() is racy and it should be avoided in favor of
mkstemp().  According to the Linux tempnam(3) man page.

  "Although tempnam() generates names that are difficult to guess,
  it is nevertheless possible that between the time that tempnam()
  returns a pathname, and the time that the program opens it, another
  program might create that pathname using open(2), or create it as
  a symbolic link.  This can lead to security holes.  To avoid such
  possibilities, use the open(2) O_EXCL flag to open the  pathname.
  Or better yet, use mkstemp(3) or tmpfile(3)."

This issue was flagged by gcc.

  ztest.o: In function `setup_data_fd': cmd/ztest/ztest.c:5822:
  warning: the use of `tempnam' is dangerous, better use `mkstemp'

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-10-04 13:19:10 -07:00
Brian Behlendorf 483106eb71 Minimize ztest stack frame size
To ensure ztest behaves as similarly as possible to the kernel
implementation of ZFS we attempt to honor the kernel stack limits.
This includes keeping the individual stack frame sizes under 1K
in size.  We currently use gcc to detect and enforce this limit.

Therefore to get this building cleanly with full debugging enabled
the stack usage in the following functions has been reduced by
moving the buffer to the heap.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-10-04 13:19:09 -07:00
Etienne Dechamps 9d81146b01 Use dynamic file descriptor numbers in ztest.
Currently, ztest expects to get 3 and 4 as the file descriptors for
data and random files, respectively. This is quite fragile and breaks
easily if ztest is run with these file descriptors already opened
(e.g. in a complex shell script).

This patch fixes the issue by removing the assumptions on the file
descriptor numbers that open() returns.

For the random file (/dev/urandom), the new code doesn't rely on a
shared file descriptor; instead, it reopens the file in the child.

For the data file, the new code writes the file descriptor number into
a "ZTEST_FD_DATA" environment variable so that it can be recovered
after the execv() call.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-10-04 13:19:09 -07:00
Christopher Siden 22257dc0d5 Fix mmap() usage in ztest.
illumos/illumos-gate@ad135b5d64
Illumos changeset: 13700:2889e2596bd6

Note that this is only a partial port of the aforementioned Illumos
changeset.

Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <gwilson@delphix.com>
Reviewed by: Richard Lowe <richlowe@richlowe.net>
Reviewed by: Dan Kruchinin <dan.kruchinin@gmail.com>
Approved by: Eric Schrock <Eric.Schrock@delphix.com>

Ported to zfsonlinux by: Etienne Dechamps <etienne.dechamps@ovh.net>
2012-10-04 13:19:09 -07:00
Chris Siden c242c188fd Illumos #1950: ztest backwards compatibility testing option.
illumos/illumos-gate@420dfc9585
Illumos changeset: 13571:a5771a96228c

1950 ztest backwards compatibility testing option

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Richard Lowe <richlowe@richlowe.net>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Approved by: Eric Schrock <eric.schrock@delphix.com>

Ported-by: Etienne Dechamps <etienne.dechamps@ovh.net>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-10-04 13:18:53 -07:00
Etienne Dechamps d135245791 Temporarily disable the reguid test.
Currently, ztest fails with the following error:

    error: Pool 'ztest' has encountered an uncorrectable I/O failure
    and the failure mode property for this pool is set to panic.

We know how to fix it (see issue #939), but it may take some time
before we get around to merging the fix, which has some heavy
dependencies.

In the mean time, it is not ideal to be unable to use ztest just
because of a small isolated issue, so this patch works around the
problem by disabling the reguid test. This is just a temporary hack to
keep ztest usable.

The reguid test will be enabled again when the proper fix is merged.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #997
2012-10-03 13:59:02 -07:00
Etienne Dechamps 6aec1cd5a6 Fix ztest vdev file paths.
Currently, in several instances (but not all), ztest generates vdev
file paths using a statement similar to this:

    snprintf(path, sizeof (path), ztest_dev_template, ...);

This worked fine until 40b84e7aec, which
changed path to be a pointer to the heap instead of an array allocated
on the stack. Before this change, sizeof(path) would return the size of
the array; now, it returns the size of the pointer instead.

As a result, the aforementioned sprintf statement uses the wrong size
and truncates the vdev file path to the first 4 or 8 bytes (depending
on the architecture). Typically, with default settings, the file path
will become "/tmp/zt" instead of "/test/ztest.XXX".

This issue only exists in ztest_vdev_attach_detach() and
ztest_fault_inject(), which explains why ztest doesn't fail right away.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #989
2012-10-03 13:32:48 -07:00
Etienne Dechamps 0aebd4f9e3 Create threads in detached state in userspace.
Currently, thread_create(), when called in userspace, creates a
joinable (i.e. not detached thread). This is the pthread default.

Unfortunately, this does not reproduce kthreads behavior (kthreads
are always detached). In addition, this contradicts the original
Solaris code which creates userspace threads in detached mode.

These joinable threads are never joined, which leads to a leakage of
pthread thread objects ("zombie threads"). This in turn results in
excessive ressource consumption, and possible ressource exhaustion in
extreme cases (e.g. long ztest runs).

This patch fixes the issue by creating userspace threads in detached
mode. The only exception is ztest worker threads which are meant to be
joinable.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #989
2012-10-03 13:32:48 -07:00
Bill Pijewski 37abac6d55 Illumos #2703: add mechanism to report ZFS send progress
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed by: Richard Lowe <richlowe@richlowe.net>
Approved by: Eric Schrock <Eric.Schrock@delphix.com>

References:
  https://www.illumos.org/issues/2703

Ported by: Martin Matuska <martin@matuska.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-09-19 13:39:06 -07:00
Chris Siden 1bd201e70d Illumos #1948: zpool list should show more detailed pool info
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: Richard Lowe <richlowe@richlowe.net>
Reviewed by: Albert Lee <trisk@nexenta.com>
Reviewed by: Dan McDonald <danmcd@nexenta.com>
Reviewed by: Garrett D'Amore <garrett@damore.org>
Approved by: Eric Schrock <eric.schrock@delphix.com>

References:
  https://www.illumos.org/issues/1948

Ported by:	Martin Matuska <martin@matuska.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #685
2012-09-19 13:39:05 -07:00
Richard Lowe dd4769adc0 Illumos #2088 zdb could use a reasonable manual page
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
Reviewed by: Garrett D'Amore <garrett@damore.org>
Reviewed by: George Wilson <gwilson@zfsmail.com>
Reviewed by: Steve Gonczi <gonczi@comcast.net>
Reviewed by: Richard Elling <richard.elling@richardelling.com>
Approved by: Garrett D'Amore <garrett@damore.org>

References:
      https://www.illumos.org/issues/2088

Ported by: Cyril Plisko <cyril.plisko@mountall.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #682
2012-09-18 09:09:13 -07:00
Brian Behlendorf 44867b6d6e Improve `zpool import` search behavior
The goal of this change is to make 'zpool import' prefer to use
the peristent /dev/mapper or /dev/disk/by-* paths.  These are far
preferable to the devices in /dev/ whos names are not persistent
and are determined by the order in which a device is detected.

This patch improves things by changing the default search path from
just to the top level /dev/ directory to (in order):

  /dev/disk/by-vdev   - Custom rules, use first if they exist
  /dev/disk/zpool     - Custom rules, use first if they exist
  /dev/mapper         - Use multipath devices before components
  /dev/disk/by-uuid   - Single unique entry and persistent
  /dev/disk/by-id     - May be multiple entries and persistent
  /dev/disk/by-path   - Encodes physical location and persistent
  /dev/disk/by-label  - Custom persistent labels
  /dev                - UNSAFE device names will change

The default search path can be overriden by setting the
ZPOOL_IMPORT_PATH environment variable.  This must be a colon
delimited list of paths which are searched for vdevs.  If the
'zpool import -d' option is specified only those listed paths
will be searched.

Finally, when multiple paths to the same device are found.  If one
of the paths is an exact match for the path used last time to import
the pool it will be used.  When there are no exact matches the
prefered path will be determined by the provided search order.

This means you can still import a pool and force specific names by
providing the -d <path> option.  And the prefered names will persist
as long as those paths exist on your system.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #965
2012-09-17 13:49:07 -07:00
Cyril Plisko 8e8e7f35b7 Fix zdb printf format string for ZIL data blocks
Without this fix the zdb printouts of ZIL data blocks look full of FF
due to printf() handling its arguments as int by default.

Here is the output before the fix

                TX_WRITE            len   4136, txg 1093817, seq 149231
                        foid 4242, offset 0, length f68
                        G FFFFFF8EFFFFFF87FFFFFF91FFFFFFCC 1c
FFFFFFAFFFFFFFC9FFFFFFBAZ FFFFFFC3

And the same after the fix

                TX_WRITE            len   4136, txg 1093817, seq 149231
                        foid 4242, offset 0, length f68
                        G 8E8791CC 1cAFC9BAZ C3

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #962
2012-09-13 09:02:12 -07:00
Cyril Plisko af909a1089 Illumos #3064: usr/src/cmd/zpool/zpool_main.c misspells "successful"
Reviewed by: Andrew Stormont <Andrew.Stormont@nexenta.com>
Reviewed by: Kartik Mistry <kartik.mistry@gmail.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>

References:
      https://www.illumos.org/issues/3064

Signed-off-by: Cyril Plisko <cyril.plisko@mountall.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-09-11 10:19:50 -07:00
Etienne Dechamps b815ff9a8f Silence "setting dataset to sync always" message in ztest.
ztest outputs a message when testing sync=always no matter what the
verbosity level is. There is no point outputting this message for low
verbosity levels.

With this patch the message is only displayed at verbosity level 5 or
above. The result is less output pollution.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #951
2012-09-10 10:55:44 -07:00
Brian Behlendorf 1ecc6d1265 Add zstreamdump .gitignore
When zstreamdump was merged in commit b79fc3f we failed to add
the needed .gitignore file.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-09-06 14:23:11 -07:00
Etienne Dechamps ba7dbeb22e Add libnvpair to mount_zfs dependencies
Commit e6f290535c added libzpool to
the mount_zfs dependencies.  This brought in the nvpair symbols
which are used by libzpool.  To resolve this include the libnvpair
library for mount_zfs even though mount_zfs doesn't directly
require any of these symbols.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #926
2012-09-02 15:36:09 -07:00
Martin Matuska b79fc3fea9 Add zstreamdump(8) command to examine ZFS send streams.
Obtained from: illumos-gate revision 11935:538c866aaac6
Source: ssh://anonhg@hg.illumos.org/illumos-gate

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #905
2012-09-02 14:54:27 -07:00
Etienne Dechamps e6f290535c Fix mount_zfs dependency on libzpool.
mount_zfs depends on libzpool for zfs_prop_written since
330d06f90d. Unfortunately, the Makefile
for mount_zfs has not been modified to reflect this. As a result,
libtool doesn't know about the dependency, which may result in the wrong
libzpool being used during the build (e.g. the libzpool from the system
instead of the libzpool from the build directory).

This patch adds the dependency to fix the issue.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Fixes #909.
2012-08-30 16:06:46 -07:00
Brian Behlendorf ca8b5af89d Remove autotools products
Remove all of the generated autotools products from the repository
and update the .gitignore files accordingly.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #718
2012-08-27 11:47:44 -07:00
Christopher Siden e956d65106 Illumos #1796, #2871, #2903, #2957
1796 "ZFS HOLD" should not be used when doing "ZFS SEND" from a read-only pool
2871 support for __ZFS_POOL_RESTRICT used by ZFS test suite
2903 zfs destroy -d does not work
2957 zfs destroy -R/r sometimes fails when removing defer-destroyed snapshot
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Eric Schrock <Eric.Schrock@delphix.com>

References:
  https://www.illumos.org/issues/1796
  https://www.illumos.org/issues/2871
  https://www.illumos.org/issues/2903
  https://www.illumos.org/issues/2957

Ported by: Martin Matuska <martin@matuska.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-08-23 10:40:02 -07:00
Eric Schrock db49968e5c Illumos #2635: 'zfs rename -f' to perform force unmount
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: George Wilson <George.Wilson@delphix.com>
Reviewed by: Bill Pijewski <wdp@joyent.com>
Reviewed by: Richard Elling <richard.elling@richardelling.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/2635

Ported by: Martin Matuska <martin@matuska.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #717
2012-08-23 10:39:43 -07:00
Andrew Stormont e346ec25af Illumos #1936: add support for "-t <datatype>" argument to zfs get
Reviewed by: Kartik Mistry <kartik@nexenta.com>
Reviewed by: Dan McDonald <danmcd@nexenta.com>
Reviewed by: Richard Elling <richard.elling@gmail.com>
Reviewed by: Garrett D'Amore <garrett@damore.org>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/1936

Ported by: Martin Matuska <martin@matuska.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #681
2012-08-23 10:35:59 -07:00
Alexander Eremin 684e8c0643 Illumos #1726: Removal of pyzfs broke delegation for volumes
Reviewed by: Andrew Stormont <andyjstormont@googlemail.com>
Reviewed by: Garrett D'Amore <garrett@nexenta.com>
Reviewed by: Richard Lowe <richlowe@richlowe.net>
Reviewed by: Albert Lee <trisk@nexenta.com>
Approved by: Garrett D'Amore <garrett@nexenta.com>

References:
  https://www.illumos.org/issues/1726

Ported by: Martin Matuska <martin@matuska.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-08-23 10:35:37 -07:00
Alexander Eremin 79e722432f Illumos #1977: zfs allow arguments not parsed correctly after pyzfs removal
Reviewed by: Garrett D'Amore <garrett.damore@gmail.com>
Reviewed by: Albert Lee <trisk@nexenta.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/1977

Ported by: Martin Matuska <martin@matuska.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-08-23 10:21:04 -07:00
Dan McDonald d96eb2b153 Illumos #1693: persistent 'comment' field for a zpool
Reviewed by: George Wilson <gwilson@zfsmail.com>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/1693

Ported by: Martin Matuska <martin@matuska.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #678
2012-08-08 11:49:37 -07:00
Etienne Dechamps ee5fd0bb80 Set zvol discard_granularity to the volblocksize.
Currently, zvols have a discard granularity set to 0, which suggests to
the upper layer that discard requests of arbirarily small size and
alignment can be made efficiently.

In practice however, ZFS does not handle unaligned discard requests
efficiently: indeed, it is unable to free a part of a block. It will
write zeros to the specified range instead, which is both useless and
inefficient (see dnode_free_range).

With this patch, zvol block devices expose volblocksize as their discard
granularity, so the upper layer is aware that it's not supposed to send
discard requests smaller than volblocksize.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #862
2012-08-07 14:55:31 -07:00
Matthew Ahrens 330d06f90d Illumos #1644, #1645, #1646, #1647, #1708
1644 add ZFS "clones" property
1645 add ZFS "written" and "written@..." properties
1646 "zfs send" should estimate size of stream
1647 "zfs destroy" should determine space reclaimed by
     destroying multiple snapshots
1708 adjust size of zpool history data

References:
  https://www.illumos.org/issues/1644
  https://www.illumos.org/issues/1645
  https://www.illumos.org/issues/1646
  https://www.illumos.org/issues/1647
  https://www.illumos.org/issues/1708

This commit modifies the user to kernel space ioctl ABI.  Extra
care should be taken when updating to ensure both the kernel
modules and utilities are updated.  This change has reordered
all of the new ioctl()s to the end of the list.  This should
help minimize this issue in the future.

Reviewed by: Richard Lowe <richlowe@richlowe.net>
Reviewed by: George Wilson <gwilson@zfsmail.com>
Reviewed by: Albert Lee <trisk@opensolaris.org>
Approved by: Garrett D'Amore <garret@nexenta.com>

Ported by: Martin Matuska <martin@matuska.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #826
Closes #664
2012-07-31 09:25:30 -07:00
Brian Behlendorf c171ea71bb Allow '-o remount' for non-legacy datasets
This is done for compatibility with existing Linux infrastructure.

In particular, when using zfs as a root filesystem there are init
scripts which as part of shutdown remount root read-only.  Also,
the new systemd infrastructure being used by Fedora expects to be
able to remount a file system read-write.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #847
2012-07-30 15:58:02 -07:00
Richard Yao 739a1a82e0 Linux 3.5 compat, end_writeback() changed to clear_inode()
The end_writeback() function was changed by moving the call to
inode_sync_wait() earlier in to evict().   This effecitvely changes
the ordering of the sync but it does not impact the details of
the zfs implementation.

However, as part of this change end_writeback() was renamed to
clear_inode() to reflect the new semantics.  This change does
impact us and clear_inode() now maps to end_writeback() for
kernels prior to 3.5.

Signed-off-by: Richard Yao <ryao@cs.stonybrook.edu>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #784
2012-07-23 12:29:36 -07:00
Richard Yao ea1fdf46e2 Linux 3.5 compat, iops->truncate_range() removed
The vmtruncate_range() support has been removed from the kernel in
favor of using the fallocate method in the file_operations table.

Signed-off-by: Richard Yao <ryao@cs.stonybrook.edu>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #784
2012-07-23 12:29:32 -07:00
Richard Yao 756c3e5a9c Linux 3.5 compat, eops->encode_fh() takes inodes
The export_operations member ->encode_fh() has been updated to
take both the child and parent inodes.  This interface used to
take the child dentry and a bool describing if the parent is needed.

NOTE: While updating this code I noticed that we do not currently
cleanly handle the case where we're passed a connectable parent.
This code should be audited to make sure we're doing the right thing.

Signed-off-by: Richard Yao <ryao@cs.stonybrook.edu>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #784
2012-07-23 12:29:23 -07:00
Etienne Dechamps b5a28807cd Move partition scanning from userspace to module.
Currently, zpool online -e (dynamic vdev expansion) doesn't work on
whole disks because we're invoking ioctl(BLKRRPART) from userspace
while ZFS still has a partition open on the disk, which results in
EBUSY.

This patch moves the BLKRRPART invocation from the zpool utility to the
module. Specifically, this is done just before opening the device in
vdev_disk_open() which is called inside vdev_reopen(). This requires
jumping through some hoops to get to the disk device from the partition
device, and to make sure we can still open the partition after the
BLKRRPART call.

Note that this new code path is triggered on dynamic vdev expansion
only; other actions, like creating a new pool, are unchanged and still
call BLKRRPART from userspace.

This change also depends on API changes which are available in 2.6.37
and latter kernels.  The build system has been updated to detect this,
but there is no compatibility mode for older kernels.  This means that
online expansion will NOT be available in older kernels.  However, it
will still be possible to expand the vdev offline.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #808
2012-07-17 09:17:31 -07:00
Garrett D'Amore 3541dc6d02 Illumos #1748: desire support for reguid in zfs
Reviewed by: George Wilson <gwilson@zfsmail.com>
Reviewed by: Igor Kozhukhov <ikozhukhov@gmail.com>
Reviewed by: Alexander Eremin <alexander.eremin@nexenta.com>
Reviewed by: Alexander Stetsenko <ams@nexenta.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/1748

This commit modifies the user to kernel space ioctl ABI.  Extra
care should be taken when updating to ensure both the kernel
modules and utilities are updated.  If only the user space
component is updated both the 'zpool events' command and the
'zpool reguid' command will not work until the kernel modules
are updated.

Ported by:     Martin Matuska <martin@matuska.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #665
2012-07-11 13:08:56 -07:00
Pawel Jakub Dawidek 0cee24064a Speed up 'zfs list -t snapshot -o name -s name'
FreeBSD #xxx:  Dramatically optimize listing snapshots when user
requests only snapshot names and wants to sort them by name, ie.
when executes:

  # zfs list -t snapshot -o name -s name

Because only name is needed we don't have to read all snapshot
properties.

Below you can find how long does it take to list 34509 snapshots
from a single disk pool before and after this change with cold and
warm cache:

    before:

        # time zfs list -t snapshot -o name -s name > /dev/null
        cold cache: 525s
        warm cache: 218s

    after:

        # time zfs list -t snapshot -o name -s name > /dev/null
        cold cache: 1.7s
        warm cache: 1.1s

NOTE: This patch only appears in FreeBSD.  If/when Illumos picks up
the change we may want to drop this patch and adopt their version.
However, for now this addresses a real issue.

Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #450
2012-06-14 09:49:04 -07:00
Brian Behlendorf fe2fc8f6d3 Workaround for failing zvol_id
This is not a proper fix.  It is just a workaround for the stack
smashing detected by gcc in zvol_id.  We simply disable the gcc
stack protector for now when building the zvol_id udev helper.
Once the root cause is resolved this patch should be reverted.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issues #569
2012-06-13 11:25:17 -07:00
Daniel Verite 92e91da208 Include <locale.h> to avoid error: 'LC_ALL' undeclared.
When compiling ZFS with CFLAGS=-O0 it will trigger the following error.
Resolve the issue by properly including locale.h.

  ../../cmd/mount_zfs/mount_zfs.c: In function 'main':
  ../../cmd/mount_zfs/mount_zfs.c:318:2: warning: implicit declaration
      of function 'setlocale' [-Wimplicit-function-declaration]
  ../../cmd/mount_zfs/mount_zfs.c:318:19: error: 'LC_ALL' undeclared
      (first use in this function)

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #724
2012-06-11 10:21:40 -07:00
Richard Yao 6a0936babc Linux 3.4 compat, d_make_root() replaces d_alloc_root()
torvalds/linux@adc0e91ab1 introduced
introduced d_make_root() as a replacement for d_alloc_root(). Further
commits appear to have removed d_alloc_root() from the Linux source
tree. This causes the following failure:

  error: implicit declaration of function 'd_alloc_root'
  [-Werror=implicit-function-declaration]

To correct this we update the code to use the current d_make_root()
interface for readability.  Then we introduce an autotools check
to determine if d_make_root() is available.  If it isn't then we
define some compatibility logic which used the older d_alloc_root()
interface.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #776
2012-06-11 10:04:49 -07:00
Ned A. Bass 821b683436 Add vdev_id for JBOD-friendly udev aliases
vdev_id parses the file /etc/zfs/vdev_id.conf to map a physical path
in a storage topology to a channel name.  The channel name is combined
with a disk enclosure slot number to create an alias that reflects the
physical location of the drive.  This is particularly helpful when it
comes to tasks like replacing failed drives.  Slot numbers may also be
re-mapped in case the default numbering is unsatisfactory.  The drive
aliases will be created as symbolic links in /dev/disk/by-vdev.

The only currently supported topologies are sas_direct and sas_switch:

o  sas_direct - a channel is uniquely identified by a PCI slot and a
   HBA port

o  sas_switch - a channel is uniquely identified by a SAS switch port

A multipath mode is supported in which dm-mpath devices are handled by
examining the first running component disk, as reported by 'multipath
-l'.  In multipath mode the configuration file should contain a
channel definition with the same name for each path to a given
enclosure.

vdev_id can replace the existing zpool_id script on systems where the
storage topology conforms to sas_direct or sas_switch.  The script
could be extended to support other topologies as well.  The advantage
of vdev_id is that it is driven by a single static input file that can
be shared across multiple nodes having a common storage toplogy.
zpool_id, on the other hand, requires a unique /etc/zfs/zdev.conf per
node and a separate slot-mapping file.  However, zpool_id provides the
flexibility of using any device names that show up in
/dev/disk/by-path, so it may still be needed on some systems.

vdev_id's functionality subsumes that of the sas_switch_id script, and
it is unlikely that anyone is using it, so sas_switch_id is removed.

Finally, /dev/disk/by-vdev is added to the list of directories that
'zpool import' will scan.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #713
2012-06-01 08:55:14 -07:00
Brian Behlendorf b39d3b9f7b Linux 3.3 compat, iops->create()/mkdir()/mknod()
The mode argument of iops->create()/mkdir()/mknod() was changed from
an 'int' to a 'umode_t'.  To prevent a compiler warning an autoconf
check was added to detect the API change and then correctly set a
zpl_umode_t typedef.  There is no functional change.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #701
2012-04-30 12:52:38 -07:00
Richard Lowe ad60af8e1b Illumos #2067: uninitialized variables in zfs(1M) may make snapshots undestroyable
Reviewed by: Joshua M. Clulow <josh@sysmgr.org>
Reviewed by: Milan Jurik <milan.jurik@xylab.cz>
Reviewed by: Igor Kozhukhov <ikozhukhov@gmail.com>
Reviewed by: Garrett D'Amore <garrett@damore.org>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: Steve Gonczi <gonczi@comcast.net>
Approved by: Garrett D'Amore <garrett@damore.org>

References:
	https://www.illumos.org/issues/2067

Ported by: Richard Yao <ryao@cs.stonybrook.edu>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-04-27 15:17:23 -07:00
Frederik Wessels 95bcd51ecc Illumos #1946: incorrect formatting when listing output of multiple pools with zpool iostat -v
Reviewed by: Richard Elling <richard.elling@richardelling.com>
Reviewed by: Joshua M. Clulow <josh@sysmgr.org>
Approved by: Richard Lowe <richlowe@richlowe.net>

Reference to Illumos issue:
  https://www.illumos.org/issues/1946

Ported by: Richard Yao <ryao@cs.stonybrook.edu>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-04-27 15:15:37 -07:00
Mike Harsch 187632dcef Illumos #952: separate intent logs should be obvious in 'zpool iostat' output
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: Dan McDonald <danmcd@nexenta.com>
Reviewed by: Garrett D'Amore <garrett@nexenta.com>
Approved by: Eric Schrock <eric.schrock@delphix.com>

Refererce to Illumos issue:
  https://www.illumos.org/issues/952

Ported-by: Richard Yao <ryao@cs.stonybrook.edu>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #607
2012-04-27 15:12:43 -07:00
P.SCH cf81b00a73 ZFS list snapshot property alias
Add support for the `zfs list -t snap` alias which is available under
Oracle Solaris 11.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #640
2012-04-11 12:02:46 -07:00
P.SCH 10b75496bb ZFS snapshot alias
For consistency, and because it's handy, add the 'zfs snap' alias which
was introduced by Oracle Solaris 11.  This includes an update to the
man page to reflect all the available alias (snap, umount, and recv).

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #640
2012-04-11 12:02:31 -07:00
Richard Yao 847de12271 Print human readable error message for ENOENT
A cryptic error code is printed when mounting a legacy dataset to a
non-existent mountpoint. This patch changes this behavior to print
"mount point '%s' does not exist", which is similar to the error
message printed when mounting procfs.

The single quotes were added to be consistent with the existing EBUSY
error message, which is the only difference between this error message
and the one that is printed when the same condition occurs when mounting
procfs.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #633
2012-04-03 10:24:34 -07:00
Craig Sanders 9fc60702c6 Remove hard-coded 80 column output
When stdout is detected to be a tty use the number of columns
specified by the terminal.  If that fails fall back to a default
80 column width.  In the non-tty case allow for 999 column lines.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-03-27 15:01:08 -07:00
Brian Behlendorf f47e1351db Fix executable permissions
Caught by lint, this permission change was accidentally introduced
by commit 42cb3819f1.  Restore the
correct permissions and while I'm at it add a missing whack-bang
to config/ltmain.sh.

  lint: executable-not-elf-or-script: zpool_main.c zfs_main.c

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #620
2012-03-26 11:52:44 -07:00
Brian Behlendorf 1c5de20ae2 Add --enable-debug-dmu-tx configure option
Allow rigorous (and expensive) tx validation to be enabled/disabled
indepentantly from the standard zfs debugging.  When enabled these
checks ensure that all txs are constructed properly and that a dbuf
is never dirtied without taking the correct tx hold.

This checking is particularly helpful when adding new dmu consumers
like Lustre.  However, for established consumers such as the zpl
with no known outstanding tx construction problems this is just
overhead.

--enable-debug-dmu-tx  - Enable/disable validation of each tx as
--disable-debug-dmu-tx   it is constructed.  By default validation
                         is disabled due to performance concerns.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-03-23 12:25:17 -07:00
Brian Behlendorf ebe7e575ea Add .zfs control directory
Add support for the .zfs control directory.  This was accomplished
by leveraging as much of the existing ZFS infrastructure as posible
and updating it for Linux as required.  The bulk of the core
functionality is now all there with the following limitations.

*) The .zfs/snapshot directory automount support requires a 2.6.37
   or newer kernel.  The exception is RHEL6.2 which has backported
   the d_automount patches.

*) Creating/destroying/renaming snapshots with mkdir/rmdir/mv
   in the .zfs/snapshot directory works as expected.  However,
   this functionality is only available to root until zfs
   delegations are finished.

      * mkdir - create a snapshot
      * rmdir - destroy a snapshot
      * mv    - rename a snapshot

The following issues are known defeciences, but we expect them to
be addressed by future commits.

*) Add automount support for kernels older the 2.6.37.  This should
   be possible using follow_link() which is what Linux did before.

*) Accessing the .zfs/snapshot directory via NFS is not yet possible.
   The majority of the ground work for this is complete.  However,
   finishing this work will require resolving some lingering
   integration issues with the Linux NFS kernel server.

*) The .zfs/shares directory exists but no futher smb functionality
   has yet been implemented.

Contributions-by: Rohan Puri <rohan.puri15@gmail.com>
Contributiobs-by: Andrew Barnes <barnes333@gmail.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #173
2012-03-22 13:03:47 -07:00
Gregor Kopka 42cb3819f1 Use stderr for 'no pools/datasets available' error
The 'zfs list' and 'zpool list' commands output the message
'no datasets/pools available' to stdout.  This should go to
stderr and only the available datasets/pools should go to
stdout.  Returning nothing to stdout is expected behavior
when there is nothing to list.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #581
2012-03-15 10:24:00 -07:00
Brian Behlendorf 4b787d75c8 Cleanly support debug packages
Allow a source rpm to be rebuilt with debugging enabled.  This
avoids the need to have to manually modify the spec file.  By
default debugging is still largely disabled.  To enable specific
debugging features use the following options with rpmbuild.

  '--with debug'               - Enables ASSERTs

  # For example:
  $ rpmbuild --rebuild --with debug zfs-modules-0.6.0-rc6.src.rpm

Additionally, ZFS_CONFIG has been added to zfs_config.h for
packages which build against these headers.  This is critical
to ensure both zfs and the dependant package are using the same
prototype and structure definitions.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-02-27 14:08:17 -08:00
Ned Bass 3a4f6caf08 Return success from check_slice() if device doesn't exist
When creating a new pool, make_root_vdev() calls check_in_use() to
ensure that none of the consituent disks are in use.  If the disk
contains a valid vdev label it is read to retrieve the list of its
child vdevs and these are checked recursively.  However, the
partitions stored in the vdev label my no longer exist, for example
if the partition table has since been altered.  In any such case we
would want the pool creation to proceed, so this change removes the
check from check_slice() that returns an error if the device doesn't
exist.  As an added assurance, the Solaris implementation also
returns sucess on ENOENT.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-02-27 08:52:38 -08:00
Etienne Dechamps 30930fba21 Add support for DISCARD to ZVOLs.
DISCARD (REQ_DISCARD, BLKDISCARD) is useful for thin provisioning.
It allows ZVOL clients to discard (unmap, trim) block ranges from
a ZVOL, thus optimizing disk space usage by allowing a ZVOL to
shrink instead of just grow.

We can't use zfs_space() or zfs_freesp() here, since these functions
only work on regular files, not volumes. Fortunately we can use the
low-level function dmu_free_long_range() which does exactly what we
want.

Currently the discard operation is not added to the log. That's not
a big deal since losing discard requests cannot result in data
corruption. It would however result in disk space usage higher than
it should be. Thus adding log support to zvol_discard() is probably
a good idea for a future improvement.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-02-09 16:19:38 -08:00
Etienne Dechamps cb2d19010d Support the fallocate() file operation.
Currently only the (FALLOC_FL_PUNCH_HOLE) flag combination is
supported, since it's the only one that matches the behavior of
zfs_space(). This makes it pretty much useless in its current
form, but it's a start.

To support other flag combinations we would need to modify
zfs_space() to make it more flexible, or emulate the desired
functionality in zpl_fallocate().

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #334
2012-02-09 16:19:32 -08:00
Etienne Dechamps 34037afe24 Improve ZVOL queue behavior.
The Linux block device queue subsystem exposes a number of configurable
settings described in Linux block/blk-settings.c. The defaults for these
settings are tuned for hard drives, and are not optimized for ZVOLs. Proper
configuration of these options would allow upper layers (I/O scheduler) to
take better decisions about write merging and ordering.

Detailed rationale:

 - max_hw_sectors is set to unlimited (UINT_MAX). zvol_write() is able to
   handle writes of any size, so there's no reason to impose a limit. Let the
   upper layer decide.

 - max_segments and max_segment_size are set to unlimited. zvol_write() will
   copy the requests' contents into a dbuf anyway, so the number and size of
   the segments are irrelevant. Let the upper layer decide.

 - physical_block_size and io_opt are set to the ZVOL's block size. This
   has the potential to somewhat alleviate issue #361 for ZVOLs, by warning
   the upper layers that writes smaller than the volume's block size will be
   slow.

 - The NONROT flag is set to indicate this isn't a rotational device.
   Although the backing zpool might be composed of rotational devices, the
   resulting ZVOL often doesn't exhibit the same behavior due to the COW
   mechanisms used by ZFS. Setting this flag will prevent upper layers from
   making useless decisions (such as reordering writes) based on incorrect
   assumptions about the behavior of the ZVOL.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-02-07 16:23:06 -08:00
Etienne Dechamps b18019d2d8 Fix synchronicity for ZVOLs.
zvol_write() assumes that the write request must be written to stable storage
if rq_is_sync() is true. Unfortunately, this assumption is incorrect. Indeed,
"sync" does *not* mean what we think it means in the context of the Linux
block layer. This is well explained in linux/fs.h:

    WRITE:       A normal async write. Device will be plugged.
    WRITE_SYNC:  Synchronous write. Identical to WRITE, but passes down
                 the hint that someone will be waiting on this IO
                 shortly.
    WRITE_FLUSH: Like WRITE_SYNC but with preceding cache flush.
    WRITE_FUA:   Like WRITE_SYNC but data is guaranteed to be on
                 non-volatile media on completion.

In other words, SYNC does not *mean* that the write must be on stable storage
on completion. It just means that someone is waiting on us to complete the
write request. Thus triggering a ZIL commit for each SYNC write request on a
ZVOL is unnecessary and harmful for performance. To make matters worse, ZVOL
users have no way to express that they actually want data to be written to
stable storage, which means the ZIL is broken for ZVOLs.

The request for stable storage is expressed by the FUA flag, so we must
commit the ZIL after the write if the FUA flag is set. In addition, we must
commit the ZIL before the write if the FLUSH flag is set.

Also, we must inform the block layer that we actually support FLUSH and FUA.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-02-07 16:23:06 -08:00
Brian Behlendorf 47621f3d76 Linux 3.3 compat, sops->show_options()
The second argument of sops->show_options() was changed from a
'struct vfsmount *' to a 'struct dentry *'.  Add an autoconf check
to detect the API change and then conditionally define the expected
interface.  In either case we are only interested in the zfs_sb_t.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #549
2012-02-03 10:02:01 -08:00
Darik Horn 750562833f Combine libraries: spl, avl, efi, share, unicode.
These libraries, which are an artifact of the ZoL development
process, conflict with packages that are already in distribution:

  * libspl: SPL Programming Language
  * libavl: AVL for Linux
  * libefi: GRUB

And these libraries are potential conflicts:

  * libshare: the Linux Mount Manager
  * libunicode: Perl and Python

Recompose these five ZoL components into the four libraries that are
conventionally provided by Solaris and FreeBSD systems:

  + libnvpair
  + libuutil
  + libzpool
  + libzfs

This change resolves the name conflict, makes ZoL more compatible
with existing software that uses autotools to detect ZFS, and allows
pkg-zfs to better reflect the official Debian kFreeBSD packaging.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes: #430
2012-01-17 15:19:50 -08:00
Suman Chakravartula e18be9a637 Add overlay(-O) mount option support
Linux supports mounting over non-empty directories by default.
In Solaris this is not the case and -O option is required for
zfs mount to mount a zfs filesystem over a non-empty directory.

For compatibility, I've added support for -O option to mount
zfs filesystems over non-empty directories if the user wants
to, just like in Solaris.

I've defined MS_OVERLAY to record it in the flags variable if
the -O option is supplied.  The flags variable passes through
a few functions and its checked before performing the empty
directory check in zfs_mount function.  If -O is given, the
check is not performed.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #473
2012-01-12 15:49:38 -08:00
Darik Horn b97f368d04 Avoid using awk in the zpool_id script.
Some implementations of `awk` incorrectly parse the \< and \> regex
symbols, so use a `while read` loop and regular globbing instead.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes: #259
2012-01-11 11:56:56 -08:00
Brian Behlendorf ab26409db7 Linux 3.1 compat, super_block->s_shrink
The Linux 3.1 kernel has introduced the concept of per-filesystem
shrinkers which are directly assoicated with a super block.  Prior
to this change there was one shared global shrinker.

The zfs code relied on being able to call the global shrinker when
the arc_meta_limit was exceeded.  This would cause the VFS to drop
references on a fraction of the dentries in the dcache.  The ARC
could then safely reclaim the memory used by these entries and
honor the arc_meta_limit.  Unfortunately, when per-filesystem
shrinkers were added the old interfaces were made unavailable.

This change adds support to use the new per-filesystem shrinker
interface so we can continue to honor the arc_meta_limit.  The
major benefit of the new interface is that we can now target
only the zfs filesystem for dentry and inode pruning.  Thus we
can minimize any impact on the caching of other filesystems.

In the context of making this change several other important
issues related to managing the ARC were addressed, they include:

* The dnlc_reduce_cache() function which was called by the ARC
to drop dentries for the Posix layer was replaced with a generic
zfs_prune_t callback.  The ZPL layer now registers a callback to
drop these dentries removing a layering violation which dates
back to the Solaris code.  This callback can also be used by
other ARC consumers such as Lustre.

  arc_add_prune_callback()
  arc_remove_prune_callback()

* The arc_reduce_dnlc_percent module option has been changed to
arc_meta_prune for clarity.  The dnlc functions are specific to
Solaris's VFS and have already been largely eliminated already.
The replacement tunable now represents the number of bytes the
prune callback will request when invoked.

* Less aggressively invoke the prune callback.  We used to call
this whenever we exceeded the arc_meta_limit however that's not
strictly correct since it results in over zeleous reclaim of
dentries and inodes.  It is now only called once the arc_meta_limit
is exceeded and every effort has been made to evict other data from
the ARC cache.

* More promptly manage exceeding the arc_meta_limit.  When reading
meta data in to the cache if a buffer was unable to be recycled
notify the arc_reclaim thread to invoke the required prune.

* Added arcstat_prune kstat which is incremented when the ARC
is forced to request that a consumer prune its cache.  Remember
this will only occur when the ARC has no other choice.  If it
can evict buffers safely without invoking the prune callback
it will.

* This change is also expected to resolve the unexpect collapses
of the ARC cache.  This would occur because when exceeded just the
arc_meta_limit reclaim presure would be excerted on the arc_c
value via arc_shrink().  This effectively shrunk the entire cache
when really we just needed to reclaim meta data.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #466
Closes #292
2012-01-11 11:46:02 -08:00
Darik Horn afd7da0ce7 Add LIBSELINUX to mount_zfs_LDFLAGS.
Regenerating the autotools configuration on Debian and Ubuntu systems
causes compilation to fail with this error message:

  cmd/mount_zfs/../../cmd/mount_zfs/mount_zfs.c:403:
    undefined reference to `is_selinux_enabled'

In the automake template, set "mount_zfs_LDFLAGS = ... $(LIBSELINUX)"
so that the /sbin/mount.zfs utility is linked to libselinux.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2011-12-16 20:04:42 -08:00
Darik Horn 28eb9213d8 Linux 3.2 compat: set_nlink()
Directly changing inode->i_nlink is deprecated in Linux 3.2 by commit

  SHA: bfe8684869601dacfcb2cd69ef8cfd9045f62170

Use the new set_nlink() kernel function instead.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes: #462
2011-12-16 20:02:52 -08:00
Prakash Surya 6ba3b44614 Add make rule for building Arch Linux packages
Added the necessary build infrastructure for building packages
compatible with the Arch Linux distribution. As such, one can now run:

    $ ./configure
    $ make pkg     # Alternatively, one can run 'make arch' as well

on the Arch Linux machine to create two binary packages compatible with
the pacman package manager, one for the zfs userland utilities and
another for the zfs kernel modules. The new packages can then be
installed by running:

    # pacman -U $package.pkg.tar.xz

In addition, source-only packages suitable for an Arch Linux chroot
environment or remote builder can also be build using the 'sarch' make
rule.

NOTE: Since the source dist tarball is created on the fly from the head
of the build tree, it's MD5 hash signature will be continually influx.
As a result, the md5sum variable was intentionally omitted from the
PKGBUILD files, and the '--skipinteg' makepkg option is used. This may
or may not have any serious security implications, as the source tarball
is not being downloaded from an outside source.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #491
2011-12-14 19:14:23 -08:00
Darik Horn db7c1771da Demote the whackbang in the zpool_id script.
The zpool_id script is posixly correct and does not use bash
features, so change its whackbang from /bin/bash to /bin/sh.

Debian policy also stipulates that system scripts be dash compatible.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2011-12-05 09:48:18 -08:00
Darik Horn 87193e2b61 Demote egrep to grep in the zpool_id script.
Direct invocation of GNU egrep is deprecated by its man page, and the
its argument in the zpool_id script is not an extended expression.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2011-12-05 09:48:01 -08:00
Darik Horn 04bf5ecc1f Quote variables in the zpool_id script.
For consistency and safety, quote all variables in the zpool_id
script. This accomodates a `-c CONFIG` parameter value with
whitespace in the path name.

Also fix a typo in the usage synopsis for `-h`.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #439
2011-12-05 09:47:03 -08:00
Darik Horn 9c8254f6f9 Support path_id changes in udev 174.
The /lib/udev/path_id helper became a builtin command in the udev 174
release, so test whether path_id is external in the zpool_id script.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes: #429
2011-12-05 09:46:48 -08:00
Brian Behlendorf 5547c2f1bf Simplify BDI integration
Update the code to use the bdi_setup_and_register() helper to
simplify the bdi integration code.  The updated code now just
registers the bdi during mount and destroys it during unmount.

The only complication is that for 2.6.32 - 2.6.33 kernels the
helper wasn't available so in these cases the zfs code must
provide it.  Luckily the bdi_setup_and_register() function
is trivial.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #367
2011-11-08 10:19:03 -08:00
Darik Horn 3cee2262a6 Change sun.com URLs to zfsonlinux.org
ZFS contains error messages that point to the defunct www.sun.com
domain, which is currently offline.  Change these error messages
to use the zfsonlinux.org mirror instead.

This commit depends on:

  zfsonlinux/zfsonlinux.github.com@8e10ead3dc

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2011-10-24 09:52:21 -07:00
Brian Behlendorf 86f35f34f4 Export symbols for the VFS API
Export all symbols already marked extern in the zfs_vfsops.h
header.  Several non-static symbols have also been added to
the header and exportewd.  This allows external modules to
more easily create and manipulate properly created ZFS
filesystem type datasets.

Rename zfsvfs_teardown() to zfs_sb_teardown and export it.
This is done simply for consistency with the rest of the code
base.  All other zfsvfs_* functions have already been renamed.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2011-10-11 10:25:59 -07:00
Brian Behlendorf c70602f1ea Fix uninitialized varible in zfs_do_userspace()
When compiling under Debian Lenny with gcc version 4.3.2
(Debian 4.3.2-1.1) the following warning occurs.  To quiet
the warning initialize 'error' to zero.  Newer versions of
gcc correctly determine that this uninitialized varible is
impossible because ZFS_NUM_USERQUOTA_PROPS is known to be
greater than zero.

  cmd/zfs/zfs_main.c:2377: warning: "error" may be
  used uninitialized in this function

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2011-09-27 16:56:38 -07:00
Brian Behlendorf 4c069d3494 Fixed uninitialized variable
This warning was accidentally introduced by commit
b7936d5c2337bc976ac831c1c38de563844c36b.  The fix is to
simply initialize the variable to ZFS_DELEG_WHO_UNKNOWN.

  cmd/zfs/zfs_main.c:4460:25: warning: 'who_type' may be
  used uninitialized in this function

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2011-08-19 16:26:06 -07:00
Brian Behlendorf 29b35200a7 Fix missing format arguments
These warnings were accidentally introduced by commit
b7936d5c2337bc976ac831c1c38de563844c36b.  The fix is to
simply add the missing format specifier.

  cmd/zfs/zfs_main.c:4565: warning: format not a string
  literal and no format arguments

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2011-08-19 15:16:34 -07:00
Brian Behlendorf b740d602bd Disable zfs /etc/mtab updates
Completely disable the zfs binary from attempting to directly update
/etc/mtab.  The Linux port relies entirely on the mount.zfs helper
to safely update /etc/mtab.  If we left the /etc/mtab updates to
the zfs binary then they could race with concurrent non-zfs mounts.
Routing everything through the system mount command ensures the
/etc/mtab updates are locked properly.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #329
2011-08-19 11:53:01 -07:00
Brian Behlendorf de0a1c099b Autogen refresh for udev changes
Run autogen.sh using the same autotools versions as upstream:

 * autoconf-2.63
 * automake-1.11.1
 * libtool-2.2.6b
2011-08-08 16:30:27 -07:00
Kyle Fuller 12d06bac9b Move udev rules from /etc/udev to /lib/udev
This change moves the default install location for the zfs udev
rules from /etc/udev/ to /lib/udev/.  The correct convention is
for rules provided by a package to be installed in /lib/udev/.
The /etc/udev/ directory is reserved for custom rules or local
overrides.

Additionally, this patch cleans up some abuse of the bindir install
location by adding a udevdir and udevruledir install directories.
This allows us to revert to the default bin install location.  The
udev install directories can be set with the following new options.

  --with-udevdir=DIR      install udev helpers [EPREFIX/lib/udev]
  --with-udevruledir=DIR  install udev rules [UDEVDIR/rules.d]

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #356
2011-08-08 16:21:10 -07:00
Brian Behlendorf 76659dc110 Add backing_device_info per-filesystem
For a long time now the kernel has been moving away from using the
pdflush daemon to write 'old' dirty pages to disk.  The primary reason
for this is because the pdflush daemon is single threaded and can be
a limiting factor for performance.  Since pdflush sequentially walks
the dirty inode list for each super block any delay in processing can
slow down dirty page writeback for all filesystems.

The replacement for pdflush is called bdi (backing device info).  The
bdi system involves creating a per-filesystem control structure each
with its own private sets of queues to manage writeback.  The advantage
is greater parallelism which improves performance and prevents a single
filesystem from slowing writeback to the others.

For a long time both systems co-existed in the kernel so it wasn't
strictly required to implement the bdi scheme.  However, as of
Linux 2.6.36 kernels the pdflush functionality has been retired.

Since ZFS already bypasses the page cache for most I/O this is only
an issue for mmap(2) writes which must go through the page cache.
Even then adding this missing support for newer kernels was overlooked
because there are other mechanisms which can trigger writeback.

However, there is one critical case where not implementing the bdi
functionality can cause problems.  If an application handles a page
fault it can enter the balance_dirty_pages() callpath.  This will
result in the application hanging until the number of dirty pages in
the system drops below the dirty ratio.

Without a registered backing_device_info for the filesystem the
dirty pages will not get written out.  Thus the application will hang.
As mentioned above this was less of an issue with older kernels because
pdflush would eventually write out the dirty pages.

This change adds a backing_device_info structure to the zfs_sb_t
which is already allocated per-super block.  It is then registered
when the filesystem mounted and unregistered on unmount.  It will
not be registered for mounted snapshots which are read-only.  This
change will result in flush-<pool> thread being dynamically created
and destroyed per-mounted filesystem for writeback.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #174
2011-08-04 13:37:38 -07:00
Alexander Stetsenko 0b7936d5c2 Illumos #278: get rid zfs of python and pyzfs dependencies
Remove all python and pyzfs dependencies for consistency and
to ensure full functionality even in a mimimalist environment.

Reviewed by: gordon.w.ross@gmail.com
Reviewed by: trisk@opensolaris.org
Reviewed by: alexander.r.eremin@gmail.com
Reviewed by: jerry.jelinek@joyent.com
Approved by: garrett@nexenta.com

References to Illumos issue and patch:
- https://www.illumos.org/issues/278
- https://github.com/illumos/illumos-gate/commit/1af68beac3

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #340
Issue #160

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2011-08-01 12:09:36 -07:00
Eric Schrock 3e31d2b080 Illumos #883: ZIL reuse during remount corruption
Moving the zil_free() cleanup to zil_close() prevents this
problem from occurring in the first place.  There is a very
good description of the issue and fix in Illumus #883.

Reviewed by: Matt Ahrens <Matt.Ahrens@delphix.com>
Reviewed by: Adam Leventhal <Adam.Leventhal@delphix.com>
Reviewed by: Albert Lee <trisk@nexenta.com>
Reviewed by: Gordon Ross <gwr@nexenta.com>
Reviewed by: Garrett D'Amore <garrett@nexenta.com>
Reivewed by: Dan McDonald <danmcd@nexenta.com>
Approved by: Gordon Ross <gwr@nexenta.com>

References to Illumos issue and patch:
- https://www.illumos.org/issues/883
- https://github.com/illumos/illumos-gate/commit/c9ba2a43cb

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #340
2011-08-01 12:09:11 -07:00
George Wilson 6d974228ef Illumos #1051: zfs should handle imbalanced luns
Today zfs tries to allocate blocks evenly across all devices.
This means when devices are imbalanced zfs will use lots of
CPU searching for space on devices which tend to be pretty
full.  It should instead fail quickly on the full LUNs and
move onto devices which have more availability.

Reviewed by: Eric Schrock <Eric.Schrock@delphix.com>
Reviewed by: Matt Ahrens <Matt.Ahrens@delphix.com>
Reviewed by: Adam Leventhal <Adam.Leventhal@delphix.com>
Reviewed by: Albert Lee <trisk@nexenta.com>
Reviewed by: Gordon Ross <gwr@nexenta.com>
Approved by: Garrett D'Amore <garrett@nexenta.com>

References to Illumos issue and patch:
- https://www.illumos.org/issues/510
- https://github.com/illumos/illumos-gate/commit/5ead3ed965

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #340
2011-08-01 12:09:11 -07:00
Shampavman bb939d1085 Illumos #510: 'zfs get' enhancement - mountpoint as an argument
The 'zfs get' command should be able to deal with mountpoint
as an argument.  It already works with 'zfs list' command:

  # zfs list /export/home/estibi
  NAME                       USED  AVAIL  REFER  MOUNTPOINT
  rpool/export/home/estibi  1.14G  3.86G  1.14G  /export/home/estibi

but it fails with 'zfs get':

  # zfs get all /export/home/estibi
  cannot open '/export/home/estibi': invalid dataset name

Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: Deano <deano@rattie.demon.co.uk>
Reviewed by: Garrett D'Amore <garrett@nexenta.com>
Approved by: Garrett D'Amore <garrett@nexenta.com>

References to Illumos issue and patch:
- https://www.illumos.org/issues/510
- https://github.com/illumos/illumos-gate/commit/5ead3ed965

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #340
2011-08-01 12:09:11 -07:00
Kyle Fuller 615ab66d18 Provide a rc.d script for archlinux
Unlike most other Linux distributions archlinux installs its
init scripts in /etc/rc.d insead of /etc/init.d.  This commit
provides an archlinux rc.d script for zfs and extends the
build infrastructure to ensure it get's installed in the
correct place.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #322
2011-07-11 14:12:23 -07:00
Brian Behlendorf e0f86c9862 Update 'zfs send' documentation
The -D and -p options were missing from the manpage.  This commit
adds documentation for these features.

Closes #311
2011-07-08 12:16:09 -07:00
Brian Behlendorf 341b5f1d4c Update ztest paths
Unfortunately, ztest is hard coded to export the zdb utility to
be installed in a certain location.  When the packaging was updated
to install zdb in /sbin/ ztest was broken.  To fix this I'm updating
ztest to check both common install paths.
2011-07-06 12:30:09 -07:00
Prasad Joshi 5a52105925 Use consistent error message in zpool sub-command
The zpool sub-commands like iostat, list, and status should
display consistent message when a given pool is unavailable or
no pool is present.  This change unifies the default behavior
as follows:

  root@prasad:~# ./zpool list 1 2
  no pools available
  no pools available

  root@prasad:~# ./zpool iostat  1 2
  no pools available
  no pools available

  root@prasad:~# ./zpool status 1 2
  no pools available
  no pools available

  root@prasad:~# ./zpool list tan 1 2
  cannot open 'tan': no such pool

  root@prasad:~# ./zpool iostat tan 1 2
  cannot open 'tan': no such pool

  root@prasad:~# ./zpool status tan 1 2
  cannot open 'tan': no such pool

Reported-by: Rajshree Thorat <rthorat@stec-inc.com>
Signed-off-by: Prasad Joshi <pjoshi@stec-inc.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>

Closes #306
2011-07-04 19:57:01 -07:00
Brian Behlendorf 2cf7f52bc4 Linux compat 2.6.39: mount_nodev()
The .get_sb callback has been replaced by a .mount callback
in the file_system_type structure.  When using the new
interface the caller must now use the mount_nodev() helper.

Unfortunately, the new interface no longer passes the vfsmount
down to the zfs layers.  This poses a problem for the existing
implementation because we currently save this pointer in the
super block for latter use.  It provides our only entry point
in to the namespace layer for manipulating certain mount options.

This needed to be done originally to allow commands like
'zfs set atime=off tank' to work properly.  It also allowed me
to keep more of the original Solaris code unmodified.  Under
Solaris there is a 1-to-1 mapping between a mount point and a
file system so this is a fairly natural thing to do.  However,
under Linux they many be multiple entries in the namespace
which reference the same filesystem.  Thus keeping a back
reference from the filesystem to the namespace is complicated.

Rather than introduce some ugly hack to get the vfsmount and
continue as before.  I'm leveraging this API change to update
the ZFS code to do things in a more natural way for Linux.
This has the upside that is resolves the compatibility issue
for the long term and fixes several other minor bugs which
have been reported.

This commit updates the code to remove this vfsmount back
reference entirely.  All modifications to filesystem mount
options are now passed in to the kernel via a '-o remount'.
This is the expected Linux mechanism and allows the namespace
to properly handle any options which apply to it before passing
them on to the file system itself.

Aside from fixing the compatibility issue, removing the
vfsmount has had the benefit of simplifying the code.  This
change which fairly involved has turned out nicely.

Closes #246
Closes #217
Closes #187
Closes #248
Closes #231
2011-07-01 13:36:39 -07:00