This change should wrap up the last of the missing block device
support in the vdev_disk layer. With this change I can now
successfully create and use zpools which are layered on top of
md and lvm virtual devices. The following changes include:
1) The big one, properly handle the case when page cannot be added
to a bio due to dynamic limitation of a merge_bdev handler. For
example the md device will limit a bio to the configured stripe
size. Our bio size may also end up being limited by the maximum
request size, and other factors determined during bio construction.
To handle all of the above cases the code has been updated to
handle failures from bio_add_page(). This had been hardcoded to
never fail for the prototype proof of concept implementation. In
the case of a failure the number of bytes which still need to be
added to a bio are returned. New bio's are allocated and attached
to the dio until the entire data buffer is mapped to bios. It is
then submitted as before to the request queue, and once all the bio's
attached to a dio have finished the completion callback is run.
2) The devid comments have been removed because it is not clear to
me that we will not need devid support. They have been replaced
with a comment explaining that udev can and should be used.
Remove the hard coded 512 byte SECTOR_SIZE and replace it with
bdev_hardsect_size() to get the correct hardware sector size.
Usage of get_capacity() was incorrect. We the block_device
references a partition we need to return bdev->part->nr_sects.
If get_capacity() is used the entire device size will be returned
ignoring partition information. This is however the correct thing
to do when the block device in question has not partition table.
Exposed by the fc11 debug kernel we need to hold a reference over all
calls to submit_bio(). Otherwise it is possible all the completion
callbacks run before we exit __vdev_disk_physio(), and we end up with
a GPF. This was quickly exposed when slab poisoning was enabled. I
have added helper functions to cleanly track the reference counts. In
addition dr->dr_ref was converted from an integer to an atomic type
which removes the need for the spinlock. As a nice side effect of
these changes the code is now slightly cleaner and clearer.
With this patch applied I get the following failure 100% of the time,
I'd prefer to debug it and keep moving forward but I do not have the
time right now so I'm reverting the patch to the version which worked.
Ricardo please fix.
(gdb) bt
0 ztest_dmu_write_parallel (za=0x2aaaac898960) at
../../cmd/ztest/ztest.c:2566
1 0x0000000000405a79 in ztest_thread (arg=<value optimized out>)
at ../../cmd/ztest/ztest.c:3862
2 0x00002b2e6a7a841d in zk_thread_helper (arg=<value optimized out>)
at ../../lib/libzpool/kernel.c:131
3 0x000000379be06367 in start_thread (arg=<value optimized out>)
at pthread_create.c:297
4 0x000000379b2d30ad in clone () from /lib64/libc.so.6
This resolves previous scalabily concerns about the cost of calling
curthread which previously required a list walk. The kthread address
is now tracked as thread specific data which can be quickly returned.
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
There is concern that READA may do more than simply reorder the queue.
There may be an increased chance that a requested marked READA will
fail because the elevator considers it optional. For this reason, all
read requests, even speculative ones, have been converted back to READ.
Brian Behlendorf
Ricardo M. Correia