Archive-Team
/
zfs
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 103 Wiki Activity
zfs/include/sys
History
Brian Atkinson 7c8b7fe0f3 Fixing race condition with rangelocks 
There existed a race condition between when a Direct I/O write could
complete and if a sync operation was issued. This was due to the fact
that a Direct I/O would sleep waiting on previous TXG's to sync out
their dirty records assosciated with a dbuf if there was an ARC buffer
associated with the dbuf. This was necessay to safely destroy the ARC
buffer in case previous dirty records dr_data as pointed at that the
db_buf. The main issue with this approach is a Direct I/o write holds
the rangelock across the entire block, so when a sync on that same block
was issued and tried to grab the rangelock as reader, it would be
blocked indefinitely because the Direct I/O that was now sleeping was
holding that same rangelock as writer. This led to a complete deadlock.

This commit fixes this issue and removes the wait in
dmu_write_direct_done().

The way this is now handled is the ARC buffer is destroyed, if there an
associated one with dbuf, before ever issuing the Direct I/O write.
This implemenation heavily borrows from the block cloning
implementation.

A new function dmu_buf_wil_clone_or_dio() is called in both
dmu_write_direct() and dmu_brt_clone() that does the following:
1. Undirties a dirty record for that db if there one currently
   associated with the current TXG.
2. Destroys the ARC buffer if the previous dirty record dr_data does not
   point at the dbufs ARC buffer (db_buf).
3. Sets the dbufs data pointers to NULL.
4. Redirties the dbuf using db_state = DB_NOFILL.

As part of this commit, the dmu_write_direct_done() function was also
cleaned up. Now dmu_sync_done() is called before undirtying the dbuf
dirty record associated with a failed Direct I/O write. This is correct
logic and how it always should have been.

The additional benefits of these modifications is there is no longer a
stall in a Direct I/O write if the user is mixing bufferd and O_DIRECT
together. Also it unifies the block cloning and Direct I/O write path as
they both need to call dbuf_fix_old_data() before destroying the ARC
buffer.

As part of this commit, there is also just general code cleanup. Various
dbuf stats were removed because they are not necesary any longer.
Additionally, useless functions were removed to make the code paths
cleaner for Direct I/O.

Below is the race condtion stack trace that was being consistently
observed in the CI runs for the dio_random test case that prompted
these changes:
trace:
[ 7795.294473] sd 0:0:0:0: [sda] Synchronizing SCSI cache
[ 9954.769075] INFO: task z_wr_int:1051869 blocked for more than
120
seconds.
[ 9954.770512]       Tainted: P           OE     -------- -  -
4.18.0-553.5.1.el8_10.x86_64 #1
[ 9954.772159] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[ 9954.773848] task:z_wr_int        state:D stack:0
pid:1051869
ppid:2      flags:0x80004080
[ 9954.775512] Call Trace:
[ 9954.776406]  __schedule+0x2d1/0x870
[ 9954.777386]  ? free_one_page+0x204/0x530
[ 9954.778466]  schedule+0x55/0xf0
[ 9954.779355]  cv_wait_common+0x16d/0x280 [spl]
[ 9954.780491]  ? finish_wait+0x80/0x80
[ 9954.781450]  dmu_buf_direct_mixed_io_wait+0x84/0x1a0 [zfs]
[ 9954.782889]  dmu_write_direct_done+0x90/0x3b0 [zfs]
[ 9954.784255]  zio_done+0x373/0x1d50 [zfs]
[ 9954.785410]  zio_execute+0xee/0x210 [zfs]
[ 9954.786588]  taskq_thread+0x205/0x3f0 [spl]
[ 9954.787673]  ? wake_up_q+0x60/0x60
[ 9954.788571]  ? zio_execute_stack_check.constprop.1+0x10/0x10
[zfs]
[ 9954.790079]  ? taskq_lowest_id+0xc0/0xc0 [spl]
[ 9954.791199]  kthread+0x134/0x150
[ 9954.792082]  ? set_kthread_struct+0x50/0x50
[ 9954.793189]  ret_from_fork+0x35/0x40
[ 9954.794108] INFO: task txg_sync:1051894 blocked for more than
120
seconds.
[ 9954.795535]       Tainted: P           OE     -------- -  -
4.18.0-553.5.1.el8_10.x86_64 #1
[ 9954.797103] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[ 9954.798669] task:txg_sync        state:D stack:0
pid:1051894
ppid:2      flags:0x80004080
[ 9954.800267] Call Trace:
[ 9954.801096]  __schedule+0x2d1/0x870
[ 9954.801972]  ? __wake_up_common+0x7a/0x190
[ 9954.802963]  schedule+0x55/0xf0
[ 9954.803884]  schedule_timeout+0x19f/0x320
[ 9954.804837]  ? __next_timer_interrupt+0xf0/0xf0
[ 9954.805932]  ? taskq_dispatch+0xab/0x280 [spl]
[ 9954.806959]  io_schedule_timeout+0x19/0x40
[ 9954.807989]  __cv_timedwait_common+0x19e/0x2c0 [spl]
[ 9954.809110]  ? finish_wait+0x80/0x80
[ 9954.810068]  __cv_timedwait_io+0x15/0x20 [spl]
[ 9954.811103]  zio_wait+0x1ad/0x4f0 [zfs]
[ 9954.812255]  dsl_pool_sync+0xcb/0x6c0 [zfs]
[ 9954.813442]  ? spa_errlog_sync+0x2f0/0x3d0 [zfs]
[ 9954.814648]  spa_sync_iterate_to_convergence+0xcb/0x310 [zfs]
[ 9954.816023]  spa_sync+0x362/0x8f0 [zfs]
[ 9954.817110]  txg_sync_thread+0x27a/0x3b0 [zfs]
[ 9954.818267]  ? txg_dispatch_callbacks+0xf0/0xf0 [zfs]
[ 9954.819510]  ? spl_assert.constprop.0+0x20/0x20 [spl]
[ 9954.820643]  thread_generic_wrapper+0x63/0x90 [spl]
[ 9954.821709]  kthread+0x134/0x150
[ 9954.822590]  ? set_kthread_struct+0x50/0x50
[ 9954.823584]  ret_from_fork+0x35/0x40
[ 9954.824444] INFO: task fio:1055501 blocked for more than 120
seconds.
[ 9954.825781]       Tainted: P           OE     -------- -  -
4.18.0-553.5.1.el8_10.x86_64 #1
[ 9954.827315] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[ 9954.828871] task:fio             state:D stack:0
pid:1055501
ppid:1055490 flags:0x00004080
[ 9954.830463] Call Trace:
[ 9954.831280]  __schedule+0x2d1/0x870
[ 9954.832159]  ? dbuf_hold_copy+0xec/0x230 [zfs]
[ 9954.833396]  schedule+0x55/0xf0
[ 9954.834286]  cv_wait_common+0x16d/0x280 [spl]
[ 9954.835291]  ? finish_wait+0x80/0x80
[ 9954.836235]  zfs_rangelock_enter_reader+0xa1/0x1f0 [zfs]
[ 9954.837543]  zfs_rangelock_enter_impl+0xbf/0x1b0 [zfs]
[ 9954.838838]  zfs_get_data+0x566/0x810 [zfs]
[ 9954.840034]  zil_lwb_commit+0x194/0x3f0 [zfs]
[ 9954.841154]  zil_lwb_write_issue+0x68/0xb90 [zfs]
[ 9954.842367]  ? __list_add+0x12/0x30 [zfs]
[ 9954.843496]  ? __raw_spin_unlock+0x5/0x10 [zfs]
[ 9954.844665]  ? zil_alloc_lwb+0x217/0x360 [zfs]
[ 9954.845852]  zil_commit_waiter_timeout+0x1f3/0x570 [zfs]
[ 9954.847203]  zil_commit_waiter+0x1d2/0x3b0 [zfs]
[ 9954.848380]  zil_commit_impl+0x6d/0xd0 [zfs]
[ 9954.849550]  zfs_fsync+0x66/0x90 [zfs]
[ 9954.850640]  zpl_fsync+0xe5/0x140 [zfs]
[ 9954.851729]  do_fsync+0x38/0x70
[ 9954.852585]  __x64_sys_fsync+0x10/0x20
[ 9954.853486]  do_syscall_64+0x5b/0x1b0
[ 9954.854416]  entry_SYSCALL_64_after_hwframe+0x61/0xc6
[ 9954.855466] RIP: 0033:0x7eff236bb057
[ 9954.856388] Code: Unable to access opcode bytes at RIP
0x7eff236bb02d.
[ 9954.857651] RSP: 002b:00007ffffb8e5320 EFLAGS: 00000293
ORIG_RAX:
000000000000004a
[ 9954.859141] RAX: ffffffffffffffda RBX: 0000000000000006 RCX:
00007eff236bb057
[ 9954.860496] RDX: 0000000000000000 RSI: 000055e4d1f13ac0 RDI:
0000000000000006
[ 9954.861945] RBP: 00007efeb8ed8000 R08: 0000000000000000 R09:
0000000000000000
[ 9954.863327] R10: 0000000000056000 R11: 0000000000000293 R12:
0000000000000003
[ 9954.864765] R13: 000055e4d1f13ac0 R14: 0000000000000000 R15:
000055e4d1f13ae8
[ 9954.866149] INFO: task fio:1055502 blocked for more than 120
seconds.
[ 9954.867490]       Tainted: P           OE     -------- -  -
4.18.0-553.5.1.el8_10.x86_64 #1
[ 9954.869029] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[ 9954.870571] task:fio             state:D stack:0
pid:1055502
ppid:1055490 flags:0x00004080
[ 9954.872162] Call Trace:
[ 9954.872947]  __schedule+0x2d1/0x870
[ 9954.873844]  schedule+0x55/0xf0
[ 9954.874716]  schedule_timeout+0x19f/0x320
[ 9954.875645]  ? __next_timer_interrupt+0xf0/0xf0
[ 9954.876722]  io_schedule_timeout+0x19/0x40
[ 9954.877677]  __cv_timedwait_common+0x19e/0x2c0 [spl]
[ 9954.878822]  ? finish_wait+0x80/0x80
[ 9954.879694]  __cv_timedwait_io+0x15/0x20 [spl]
[ 9954.880763]  zio_wait+0x1ad/0x4f0 [zfs]
[ 9954.881865]  dmu_write_abd+0x174/0x1c0 [zfs]
[ 9954.883074]  dmu_write_uio_direct+0x79/0x100 [zfs]
[ 9954.884285]  dmu_write_uio_dnode+0xb2/0x320 [zfs]
[ 9954.885507]  dmu_write_uio_dbuf+0x47/0x60 [zfs]
[ 9954.886687]  zfs_write+0x581/0xe20 [zfs]
[ 9954.887822]  ? iov_iter_get_pages+0xe9/0x390
[ 9954.888862]  ? trylock_page+0xd/0x20 [zfs]
[ 9954.890005]  ? __raw_spin_unlock+0x5/0x10 [zfs]
[ 9954.891217]  ? zfs_setup_direct+0x7e/0x1b0 [zfs]
[ 9954.892391]  zpl_iter_write_direct+0xd4/0x170 [zfs]
[ 9954.893663]  ? rrw_exit+0xc6/0x200 [zfs]
[ 9954.894764]  zpl_iter_write+0xd5/0x110 [zfs]
[ 9954.895911]  new_sync_write+0x112/0x160
[ 9954.896881]  vfs_write+0xa5/0x1b0
[ 9954.897701]  ksys_write+0x4f/0xb0
[ 9954.898569]  do_syscall_64+0x5b/0x1b0
[ 9954.899417]  entry_SYSCALL_64_after_hwframe+0x61/0xc6
[ 9954.900515] RIP: 0033:0x7eff236baa47
[ 9954.901363] Code: Unable to access opcode bytes at RIP
0x7eff236baa1d.
[ 9954.902673] RSP: 002b:00007ffffb8e5330 EFLAGS: 00000293
ORIG_RAX:
0000000000000001
[ 9954.904099] RAX: ffffffffffffffda RBX: 0000000000000005 RCX:
00007eff236baa47
[ 9954.905535] RDX: 00000000000e4000 RSI: 00007efeb7dd4000 RDI:
0000000000000005
[ 9954.906902] RBP: 00007efeb7dd4000 R08: 0000000000000000 R09:
0000000000000000
[ 9954.908339] R10: 0000000000000000 R11: 0000000000000293 R12:
00000000000e4000
[ 9954.909705] R13: 000055e4d1f13ac0 R14: 00000000000e4000 R15:
000055e4d1f13ae8
[ 9954.911129] INFO: task fio:1055504 blocked for more than 120
seconds.
[ 9954.912381]       Tainted: P           OE     -------- -  -
4.18.0-553.5.1.el8_10.x86_64 #1
[ 9954.913978] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[ 9954.915434] task:fio             state:D stack:0
pid:1055504
ppid:1055493 flags:0x00000080
[ 9954.917082] Call Trace:
[ 9954.917773]  __schedule+0x2d1/0x870
[ 9954.918648]  ? zilog_dirty+0x4f/0xc0 [zfs]
[ 9954.919831]  schedule+0x55/0xf0
[ 9954.920717]  cv_wait_common+0x16d/0x280 [spl]
[ 9954.921704]  ? finish_wait+0x80/0x80
[ 9954.922639]  zfs_rangelock_enter_writer+0x46/0x1c0 [zfs]
[ 9954.923940]  zfs_rangelock_enter_impl+0x12a/0x1b0 [zfs]
[ 9954.925306]  zfs_write+0x703/0xe20 [zfs]
[ 9954.926406]  zpl_iter_write_buffered+0xb2/0x120 [zfs]
[ 9954.927687]  ? rrw_exit+0xc6/0x200 [zfs]
[ 9954.928821]  zpl_iter_write+0xbe/0x110 [zfs]
[ 9954.930028]  new_sync_write+0x112/0x160
[ 9954.930913]  vfs_write+0xa5/0x1b0
[ 9954.931758]  ksys_write+0x4f/0xb0
[ 9954.932666]  do_syscall_64+0x5b/0x1b0
[ 9954.933544]  entry_SYSCALL_64_after_hwframe+0x61/0xc6
[ 9954.934689] RIP: 0033:0x7fcaee8f0a47
[ 9954.935551] Code: Unable to access opcode bytes at RIP
0x7fcaee8f0a1d.
[ 9954.936893] RSP: 002b:00007fff56b2c240 EFLAGS: 00000293
ORIG_RAX:
0000000000000001
[ 9954.938327] RAX: ffffffffffffffda RBX: 0000000000000006 RCX:
00007fcaee8f0a47
[ 9954.939777] RDX: 000000000001d000 RSI: 00007fca8300b010 RDI:
0000000000000006
[ 9954.941187] RBP: 00007fca8300b010 R08: 0000000000000000 R09:
0000000000000000
[ 9954.942655] R10: 0000000000000000 R11: 0000000000000293 R12:
000000000001d000
[ 9954.944062] R13: 0000557a2006bac0 R14: 000000000001d000 R15:
0000557a2006bae8
[ 9954.945525] INFO: task fio:1055505 blocked for more than 120
seconds.
[ 9954.946819]       Tainted: P           OE     -------- -  -
4.18.0-553.5.1.el8_10.x86_64 #1
[ 9954.948466] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[ 9954.949959] task:fio             state:D stack:0
pid:1055505
ppid:1055493 flags:0x00004080
[ 9954.951653] Call Trace:
[ 9954.952417]  __schedule+0x2d1/0x870
[ 9954.953393]  ? finish_wait+0x3e/0x80
[ 9954.954315]  schedule+0x55/0xf0
[ 9954.955212]  cv_wait_common+0x16d/0x280 [spl]
[ 9954.956211]  ? finish_wait+0x80/0x80
[ 9954.957159]  zil_commit_waiter+0xfa/0x3b0 [zfs]
[ 9954.958343]  zil_commit_impl+0x6d/0xd0 [zfs]
[ 9954.959524]  zfs_fsync+0x66/0x90 [zfs]
[ 9954.960626]  zpl_fsync+0xe5/0x140 [zfs]
[ 9954.961763]  do_fsync+0x38/0x70
[ 9954.962638]  __x64_sys_fsync+0x10/0x20
[ 9954.963520]  do_syscall_64+0x5b/0x1b0
[ 9954.964470]  entry_SYSCALL_64_after_hwframe+0x61/0xc6
[ 9954.965567] RIP: 0033:0x7fcaee8f1057
[ 9954.966490] Code: Unable to access opcode bytes at RIP
0x7fcaee8f102d.
[ 9954.967752] RSP: 002b:00007fff56b2c230 EFLAGS: 00000293
ORIG_RAX:
000000000000004a
[ 9954.969260] RAX: ffffffffffffffda RBX: 0000000000000005 RCX:
00007fcaee8f1057
[ 9954.970628] RDX: 0000000000000000 RSI: 0000557a2006bac0 RDI:
0000000000000005
[ 9954.972092] RBP: 00007fca84152a18 R08: 0000000000000000 R09:
0000000000000000
[ 9954.973484] R10: 0000000000035000 R11: 0000000000000293 R12:
0000000000000003
[ 9954.974958] R13: 0000557a2006bac0 R14: 0000000000000000 R15:
0000557a2006bae8
[10077.648150] INFO: task z_wr_int:1051869 blocked for more than
120
seconds.
[10077.649541]       Tainted: P           OE     -------- -  -
4.18.0-553.5.1.el8_10.x86_64 #1
[10077.651116] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[10077.652782] task:z_wr_int        state:D stack:0
pid:1051869
ppid:2      flags:0x80004080
[10077.654420] Call Trace:
[10077.655267]  __schedule+0x2d1/0x870
[10077.656179]  ? free_one_page+0x204/0x530
[10077.657192]  schedule+0x55/0xf0
[10077.658004]  cv_wait_common+0x16d/0x280 [spl]
[10077.659018]  ? finish_wait+0x80/0x80
[10077.660013]  dmu_buf_direct_mixed_io_wait+0x84/0x1a0 [zfs]
[10077.661396]  dmu_write_direct_done+0x90/0x3b0 [zfs]
[10077.662617]  zio_done+0x373/0x1d50 [zfs]
[10077.663783]  zio_execute+0xee/0x210 [zfs]
[10077.664921]  taskq_thread+0x205/0x3f0 [spl]
[10077.665982]  ? wake_up_q+0x60/0x60
[10077.666842]  ? zio_execute_stack_check.constprop.1+0x10/0x10
[zfs]
[10077.668295]  ? taskq_lowest_id+0xc0/0xc0 [spl]
[10077.669360]  kthread+0x134/0x150
[10077.670191]  ? set_kthread_struct+0x50/0x50
[10077.671209]  ret_from_fork+0x35/0x40
[10077.672076] INFO: task txg_sync:1051894 blocked for more than
120
seconds.
[10077.673467]       Tainted: P           OE     -------- -  -
4.18.0-553.5.1.el8_10.x86_64 #1
[10077.675112] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[10077.676612] task:txg_sync        state:D stack:0
pid:1051894
ppid:2      flags:0x80004080
[10077.678288] Call Trace:
[10077.679024]  __schedule+0x2d1/0x870
[10077.679948]  ? __wake_up_common+0x7a/0x190
[10077.681042]  schedule+0x55/0xf0
[10077.681899]  schedule_timeout+0x19f/0x320
[10077.682951]  ? __next_timer_interrupt+0xf0/0xf0
[10077.684005]  ? taskq_dispatch+0xab/0x280 [spl]
[10077.685085]  io_schedule_timeout+0x19/0x40
[10077.686080]  __cv_timedwait_common+0x19e/0x2c0 [spl]
[10077.687227]  ? finish_wait+0x80/0x80
[10077.688123]  __cv_timedwait_io+0x15/0x20 [spl]
[10077.689206]  zio_wait+0x1ad/0x4f0 [zfs]
[10077.690300]  dsl_pool_sync+0xcb/0x6c0 [zfs]
[10077.691435]  ? spa_errlog_sync+0x2f0/0x3d0 [zfs]
[10077.692636]  spa_sync_iterate_to_convergence+0xcb/0x310 [zfs]
[10077.693997]  spa_sync+0x362/0x8f0 [zfs]
[10077.695112]  txg_sync_thread+0x27a/0x3b0 [zfs]
[10077.696239]  ? txg_dispatch_callbacks+0xf0/0xf0 [zfs]
[10077.697512]  ? spl_assert.constprop.0+0x20/0x20 [spl]
[10077.698639]  thread_generic_wrapper+0x63/0x90 [spl]
[10077.699687]  kthread+0x134/0x150
[10077.700567]  ? set_kthread_struct+0x50/0x50
[10077.701502]  ret_from_fork+0x35/0x40
[10077.702430] INFO: task fio:1055501 blocked for more than 120
seconds.
[10077.703697]       Tainted: P           OE     -------- -  -
4.18.0-553.5.1.el8_10.x86_64 #1
[10077.705309] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[10077.706780] task:fio             state:D stack:0
pid:1055501
ppid:1055490 flags:0x00004080
[10077.708479] Call Trace:
[10077.709231]  __schedule+0x2d1/0x870
[10077.710190]  ? dbuf_hold_copy+0xec/0x230 [zfs]
[10077.711368]  schedule+0x55/0xf0
[10077.712286]  cv_wait_common+0x16d/0x280 [spl]
[10077.713316]  ? finish_wait+0x80/0x80
[10077.714262]  zfs_rangelock_enter_reader+0xa1/0x1f0 [zfs]
[10077.715566]  zfs_rangelock_enter_impl+0xbf/0x1b0 [zfs]
[10077.716878]  zfs_get_data+0x566/0x810 [zfs]
[10077.718032]  zil_lwb_commit+0x194/0x3f0 [zfs]
[10077.719234]  zil_lwb_write_issue+0x68/0xb90 [zfs]
[10077.720413]  ? __list_add+0x12/0x30 [zfs]
[10077.721525]  ? __raw_spin_unlock+0x5/0x10 [zfs]
[10077.722708]  ? zil_alloc_lwb+0x217/0x360 [zfs]
[10077.723931]  zil_commit_waiter_timeout+0x1f3/0x570 [zfs]
[10077.725273]  zil_commit_waiter+0x1d2/0x3b0 [zfs]
[10077.726438]  zil_commit_impl+0x6d/0xd0 [zfs]
[10077.727586]  zfs_fsync+0x66/0x90 [zfs]
[10077.728675]  zpl_fsync+0xe5/0x140 [zfs]
[10077.729755]  do_fsync+0x38/0x70
[10077.730607]  __x64_sys_fsync+0x10/0x20
[10077.731482]  do_syscall_64+0x5b/0x1b0
[10077.732415]  entry_SYSCALL_64_after_hwframe+0x61/0xc6
[10077.733487] RIP: 0033:0x7eff236bb057
[10077.734399] Code: Unable to access opcode bytes at RIP
0x7eff236bb02d.
[10077.735657] RSP: 002b:00007ffffb8e5320 EFLAGS: 00000293
ORIG_RAX:
000000000000004a
[10077.737163] RAX: ffffffffffffffda RBX: 0000000000000006 RCX:
00007eff236bb057
[10077.738526] RDX: 0000000000000000 RSI: 000055e4d1f13ac0 RDI:
0000000000000006
[10077.739966] RBP: 00007efeb8ed8000 R08: 0000000000000000 R09:
0000000000000000
[10077.741336] R10: 0000000000056000 R11: 0000000000000293 R12:
0000000000000003
[10077.742773] R13: 000055e4d1f13ac0 R14: 0000000000000000 R15:
000055e4d1f13ae8
[10077.744168] INFO: task fio:1055502 blocked for more than 120
seconds.
[10077.745505]       Tainted: P           OE     -------- -  -
4.18.0-553.5.1.el8_10.x86_64 #1
[10077.747073] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[10077.748642] task:fio             state:D stack:0
pid:1055502
ppid:1055490 flags:0x00004080
[10077.750233] Call Trace:
[10077.751011]  __schedule+0x2d1/0x870
[10077.751915]  schedule+0x55/0xf0
[10077.752811]  schedule_timeout+0x19f/0x320
[10077.753762]  ? __next_timer_interrupt+0xf0/0xf0
[10077.754824]  io_schedule_timeout+0x19/0x40
[10077.755782]  __cv_timedwait_common+0x19e/0x2c0 [spl]
[10077.756922]  ? finish_wait+0x80/0x80
[10077.757788]  __cv_timedwait_io+0x15/0x20 [spl]
[10077.758845]  zio_wait+0x1ad/0x4f0 [zfs]
[10077.759941]  dmu_write_abd+0x174/0x1c0 [zfs]
[10077.761144]  dmu_write_uio_direct+0x79/0x100 [zfs]
[10077.762327]  dmu_write_uio_dnode+0xb2/0x320 [zfs]
[10077.763523]  dmu_write_uio_dbuf+0x47/0x60 [zfs]
[10077.764749]  zfs_write+0x581/0xe20 [zfs]
[10077.765825]  ? iov_iter_get_pages+0xe9/0x390
[10077.766842]  ? trylock_page+0xd/0x20 [zfs]
[10077.767956]  ? __raw_spin_unlock+0x5/0x10 [zfs]
[10077.769189]  ? zfs_setup_direct+0x7e/0x1b0 [zfs]
[10077.770343]  zpl_iter_write_direct+0xd4/0x170 [zfs]
[10077.771570]  ? rrw_exit+0xc6/0x200 [zfs]
[10077.772674]  zpl_iter_write+0xd5/0x110 [zfs]
[10077.773834]  new_sync_write+0x112/0x160
[10077.774805]  vfs_write+0xa5/0x1b0
[10077.775634]  ksys_write+0x4f/0xb0
[10077.776526]  do_syscall_64+0x5b/0x1b0
[10077.777386]  entry_SYSCALL_64_after_hwframe+0x61/0xc6
[10077.778488] RIP: 0033:0x7eff236baa47
[10077.779339] Code: Unable to access opcode bytes at RIP
0x7eff236baa1d.
[10077.780655] RSP: 002b:00007ffffb8e5330 EFLAGS: 00000293
ORIG_RAX:
0000000000000001
[10077.782056] RAX: ffffffffffffffda RBX: 0000000000000005 RCX:
00007eff236baa47
[10077.783507] RDX: 00000000000e4000 RSI: 00007efeb7dd4000 RDI:
0000000000000005
[10077.784890] RBP: 00007efeb7dd4000 R08: 0000000000000000 R09:
0000000000000000
[10077.786303] R10: 0000000000000000 R11: 0000000000000293 R12:
00000000000e4000
[10077.787637] R13: 000055e4d1f13ac0 R14: 00000000000e4000 R15:
000055e4d1f13ae8

Signed-off-by: Brian Atkinson <batkinson@lanl.gov>
 		2024-09-04 17:08:56 -06:00
..
crypto icp: remove skein module 2024-05-31 15:13:39 -07:00
fm Adding Direct IO Support 2024-09-04 17:06:56 -06:00
fs Adding Direct IO Support 2024-09-04 17:06:56 -06:00
lua autoconf: single-step includes 2022-05-10 10:18:51 -07:00
sysevent Teach zpool scrub to scrub only blocks in error log 2023-05-18 11:59:42 -07:00
zstd compress: change compression providers API to use ABDs 2024-08-22 16:22:24 -07:00
abd.h Adding Direct IO Support 2024-09-04 17:06:56 -06:00
abd_impl.h Adding Direct IO Support 2024-09-04 17:06:56 -06:00
aggsum.h More aggsum optimizations 2021-06-07 09:02:47 -07:00
arc.h Adding Direct IO Support 2024-09-04 17:06:56 -06:00
arc_impl.h Several improvements to ARC shrinking (#16197) 2024-07-25 10:31:14 -07:00
asm_linkage.h Unify Assembler files between Linux and Windows 2023-01-17 11:09:19 -08:00
avl.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
avl_impl.h Make sure avl_tree.avl_pad is not in kernel module (#16280) 2024-07-17 13:54:11 -07:00
bitmap.h Replace dead opensolaris.org license links 2023-03-14 14:44:01 -07:00
bitops.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
blake3.h Update BLAKE3 for using the new impl handling 2023-03-02 13:52:27 -08:00
blkptr.h OpenZFS 8067 - zdb should be able to dump literal embedded block pointer 2017-07-07 11:28:01 -07:00
bplist.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
bpobj.h Add explicit prefetches to bpobj_iterate(). 2023-07-21 11:50:48 -07:00
bptree.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
bqueue.h Batch enqueue/dequeue for bqueue 2023-01-10 13:39:22 -08:00
brt.h zdb: include cloned blocks in block statistics 2023-08-01 08:56:30 -07:00
brt_impl.h brt: lift internal definitions into _impl header 2023-11-27 13:34:43 -08:00
btree.h btree: Implement faster binary search algorithm 2023-05-26 10:03:12 -07:00
dataset_kstats.h Update the kstat dataset_name when renaming a zvol 2023-11-07 11:34:50 -08:00
dbuf.h Fixing race condition with rangelocks 2024-09-04 17:08:56 -06:00
ddt.h Add DDT prune command 2024-09-04 14:17:02 -07:00
ddt_impl.h Add DDT prune command 2024-09-04 14:17:02 -07:00
dmu.h Fixing race condition with rangelocks 2024-09-04 17:08:56 -06:00
dmu_impl.h Adding Direct IO Support 2024-09-04 17:06:56 -06:00
dmu_objset.h Adding Direct IO Support 2024-09-04 17:06:56 -06:00
dmu_recv.h nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
dmu_redact.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
dmu_send.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
dmu_traverse.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
dmu_tx.h Add dmu_tx_hold_append() interface 2023-05-09 09:03:10 -07:00
dmu_zfetch.h Speculative prefetch for reordered requests 2024-04-08 15:13:27 -07:00
dnode.h dnode: allow storage class to be overridden by object type 2024-07-29 17:05:41 -07:00
dsl_bookmark.h Increase limit of redaction list by using spill block 2023-08-26 11:34:43 -07:00
dsl_crypt.h Allow block cloning across encrypted datasets 2023-12-05 11:03:48 -08:00
dsl_dataset.h Revert zfeature_active() to static 2023-02-28 14:03:52 -08:00
dsl_deadlist.h Cleanup: 64-bit kernel module parameters should use fixed width types 2022-10-13 10:03:29 -07:00
dsl_deleg.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
dsl_destroy.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
dsl_dir.h Cleanup ->dd_space_towrite should be unsigned 2023-01-20 11:10:15 -08:00
dsl_pool.h Cleanup: 64-bit kernel module parameters should use fixed width types 2022-10-13 10:03:29 -07:00
dsl_prop.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
dsl_scan.h ddt: add "flat phys" feature 2024-08-16 12:02:39 -07:00
dsl_synctask.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
dsl_userhold.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
edonr.h Remove unused constant EdonR256_BLOCK_BITSIZE 2023-03-22 08:39:48 -07:00
efi_partition.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
frame.h Linux 5.10 compat: frame.h renamed objtool.h 2020-11-02 22:01:10 +00:00
hkdf.h Encryption patch follow-up 2017-10-11 16:54:48 -04:00
metaslab.h Selectable block allocators 2023-09-01 18:00:30 -07:00
metaslab_impl.h Reduce number of metaslab preload taskq threads. 2023-10-06 09:04:00 -07:00
mmp.h Cleanup: 64-bit kernel module parameters should use fixed width types 2022-10-13 10:03:29 -07:00
mntent.h Expose ZFS dataset case sensitivity setting via sb_opts 2022-07-14 10:38:16 -07:00
mod.h linux: module: weld all but spl.ko into zfs.ko 2022-04-20 13:28:24 -07:00
multilist.h L2ARC: Relax locking during write 2024-04-09 16:23:19 -07:00
nvpair.h nvpair: Use flexible array member for nvpair name strings 2023-03-14 15:25:55 -07:00
nvpair_impl.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
objlist.h Implement Redacted Send/Receive 2019-06-19 09:48:12 -07:00
pathname.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
qat.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
range_tree.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
rrwlock.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
sa.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
sa_impl.h Fix sa_add_projid to lookup and update SA_ZPL_DXATTR (avoid DXATTR loss) (#16288) 2024-07-31 18:41:49 -07:00
sha2.h icp: remove unused SHA2 HMAC mechanisms 2024-05-31 15:13:30 -07:00
skein.h icp: remove digest entry points 2024-05-31 15:13:16 -07:00
spa.h Adding Direct IO Support 2024-09-04 17:06:56 -06:00
spa_checkpoint.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
spa_checksum.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
spa_impl.h Add DDT prune command 2024-09-04 14:17:02 -07:00
spa_log_spacemap.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
space_map.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
space_reftree.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
sysevent.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
txg.h Cleanup: Specify unsignedness on things that should not be signed 2022-09-27 16:42:41 -07:00
txg_impl.h Properly pad struct tx_cpu to cache line 2023-10-20 11:54:05 -07:00
u8_textprep.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
u8_textprep_data.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
uberblock.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
uberblock_impl.h vdev probe to slow disk can stall mmp write checker 2024-04-29 14:35:53 -07:00
uio_impl.h Adding Direct IO Support 2024-09-04 17:06:56 -06:00
unique.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
uuid.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev.h RAID-Z expansion feature 2023-11-08 10:19:41 -08:00
vdev_disk.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_draid.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_file.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_impl.h Adding Direct IO Support 2024-09-04 17:06:56 -06:00
vdev_indirect_births.h OpenZFS 7614, 9064 - zfs device evacuation/removal 2018-04-14 12:16:17 -07:00
vdev_indirect_mapping.h OpenZFS 7614, 9064 - zfs device evacuation/removal 2018-04-14 12:16:17 -07:00
vdev_initialize.h Add the ability to uninitialize 2023-05-18 10:02:20 -07:00
vdev_raidz.h RAID-Z expansion feature 2023-11-08 10:19:41 -08:00
vdev_raidz_impl.h Workaround UBSAN errors for variable arrays 2023-11-12 16:26:07 -08:00
vdev_rebuild.h Do not report bytes skipped by scan as issued. 2023-06-30 08:47:13 -07:00
vdev_removal.h Cleanup: Specify unsignedness on things that should not be signed 2022-09-27 16:42:41 -07:00
vdev_trim.h Fix short-lived txg caused by autotrim 2023-03-28 08:43:41 -07:00
xvattr.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zap.h ddt: add support for prefetching tables into the ARC 2024-07-26 09:16:18 -07:00
zap_impl.h ZAP: Massively switch to _by_dnode() interfaces 2024-03-25 14:58:50 -07:00
zap_leaf.h zap_leaf: make l_hash[] variable length to silence UBSAN 2024-04-03 16:38:18 -07:00
zcp.h Cleanup: 64-bit kernel module parameters should use fixed width types 2022-10-13 10:03:29 -07:00
zcp_global.h OpenZFS 7431 - ZFS Channel Programs 2018-02-08 15:28:18 -08:00
zcp_iter.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zcp_prop.h OpenZFS 7431 - ZFS Channel Programs 2018-02-08 15:28:18 -08:00
zcp_set.h Support setting user properties in a channel program 2020-02-14 13:41:42 -08:00
zfeature.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_acl.h Linux 6.3 compat: idmapped mount API changes 2023-04-10 14:15:36 -07:00
zfs_bootenv.h zfs label bootenv should store data as nvlist 2020-09-15 15:42:27 -07:00
zfs_chksum.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_context.h Linux: Report reclaimable memory to kernel as such (#16385) 2024-07-30 11:40:47 -07:00
zfs_debug.h zdb/ztest: send dbgmsg output to stderr 2024-05-14 09:49:00 -07:00
zfs_delay.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_file.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_fuid.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_impl.h Add generic implementation handling and SHA2 impl 2023-03-02 13:52:21 -08:00
zfs_ioctl.h Parallel pool import 2024-04-22 09:42:38 -07:00
zfs_ioctl_impl.h Cleanup: 64-bit kernel module parameters should use fixed width types 2022-10-13 10:03:29 -07:00
zfs_onexit.h zfs_onexit_add_cb: make action_handle point to a uintptr_t 2022-11-03 09:52:12 -07:00
zfs_project.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_quota.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_racct.h Adding Direct IO Support 2024-09-04 17:06:56 -06:00
zfs_ratelimit.h Change checksum & IO delay ratelimit values 2018-03-04 17:34:51 -08:00
zfs_refcount.h Switch refcount tracking from lists to AVL-trees. 2023-06-14 08:02:27 -07:00
zfs_rlock.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_sa.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_stat.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_sysfs.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_vfsops.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_vnops.h Adding Direct IO Support 2024-09-04 17:06:56 -06:00
zfs_znode.h Adding Direct IO Support 2024-09-04 17:06:56 -06:00
zil.h zil: add stats for commit failure/fallback (#16315) 2024-07-25 16:53:59 -07:00
zil_impl.h ZIL: Improve next log block size prediction 2023-12-21 10:54:44 -08:00
zio.h Adding Direct IO Support 2024-09-04 17:06:56 -06:00
zio_checksum.h Don't emit cksum_{actual_expected} in ereport.fs.zfs.checksum events 2023-07-21 11:49:26 -07:00
zio_compress.h compress: change zio_compress API to use ABDs 2024-08-22 16:22:24 -07:00
zio_crypt.h Enable -Wwrite-strings 2022-06-29 14:08:54 -07:00
zio_impl.h Adding Direct IO Support 2024-09-04 17:06:56 -06:00
zio_priority.h Add device rebuild feature 2020-07-03 11:05:50 -07:00
zrlock.h Pack zrlock_t by 8 bytes 2023-01-05 09:31:55 -08:00
zthr.h Avoid memory allocations in the ARC eviction thread 2022-01-21 10:28:13 -08:00
zvol.h zvol: fix delayed update to block device ro entry 2023-10-31 09:50:38 -07:00
zvol_impl.h zvol: ensure device minors are properly cleaned up 2024-08-06 12:08:14 -07:00