You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
while copying around 5 TB of data between two zpools by using multiple rsync sessions, I've manually started scrub for one of the zpools. Later on I've noticed that any zfs/zpool commands are stalling and my logs are spammed with:
Dec 1 11:06:50 srv_cs1 kernel: INFO: task txg_sync:17368 blocked for more than 120 seconds.
Dec 1 11:06:50 srv_cs1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 1 11:06:50 srv_cs1 kernel: txg_sync D 0000000000000001 0 17368 2 0x00000080
Dec 1 11:06:50 srv_cs1 kernel: ffff885e43b6bac0 0000000000000046 ffff885e41c03980 ffff885e43b6bfd8
Dec 1 11:06:50 srv_cs1 kernel: ffff885e43b6bfd8 ffff885e43b6bfd8 ffff885e41c03980 ffff897eff034780
Dec 1 11:06:50 srv_cs1 kernel: 0000000000000000 7fffffffffffffff ffff887b35ab07a8 0000000000000001
Dec 1 11:06:50 srv_cs1 kernel: Call Trace:
Dec 1 11:06:50 srv_cs1 kernel: [] schedule+0x29/0x70
Dec 1 11:06:50 srv_cs1 kernel: [] schedule_timeout+0x209/0x2d0
Dec 1 11:06:50 srv_cs1 kernel: [] ? default_wake_function+0x12/0x20
Dec 1 11:06:50 srv_cs1 kernel: [] ? __wake_up_common+0x58/0x90
Dec 1 11:06:50 srv_cs1 kernel: [] ? read_tsc+0x9/0x10
Dec 1 11:06:50 srv_cs1 kernel: [] io_schedule_timeout+0xae/0x130
Dec 1 11:06:50 srv_cs1 kernel: [] ? prepare_to_wait_exclusive+0x56/0x90
Dec 1 11:06:50 srv_cs1 kernel: [] io_schedule+0x18/0x20
Dec 1 11:06:50 srv_cs1 kernel: [] cv_wait_common+0xa7/0x130 [spl]
Dec 1 11:06:50 srv_cs1 kernel: [] ? wake_up_atomic_t+0x30/0x30
Dec 1 11:06:50 srv_cs1 kernel: [] __cv_wait_io+0x18/0x20 [spl]
Dec 1 11:06:50 srv_cs1 kernel: [] zio_wait+0x10b/0x1f0 [zfs]
Dec 1 11:06:50 srv_cs1 kernel: [] dsl_pool_sync+0xbf/0x440 [zfs]
Dec 1 11:06:50 srv_cs1 kernel: [] spa_sync+0x388/0xb50 [zfs]
Dec 1 11:06:50 srv_cs1 kernel: [] ? autoremove_wake_function+0x2b/0x40
Dec 1 11:06:50 srv_cs1 kernel: [] txg_sync_thread+0x3c5/0x620 [zfs]
Dec 1 11:06:50 srv_cs1 kernel: [] ? txg_init+0x280/0x280 [zfs]
Dec 1 11:06:50 srv_cs1 kernel: [] thread_generic_wrapper+0x71/0x80 [spl]
Dec 1 11:06:50 srv_cs1 kernel: [] ? __thread_exit+0x20/0x20 [spl]
Dec 1 11:06:50 srv_cs1 kernel: [] kthread+0xcf/0xe0
Dec 1 11:06:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:06:50 srv_cs1 kernel: [] ret_from_fork+0x58/0x90
Dec 1 11:06:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:10:01 srv_cs1 systemd: Started Session 89 of user root.
Dec 1 11:10:01 srv_cs1 systemd: Starting Session 89 of user root.
Dec 1 11:10:50 srv_cs1 kernel: INFO: task txg_sync:17368 blocked for more than 120 seconds.
Dec 1 11:10:50 srv_cs1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 1 11:10:50 srv_cs1 kernel: txg_sync D 0000000000000001 0 17368 2 0x00000080
Dec 1 11:10:50 srv_cs1 kernel: ffff885e43b6bac0 0000000000000046 ffff885e41c03980 ffff885e43b6bfd8
Dec 1 11:10:50 srv_cs1 kernel: ffff885e43b6bfd8 ffff885e43b6bfd8 ffff885e41c03980 ffff897eff274780
Dec 1 11:10:50 srv_cs1 kernel: 0000000000000000 7fffffffffffffff ffff896381b29058 0000000000000001
Dec 1 11:10:50 srv_cs1 kernel: Call Trace:
Dec 1 11:10:50 srv_cs1 kernel: [] schedule+0x29/0x70
Dec 1 11:10:50 srv_cs1 kernel: [] schedule_timeout+0x209/0x2d0
Dec 1 11:10:50 srv_cs1 kernel: [] ? default_wake_function+0x12/0x20
Dec 1 11:10:50 srv_cs1 kernel: [] ? __wake_up_common+0x58/0x90
Dec 1 11:10:50 srv_cs1 kernel: [] ? read_tsc+0x9/0x10
Dec 1 11:10:50 srv_cs1 kernel: [] io_schedule_timeout+0xae/0x130
Dec 1 11:10:50 srv_cs1 kernel: [] ? prepare_to_wait_exclusive+0x56/0x90
Dec 1 11:10:50 srv_cs1 kernel: [] io_schedule+0x18/0x20
Dec 1 11:10:50 srv_cs1 kernel: [] cv_wait_common+0xa7/0x130 [spl]
Dec 1 11:10:50 srv_cs1 kernel: [] ? wake_up_atomic_t+0x30/0x30
Dec 1 11:10:50 srv_cs1 kernel: [] __cv_wait_io+0x18/0x20 [spl]
Dec 1 11:10:50 srv_cs1 kernel: [] zio_wait+0x10b/0x1f0 [zfs]
Dec 1 11:10:50 srv_cs1 kernel: [] dsl_pool_sync+0xbf/0x440 [zfs]
Dec 1 11:10:50 srv_cs1 kernel: [] spa_sync+0x388/0xb50 [zfs]
Dec 1 11:10:50 srv_cs1 kernel: [] ? autoremove_wake_function+0x2b/0x40
Dec 1 11:10:50 srv_cs1 kernel: [] txg_sync_thread+0x3c5/0x620 [zfs]
Dec 1 11:10:50 srv_cs1 kernel: [] ? txg_init+0x280/0x280 [zfs]
Dec 1 11:10:50 srv_cs1 kernel: [] thread_generic_wrapper+0x71/0x80 [spl]
Dec 1 11:10:50 srv_cs1 kernel: [] ? __thread_exit+0x20/0x20 [spl]
Dec 1 11:10:50 srv_cs1 kernel: [] kthread+0xcf/0xe0
Dec 1 11:10:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:10:50 srv_cs1 kernel: [] ret_from_fork+0x58/0x90
Dec 1 11:10:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:14:50 srv_cs1 kernel: INFO: task txg_sync:17368 blocked for more than 120 seconds.
Dec 1 11:14:50 srv_cs1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 1 11:14:50 srv_cs1 kernel: txg_sync D 0000000000000001 0 17368 2 0x00000080
Dec 1 11:14:50 srv_cs1 kernel: ffff885e43b6bac0 0000000000000046 ffff885e41c03980 ffff885e43b6bfd8
Dec 1 11:14:50 srv_cs1 kernel: ffff885e43b6bfd8 ffff885e43b6bfd8 ffff885e41c03980 ffff88beffa14780
Dec 1 11:14:50 srv_cs1 kernel: 0000000000000000 7fffffffffffffff ffff8979cad96b90 0000000000000001
Dec 1 11:14:50 srv_cs1 kernel: Call Trace:
Dec 1 11:14:50 srv_cs1 kernel: [] schedule+0x29/0x70
Dec 1 11:14:50 srv_cs1 kernel: [] schedule_timeout+0x209/0x2d0
Dec 1 11:14:50 srv_cs1 kernel: [] ? default_wake_function+0x12/0x20
Dec 1 11:14:50 srv_cs1 kernel: [] ? __wake_up_common+0x58/0x90
Dec 1 11:14:50 srv_cs1 kernel: [] ? read_tsc+0x9/0x10
Dec 1 11:14:50 srv_cs1 kernel: [] io_schedule_timeout+0xae/0x130
Dec 1 11:14:50 srv_cs1 kernel: [] ? prepare_to_wait_exclusive+0x56/0x90
Dec 1 11:14:50 srv_cs1 kernel: [] io_schedule+0x18/0x20
Dec 1 11:14:50 srv_cs1 kernel: [] cv_wait_common+0xa7/0x130 [spl]
Dec 1 11:14:50 srv_cs1 kernel: [] ? wake_up_atomic_t+0x30/0x30
Dec 1 11:14:50 srv_cs1 kernel: [] __cv_wait_io+0x18/0x20 [spl]
Dec 1 11:14:50 srv_cs1 kernel: [] zio_wait+0x10b/0x1f0 [zfs]
Dec 1 11:14:50 srv_cs1 kernel: [] dsl_pool_sync+0xbf/0x440 [zfs]
Dec 1 11:14:50 srv_cs1 kernel: [] spa_sync+0x388/0xb50 [zfs]
Dec 1 11:14:50 srv_cs1 kernel: [] ? autoremove_wake_function+0x2b/0x40
Dec 1 11:14:50 srv_cs1 kernel: [] txg_sync_thread+0x3c5/0x620 [zfs]
Dec 1 11:14:50 srv_cs1 kernel: [] ? txg_init+0x280/0x280 [zfs]
Dec 1 11:14:50 srv_cs1 kernel: [] thread_generic_wrapper+0x71/0x80 [spl]
Dec 1 11:14:50 srv_cs1 kernel: [] ? __thread_exit+0x20/0x20 [spl]
Dec 1 11:14:50 srv_cs1 kernel: [] kthread+0xcf/0xe0
Dec 1 11:14:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:14:50 srv_cs1 kernel: [] ret_from_fork+0x58/0x90
Dec 1 11:14:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:18:50 srv_cs1 kernel: INFO: task txg_sync:17368 blocked for more than 120 seconds.
Dec 1 11:18:50 srv_cs1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 1 11:18:50 srv_cs1 kernel: txg_sync D 0000000000000001 0 17368 2 0x00000080
Dec 1 11:18:50 srv_cs1 kernel: ffff885e43b6bac0 0000000000000046 ffff885e41c03980 ffff885e43b6bfd8
Dec 1 11:18:50 srv_cs1 kernel: ffff885e43b6bfd8 ffff885e43b6bfd8 ffff885e41c03980 ffff88beffa54780
Dec 1 11:18:50 srv_cs1 kernel: 0000000000000000 7fffffffffffffff ffff8972a73a8c00 0000000000000001
Dec 1 11:18:50 srv_cs1 kernel: Call Trace:
Dec 1 11:18:50 srv_cs1 kernel: [] schedule+0x29/0x70
Dec 1 11:18:50 srv_cs1 kernel: [] schedule_timeout+0x209/0x2d0
Dec 1 11:18:50 srv_cs1 kernel: [] ? default_wake_function+0x12/0x20
Dec 1 11:18:50 srv_cs1 kernel: [] ? __wake_up_common+0x58/0x90
Dec 1 11:18:50 srv_cs1 kernel: [] ? read_tsc+0x9/0x10
Dec 1 11:18:50 srv_cs1 kernel: [] io_schedule_timeout+0xae/0x130
Dec 1 11:18:50 srv_cs1 kernel: [] ? prepare_to_wait_exclusive+0x56/0x90
Dec 1 11:18:50 srv_cs1 kernel: [] io_schedule+0x18/0x20
Dec 1 11:18:50 srv_cs1 kernel: [] cv_wait_common+0xa7/0x130 [spl]
Dec 1 11:18:50 srv_cs1 kernel: [] ? wake_up_atomic_t+0x30/0x30
Dec 1 11:18:50 srv_cs1 kernel: [] __cv_wait_io+0x18/0x20 [spl]
Dec 1 11:18:50 srv_cs1 kernel: [] zio_wait+0x10b/0x1f0 [zfs]
Dec 1 11:18:50 srv_cs1 kernel: [] dsl_pool_sync+0xbf/0x440 [zfs]
Dec 1 11:18:50 srv_cs1 kernel: [] spa_sync+0x388/0xb50 [zfs]
Dec 1 11:18:50 srv_cs1 kernel: [] ? autoremove_wake_function+0x2b/0x40
Dec 1 11:18:50 srv_cs1 kernel: [] txg_sync_thread+0x3c5/0x620 [zfs]
Dec 1 11:18:50 srv_cs1 kernel: [] ? txg_init+0x280/0x280 [zfs]
Dec 1 11:18:50 srv_cs1 kernel: [] thread_generic_wrapper+0x71/0x80 [spl]
Dec 1 11:18:50 srv_cs1 kernel: [] ? __thread_exit+0x20/0x20 [spl]
Dec 1 11:18:50 srv_cs1 kernel: [] kthread+0xcf/0xe0
Dec 1 11:18:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:18:50 srv_cs1 kernel: [] ret_from_fork+0x58/0x90
Dec 1 11:18:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:20:01 srv_cs1 systemd: Started Session 90 of user root.
Dec 1 11:20:01 srv_cs1 systemd: Starting Session 90 of user root.
Dec 1 11:22:50 srv_cs1 kernel: INFO: task txg_sync:17368 blocked for more than 120 seconds.
Dec 1 11:22:50 srv_cs1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 1 11:22:50 srv_cs1 kernel: txg_sync D 0000000000000001 0 17368 2 0x00000080
Dec 1 11:22:50 srv_cs1 kernel: ffff885e43b6bac0 0000000000000046 ffff885e41c03980 ffff885e43b6bfd8
Dec 1 11:22:50 srv_cs1 kernel: ffff885e43b6bfd8 ffff885e43b6bfd8 ffff885e41c03980 ffff88beff894780
Dec 1 11:22:50 srv_cs1 kernel: 0000000000000000 7fffffffffffffff ffff88c886ca55d8 0000000000000001
Dec 1 11:22:50 srv_cs1 kernel: Call Trace:
Dec 1 11:22:50 srv_cs1 kernel: [] schedule+0x29/0x70
Dec 1 11:22:50 srv_cs1 kernel: [] schedule_timeout+0x209/0x2d0
Dec 1 11:22:50 srv_cs1 kernel: [] ? default_wake_function+0x12/0x20
Dec 1 11:22:50 srv_cs1 kernel: [] ? __wake_up_common+0x58/0x90
Dec 1 11:22:50 srv_cs1 kernel: [] ? read_tsc+0x9/0x10
Dec 1 11:22:50 srv_cs1 kernel: [] io_schedule_timeout+0xae/0x130
Dec 1 11:22:50 srv_cs1 kernel: [] ? prepare_to_wait_exclusive+0x56/0x90
Dec 1 11:22:50 srv_cs1 kernel: [] io_schedule+0x18/0x20
Dec 1 11:22:50 srv_cs1 kernel: [] cv_wait_common+0xa7/0x130 [spl]
Dec 1 11:22:50 srv_cs1 kernel: [] ? wake_up_atomic_t+0x30/0x30
Dec 1 11:22:50 srv_cs1 kernel: [] __cv_wait_io+0x18/0x20 [spl]
Dec 1 11:22:50 srv_cs1 kernel: [] zio_wait+0x10b/0x1f0 [zfs]
Dec 1 11:22:50 srv_cs1 kernel: [] dsl_pool_sync+0xbf/0x440 [zfs]
Dec 1 11:22:50 srv_cs1 kernel: [] spa_sync+0x388/0xb50 [zfs]
Dec 1 11:22:50 srv_cs1 kernel: [] ? autoremove_wake_function+0x2b/0x40
Dec 1 11:22:50 srv_cs1 kernel: [] txg_sync_thread+0x3c5/0x620 [zfs]
Dec 1 11:22:50 srv_cs1 kernel: [] ? txg_init+0x280/0x280 [zfs]
Dec 1 11:22:50 srv_cs1 kernel: [] thread_generic_wrapper+0x71/0x80 [spl]
Dec 1 11:22:50 srv_cs1 kernel: [] ? __thread_exit+0x20/0x20 [spl]
Dec 1 11:22:50 srv_cs1 kernel: [] kthread+0xcf/0xe0
Dec 1 11:22:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:22:50 srv_cs1 kernel: [] ret_from_fork+0x58/0x90
Dec 1 11:22:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:26:50 srv_cs1 kernel: INFO: task txg_sync:17368 blocked for more than 120 seconds.
Dec 1 11:26:50 srv_cs1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 1 11:26:50 srv_cs1 kernel: txg_sync D 0000000000000001 0 17368 2 0x00000080
Dec 1 11:26:50 srv_cs1 kernel: ffff885e43b6bac0 0000000000000046 ffff885e41c03980 ffff885e43b6bfd8
Dec 1 11:26:50 srv_cs1 kernel: ffff885e43b6bfd8 ffff885e43b6bfd8 ffff885e41c03980 ffff88beffab4780
Dec 1 11:26:50 srv_cs1 kernel: 0000000000000000 7fffffffffffffff ffff8889aba5c478 0000000000000001
Dec 1 11:26:50 srv_cs1 kernel: Call Trace:
Dec 1 11:26:50 srv_cs1 kernel: [] schedule+0x29/0x70
Dec 1 11:26:50 srv_cs1 kernel: [] schedule_timeout+0x209/0x2d0
Dec 1 11:26:50 srv_cs1 kernel: [] ? default_wake_function+0x12/0x20
Dec 1 11:26:50 srv_cs1 kernel: [] ? __wake_up_common+0x58/0x90
Dec 1 11:26:50 srv_cs1 kernel: [] ? read_tsc+0x9/0x10
Dec 1 11:26:50 srv_cs1 kernel: [] io_schedule_timeout+0xae/0x130
Dec 1 11:26:50 srv_cs1 kernel: [] ? prepare_to_wait_exclusive+0x56/0x90
Dec 1 11:26:50 srv_cs1 kernel: [] io_schedule+0x18/0x20
Dec 1 11:26:50 srv_cs1 kernel: [] cv_wait_common+0xa7/0x130 [spl]
Dec 1 11:26:50 srv_cs1 kernel: [] ? wake_up_atomic_t+0x30/0x30
Dec 1 11:26:50 srv_cs1 kernel: [] __cv_wait_io+0x18/0x20 [spl]
Dec 1 11:26:50 srv_cs1 kernel: [] zio_wait+0x10b/0x1f0 [zfs]
Dec 1 11:26:50 srv_cs1 kernel: [] dsl_pool_sync+0xbf/0x440 [zfs]
Dec 1 11:26:50 srv_cs1 kernel: [] spa_sync+0x388/0xb50 [zfs]
Dec 1 11:26:50 srv_cs1 kernel: [] ? autoremove_wake_function+0x2b/0x40
Dec 1 11:26:50 srv_cs1 kernel: [] txg_sync_thread+0x3c5/0x620 [zfs]
Dec 1 11:26:50 srv_cs1 kernel: [] ? txg_init+0x280/0x280 [zfs]
Dec 1 11:26:50 srv_cs1 kernel: [] thread_generic_wrapper+0x71/0x80 [spl]
Dec 1 11:26:50 srv_cs1 kernel: [] ? __thread_exit+0x20/0x20 [spl]
Dec 1 11:26:50 srv_cs1 kernel: [] kthread+0xcf/0xe0
Dec 1 11:26:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:26:50 srv_cs1 kernel: [] ret_from_fork+0x58/0x90
Dec 1 11:26:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:30:50 srv_cs1 kernel: INFO: task txg_sync:17368 blocked for more than 120 seconds.
Dec 1 11:30:50 srv_cs1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 1 11:30:50 srv_cs1 kernel: txg_sync D 0000000000000001 0 17368 2 0x00000080
Dec 1 11:30:50 srv_cs1 kernel: ffff885e43b6bac0 0000000000000046 ffff885e41c03980 ffff885e43b6bfd8
Dec 1 11:30:50 srv_cs1 kernel: ffff885e43b6bfd8 ffff885e43b6bfd8 ffff885e41c03980 ffff88beff9f4780
Dec 1 11:30:50 srv_cs1 kernel: 0000000000000000 7fffffffffffffff ffff88c8fde6c8d0 0000000000000001
Dec 1 11:30:50 srv_cs1 kernel: Call Trace:
Dec 1 11:30:50 srv_cs1 kernel: [] schedule+0x29/0x70
Dec 1 11:30:50 srv_cs1 kernel: [] schedule_timeout+0x209/0x2d0
Dec 1 11:30:50 srv_cs1 kernel: [] ? default_wake_function+0x12/0x20
Dec 1 11:30:50 srv_cs1 kernel: [] ? __wake_up_common+0x58/0x90
Dec 1 11:30:50 srv_cs1 kernel: [] ? read_tsc+0x9/0x10
Dec 1 11:30:50 srv_cs1 kernel: [] io_schedule_timeout+0xae/0x130
Dec 1 11:30:50 srv_cs1 kernel: [] ? prepare_to_wait_exclusive+0x56/0x90
Dec 1 11:30:50 srv_cs1 kernel: [] io_schedule+0x18/0x20
Dec 1 11:30:50 srv_cs1 kernel: [] cv_wait_common+0xa7/0x130 [spl]
Dec 1 11:30:50 srv_cs1 kernel: [] ? wake_up_atomic_t+0x30/0x30
Dec 1 11:30:50 srv_cs1 kernel: [] __cv_wait_io+0x18/0x20 [spl]
Dec 1 11:30:50 srv_cs1 kernel: [] zio_wait+0x10b/0x1f0 [zfs]
Dec 1 11:30:50 srv_cs1 kernel: [] dsl_pool_sync+0xbf/0x440 [zfs]
Dec 1 11:30:50 srv_cs1 kernel: [] spa_sync+0x388/0xb50 [zfs]
Dec 1 11:30:50 srv_cs1 kernel: [] ? autoremove_wake_function+0x2b/0x40
Dec 1 11:30:50 srv_cs1 kernel: [] txg_sync_thread+0x3c5/0x620 [zfs]
Dec 1 11:30:50 srv_cs1 kernel: [] ? txg_init+0x280/0x280 [zfs]
Dec 1 11:30:50 srv_cs1 kernel: [] thread_generic_wrapper+0x71/0x80 [spl]
Dec 1 11:30:50 srv_cs1 kernel: [] ? __thread_exit+0x20/0x20 [spl]
Dec 1 11:30:50 srv_cs1 kernel: [] kthread+0xcf/0xe0
Dec 1 11:30:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:30:50 srv_cs1 kernel: [] ret_from_fork+0x58/0x90
Dec 1 11:30:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:34:50 srv_cs1 kernel: INFO: task txg_sync:17368 blocked for more than 120 seconds.
Dec 1 11:34:50 srv_cs1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 1 11:34:50 srv_cs1 kernel: txg_sync D 0000000000000001 0 17368 2 0x00000080
Dec 1 11:34:50 srv_cs1 kernel: ffff885e43b6bac0 0000000000000046 ffff885e41c03980 ffff885e43b6bfd8
Dec 1 11:34:50 srv_cs1 kernel: ffff885e43b6bfd8 ffff885e43b6bfd8 ffff885e41c03980 ffff885eff954780
Dec 1 11:34:50 srv_cs1 kernel: 0000000000000000 7fffffffffffffff ffff88b322117440 0000000000000001
Dec 1 11:34:50 srv_cs1 kernel: Call Trace:
Dec 1 11:34:50 srv_cs1 kernel: [] schedule+0x29/0x70
Dec 1 11:34:50 srv_cs1 kernel: [] schedule_timeout+0x209/0x2d0
Dec 1 11:34:50 srv_cs1 kernel: [] ? default_wake_function+0x12/0x20
Dec 1 11:34:50 srv_cs1 kernel: [] ? __wake_up_common+0x58/0x90
Dec 1 11:34:50 srv_cs1 kernel: [] ? read_tsc+0x9/0x10
Dec 1 11:34:50 srv_cs1 kernel: [] io_schedule_timeout+0xae/0x130
Dec 1 11:34:50 srv_cs1 kernel: [] ? prepare_to_wait_exclusive+0x56/0x90
Dec 1 11:34:50 srv_cs1 kernel: [] io_schedule+0x18/0x20
Dec 1 11:34:50 srv_cs1 kernel: [] cv_wait_common+0xa7/0x130 [spl]
Dec 1 11:34:50 srv_cs1 kernel: [] ? wake_up_atomic_t+0x30/0x30
Dec 1 11:34:50 srv_cs1 kernel: [] __cv_wait_io+0x18/0x20 [spl]
Dec 1 11:34:50 srv_cs1 kernel: [] zio_wait+0x10b/0x1f0 [zfs]
Dec 1 11:34:50 srv_cs1 kernel: [] dsl_pool_sync+0xbf/0x440 [zfs]
Dec 1 11:34:50 srv_cs1 kernel: [] spa_sync+0x388/0xb50 [zfs]
Dec 1 11:34:50 srv_cs1 kernel: [] txg_sync_thread+0x3c5/0x620 [zfs]
Dec 1 11:34:50 srv_cs1 kernel: [] ? txg_init+0x280/0x280 [zfs]
Dec 1 11:34:50 srv_cs1 kernel: [] thread_generic_wrapper+0x71/0x80 [spl]
Dec 1 11:34:50 srv_cs1 kernel: [] ? __thread_exit+0x20/0x20 [spl]
Dec 1 11:34:50 srv_cs1 kernel: [] kthread+0xcf/0xe0
Dec 1 11:34:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:34:50 srv_cs1 kernel: [] ret_from_fork+0x58/0x90
Dec 1 11:34:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
while copying around 5 TB of data between two zpools by using multiple rsync sessions, I've manually started scrub for one of the zpools. Later on I've noticed that any zfs/zpool commands are stalling and my logs are spammed with:
Dec 1 11:06:50 srv_cs1 kernel: INFO: task txg_sync:17368 blocked for more than 120 seconds.
Dec 1 11:06:50 srv_cs1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 1 11:06:50 srv_cs1 kernel: txg_sync D 0000000000000001 0 17368 2 0x00000080
Dec 1 11:06:50 srv_cs1 kernel: ffff885e43b6bac0 0000000000000046 ffff885e41c03980 ffff885e43b6bfd8
Dec 1 11:06:50 srv_cs1 kernel: ffff885e43b6bfd8 ffff885e43b6bfd8 ffff885e41c03980 ffff897eff034780
Dec 1 11:06:50 srv_cs1 kernel: 0000000000000000 7fffffffffffffff ffff887b35ab07a8 0000000000000001
Dec 1 11:06:50 srv_cs1 kernel: Call Trace:
Dec 1 11:06:50 srv_cs1 kernel: [] schedule+0x29/0x70
Dec 1 11:06:50 srv_cs1 kernel: [] schedule_timeout+0x209/0x2d0
Dec 1 11:06:50 srv_cs1 kernel: [] ? default_wake_function+0x12/0x20
Dec 1 11:06:50 srv_cs1 kernel: [] ? __wake_up_common+0x58/0x90
Dec 1 11:06:50 srv_cs1 kernel: [] ? read_tsc+0x9/0x10
Dec 1 11:06:50 srv_cs1 kernel: [] io_schedule_timeout+0xae/0x130
Dec 1 11:06:50 srv_cs1 kernel: [] ? prepare_to_wait_exclusive+0x56/0x90
Dec 1 11:06:50 srv_cs1 kernel: [] io_schedule+0x18/0x20
Dec 1 11:06:50 srv_cs1 kernel: [] cv_wait_common+0xa7/0x130 [spl]
Dec 1 11:06:50 srv_cs1 kernel: [] ? wake_up_atomic_t+0x30/0x30
Dec 1 11:06:50 srv_cs1 kernel: [] __cv_wait_io+0x18/0x20 [spl]
Dec 1 11:06:50 srv_cs1 kernel: [] zio_wait+0x10b/0x1f0 [zfs]
Dec 1 11:06:50 srv_cs1 kernel: [] dsl_pool_sync+0xbf/0x440 [zfs]
Dec 1 11:06:50 srv_cs1 kernel: [] spa_sync+0x388/0xb50 [zfs]
Dec 1 11:06:50 srv_cs1 kernel: [] ? autoremove_wake_function+0x2b/0x40
Dec 1 11:06:50 srv_cs1 kernel: [] txg_sync_thread+0x3c5/0x620 [zfs]
Dec 1 11:06:50 srv_cs1 kernel: [] ? txg_init+0x280/0x280 [zfs]
Dec 1 11:06:50 srv_cs1 kernel: [] thread_generic_wrapper+0x71/0x80 [spl]
Dec 1 11:06:50 srv_cs1 kernel: [] ? __thread_exit+0x20/0x20 [spl]
Dec 1 11:06:50 srv_cs1 kernel: [] kthread+0xcf/0xe0
Dec 1 11:06:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:06:50 srv_cs1 kernel: [] ret_from_fork+0x58/0x90
Dec 1 11:06:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:10:01 srv_cs1 systemd: Started Session 89 of user root.
Dec 1 11:10:01 srv_cs1 systemd: Starting Session 89 of user root.
Dec 1 11:10:50 srv_cs1 kernel: INFO: task txg_sync:17368 blocked for more than 120 seconds.
Dec 1 11:10:50 srv_cs1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 1 11:10:50 srv_cs1 kernel: txg_sync D 0000000000000001 0 17368 2 0x00000080
Dec 1 11:10:50 srv_cs1 kernel: ffff885e43b6bac0 0000000000000046 ffff885e41c03980 ffff885e43b6bfd8
Dec 1 11:10:50 srv_cs1 kernel: ffff885e43b6bfd8 ffff885e43b6bfd8 ffff885e41c03980 ffff897eff274780
Dec 1 11:10:50 srv_cs1 kernel: 0000000000000000 7fffffffffffffff ffff896381b29058 0000000000000001
Dec 1 11:10:50 srv_cs1 kernel: Call Trace:
Dec 1 11:10:50 srv_cs1 kernel: [] schedule+0x29/0x70
Dec 1 11:10:50 srv_cs1 kernel: [] schedule_timeout+0x209/0x2d0
Dec 1 11:10:50 srv_cs1 kernel: [] ? default_wake_function+0x12/0x20
Dec 1 11:10:50 srv_cs1 kernel: [] ? __wake_up_common+0x58/0x90
Dec 1 11:10:50 srv_cs1 kernel: [] ? read_tsc+0x9/0x10
Dec 1 11:10:50 srv_cs1 kernel: [] io_schedule_timeout+0xae/0x130
Dec 1 11:10:50 srv_cs1 kernel: [] ? prepare_to_wait_exclusive+0x56/0x90
Dec 1 11:10:50 srv_cs1 kernel: [] io_schedule+0x18/0x20
Dec 1 11:10:50 srv_cs1 kernel: [] cv_wait_common+0xa7/0x130 [spl]
Dec 1 11:10:50 srv_cs1 kernel: [] ? wake_up_atomic_t+0x30/0x30
Dec 1 11:10:50 srv_cs1 kernel: [] __cv_wait_io+0x18/0x20 [spl]
Dec 1 11:10:50 srv_cs1 kernel: [] zio_wait+0x10b/0x1f0 [zfs]
Dec 1 11:10:50 srv_cs1 kernel: [] dsl_pool_sync+0xbf/0x440 [zfs]
Dec 1 11:10:50 srv_cs1 kernel: [] spa_sync+0x388/0xb50 [zfs]
Dec 1 11:10:50 srv_cs1 kernel: [] ? autoremove_wake_function+0x2b/0x40
Dec 1 11:10:50 srv_cs1 kernel: [] txg_sync_thread+0x3c5/0x620 [zfs]
Dec 1 11:10:50 srv_cs1 kernel: [] ? txg_init+0x280/0x280 [zfs]
Dec 1 11:10:50 srv_cs1 kernel: [] thread_generic_wrapper+0x71/0x80 [spl]
Dec 1 11:10:50 srv_cs1 kernel: [] ? __thread_exit+0x20/0x20 [spl]
Dec 1 11:10:50 srv_cs1 kernel: [] kthread+0xcf/0xe0
Dec 1 11:10:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:10:50 srv_cs1 kernel: [] ret_from_fork+0x58/0x90
Dec 1 11:10:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:14:50 srv_cs1 kernel: INFO: task txg_sync:17368 blocked for more than 120 seconds.
Dec 1 11:14:50 srv_cs1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 1 11:14:50 srv_cs1 kernel: txg_sync D 0000000000000001 0 17368 2 0x00000080
Dec 1 11:14:50 srv_cs1 kernel: ffff885e43b6bac0 0000000000000046 ffff885e41c03980 ffff885e43b6bfd8
Dec 1 11:14:50 srv_cs1 kernel: ffff885e43b6bfd8 ffff885e43b6bfd8 ffff885e41c03980 ffff88beffa14780
Dec 1 11:14:50 srv_cs1 kernel: 0000000000000000 7fffffffffffffff ffff8979cad96b90 0000000000000001
Dec 1 11:14:50 srv_cs1 kernel: Call Trace:
Dec 1 11:14:50 srv_cs1 kernel: [] schedule+0x29/0x70
Dec 1 11:14:50 srv_cs1 kernel: [] schedule_timeout+0x209/0x2d0
Dec 1 11:14:50 srv_cs1 kernel: [] ? default_wake_function+0x12/0x20
Dec 1 11:14:50 srv_cs1 kernel: [] ? __wake_up_common+0x58/0x90
Dec 1 11:14:50 srv_cs1 kernel: [] ? read_tsc+0x9/0x10
Dec 1 11:14:50 srv_cs1 kernel: [] io_schedule_timeout+0xae/0x130
Dec 1 11:14:50 srv_cs1 kernel: [] ? prepare_to_wait_exclusive+0x56/0x90
Dec 1 11:14:50 srv_cs1 kernel: [] io_schedule+0x18/0x20
Dec 1 11:14:50 srv_cs1 kernel: [] cv_wait_common+0xa7/0x130 [spl]
Dec 1 11:14:50 srv_cs1 kernel: [] ? wake_up_atomic_t+0x30/0x30
Dec 1 11:14:50 srv_cs1 kernel: [] __cv_wait_io+0x18/0x20 [spl]
Dec 1 11:14:50 srv_cs1 kernel: [] zio_wait+0x10b/0x1f0 [zfs]
Dec 1 11:14:50 srv_cs1 kernel: [] dsl_pool_sync+0xbf/0x440 [zfs]
Dec 1 11:14:50 srv_cs1 kernel: [] spa_sync+0x388/0xb50 [zfs]
Dec 1 11:14:50 srv_cs1 kernel: [] ? autoremove_wake_function+0x2b/0x40
Dec 1 11:14:50 srv_cs1 kernel: [] txg_sync_thread+0x3c5/0x620 [zfs]
Dec 1 11:14:50 srv_cs1 kernel: [] ? txg_init+0x280/0x280 [zfs]
Dec 1 11:14:50 srv_cs1 kernel: [] thread_generic_wrapper+0x71/0x80 [spl]
Dec 1 11:14:50 srv_cs1 kernel: [] ? __thread_exit+0x20/0x20 [spl]
Dec 1 11:14:50 srv_cs1 kernel: [] kthread+0xcf/0xe0
Dec 1 11:14:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:14:50 srv_cs1 kernel: [] ret_from_fork+0x58/0x90
Dec 1 11:14:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:18:50 srv_cs1 kernel: INFO: task txg_sync:17368 blocked for more than 120 seconds.
Dec 1 11:18:50 srv_cs1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 1 11:18:50 srv_cs1 kernel: txg_sync D 0000000000000001 0 17368 2 0x00000080
Dec 1 11:18:50 srv_cs1 kernel: ffff885e43b6bac0 0000000000000046 ffff885e41c03980 ffff885e43b6bfd8
Dec 1 11:18:50 srv_cs1 kernel: ffff885e43b6bfd8 ffff885e43b6bfd8 ffff885e41c03980 ffff88beffa54780
Dec 1 11:18:50 srv_cs1 kernel: 0000000000000000 7fffffffffffffff ffff8972a73a8c00 0000000000000001
Dec 1 11:18:50 srv_cs1 kernel: Call Trace:
Dec 1 11:18:50 srv_cs1 kernel: [] schedule+0x29/0x70
Dec 1 11:18:50 srv_cs1 kernel: [] schedule_timeout+0x209/0x2d0
Dec 1 11:18:50 srv_cs1 kernel: [] ? default_wake_function+0x12/0x20
Dec 1 11:18:50 srv_cs1 kernel: [] ? __wake_up_common+0x58/0x90
Dec 1 11:18:50 srv_cs1 kernel: [] ? read_tsc+0x9/0x10
Dec 1 11:18:50 srv_cs1 kernel: [] io_schedule_timeout+0xae/0x130
Dec 1 11:18:50 srv_cs1 kernel: [] ? prepare_to_wait_exclusive+0x56/0x90
Dec 1 11:18:50 srv_cs1 kernel: [] io_schedule+0x18/0x20
Dec 1 11:18:50 srv_cs1 kernel: [] cv_wait_common+0xa7/0x130 [spl]
Dec 1 11:18:50 srv_cs1 kernel: [] ? wake_up_atomic_t+0x30/0x30
Dec 1 11:18:50 srv_cs1 kernel: [] __cv_wait_io+0x18/0x20 [spl]
Dec 1 11:18:50 srv_cs1 kernel: [] zio_wait+0x10b/0x1f0 [zfs]
Dec 1 11:18:50 srv_cs1 kernel: [] dsl_pool_sync+0xbf/0x440 [zfs]
Dec 1 11:18:50 srv_cs1 kernel: [] spa_sync+0x388/0xb50 [zfs]
Dec 1 11:18:50 srv_cs1 kernel: [] ? autoremove_wake_function+0x2b/0x40
Dec 1 11:18:50 srv_cs1 kernel: [] txg_sync_thread+0x3c5/0x620 [zfs]
Dec 1 11:18:50 srv_cs1 kernel: [] ? txg_init+0x280/0x280 [zfs]
Dec 1 11:18:50 srv_cs1 kernel: [] thread_generic_wrapper+0x71/0x80 [spl]
Dec 1 11:18:50 srv_cs1 kernel: [] ? __thread_exit+0x20/0x20 [spl]
Dec 1 11:18:50 srv_cs1 kernel: [] kthread+0xcf/0xe0
Dec 1 11:18:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:18:50 srv_cs1 kernel: [] ret_from_fork+0x58/0x90
Dec 1 11:18:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:20:01 srv_cs1 systemd: Started Session 90 of user root.
Dec 1 11:20:01 srv_cs1 systemd: Starting Session 90 of user root.
Dec 1 11:22:50 srv_cs1 kernel: INFO: task txg_sync:17368 blocked for more than 120 seconds.
Dec 1 11:22:50 srv_cs1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 1 11:22:50 srv_cs1 kernel: txg_sync D 0000000000000001 0 17368 2 0x00000080
Dec 1 11:22:50 srv_cs1 kernel: ffff885e43b6bac0 0000000000000046 ffff885e41c03980 ffff885e43b6bfd8
Dec 1 11:22:50 srv_cs1 kernel: ffff885e43b6bfd8 ffff885e43b6bfd8 ffff885e41c03980 ffff88beff894780
Dec 1 11:22:50 srv_cs1 kernel: 0000000000000000 7fffffffffffffff ffff88c886ca55d8 0000000000000001
Dec 1 11:22:50 srv_cs1 kernel: Call Trace:
Dec 1 11:22:50 srv_cs1 kernel: [] schedule+0x29/0x70
Dec 1 11:22:50 srv_cs1 kernel: [] schedule_timeout+0x209/0x2d0
Dec 1 11:22:50 srv_cs1 kernel: [] ? default_wake_function+0x12/0x20
Dec 1 11:22:50 srv_cs1 kernel: [] ? __wake_up_common+0x58/0x90
Dec 1 11:22:50 srv_cs1 kernel: [] ? read_tsc+0x9/0x10
Dec 1 11:22:50 srv_cs1 kernel: [] io_schedule_timeout+0xae/0x130
Dec 1 11:22:50 srv_cs1 kernel: [] ? prepare_to_wait_exclusive+0x56/0x90
Dec 1 11:22:50 srv_cs1 kernel: [] io_schedule+0x18/0x20
Dec 1 11:22:50 srv_cs1 kernel: [] cv_wait_common+0xa7/0x130 [spl]
Dec 1 11:22:50 srv_cs1 kernel: [] ? wake_up_atomic_t+0x30/0x30
Dec 1 11:22:50 srv_cs1 kernel: [] __cv_wait_io+0x18/0x20 [spl]
Dec 1 11:22:50 srv_cs1 kernel: [] zio_wait+0x10b/0x1f0 [zfs]
Dec 1 11:22:50 srv_cs1 kernel: [] dsl_pool_sync+0xbf/0x440 [zfs]
Dec 1 11:22:50 srv_cs1 kernel: [] spa_sync+0x388/0xb50 [zfs]
Dec 1 11:22:50 srv_cs1 kernel: [] ? autoremove_wake_function+0x2b/0x40
Dec 1 11:22:50 srv_cs1 kernel: [] txg_sync_thread+0x3c5/0x620 [zfs]
Dec 1 11:22:50 srv_cs1 kernel: [] ? txg_init+0x280/0x280 [zfs]
Dec 1 11:22:50 srv_cs1 kernel: [] thread_generic_wrapper+0x71/0x80 [spl]
Dec 1 11:22:50 srv_cs1 kernel: [] ? __thread_exit+0x20/0x20 [spl]
Dec 1 11:22:50 srv_cs1 kernel: [] kthread+0xcf/0xe0
Dec 1 11:22:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:22:50 srv_cs1 kernel: [] ret_from_fork+0x58/0x90
Dec 1 11:22:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:26:50 srv_cs1 kernel: INFO: task txg_sync:17368 blocked for more than 120 seconds.
Dec 1 11:26:50 srv_cs1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 1 11:26:50 srv_cs1 kernel: txg_sync D 0000000000000001 0 17368 2 0x00000080
Dec 1 11:26:50 srv_cs1 kernel: ffff885e43b6bac0 0000000000000046 ffff885e41c03980 ffff885e43b6bfd8
Dec 1 11:26:50 srv_cs1 kernel: ffff885e43b6bfd8 ffff885e43b6bfd8 ffff885e41c03980 ffff88beffab4780
Dec 1 11:26:50 srv_cs1 kernel: 0000000000000000 7fffffffffffffff ffff8889aba5c478 0000000000000001
Dec 1 11:26:50 srv_cs1 kernel: Call Trace:
Dec 1 11:26:50 srv_cs1 kernel: [] schedule+0x29/0x70
Dec 1 11:26:50 srv_cs1 kernel: [] schedule_timeout+0x209/0x2d0
Dec 1 11:26:50 srv_cs1 kernel: [] ? default_wake_function+0x12/0x20
Dec 1 11:26:50 srv_cs1 kernel: [] ? __wake_up_common+0x58/0x90
Dec 1 11:26:50 srv_cs1 kernel: [] ? read_tsc+0x9/0x10
Dec 1 11:26:50 srv_cs1 kernel: [] io_schedule_timeout+0xae/0x130
Dec 1 11:26:50 srv_cs1 kernel: [] ? prepare_to_wait_exclusive+0x56/0x90
Dec 1 11:26:50 srv_cs1 kernel: [] io_schedule+0x18/0x20
Dec 1 11:26:50 srv_cs1 kernel: [] cv_wait_common+0xa7/0x130 [spl]
Dec 1 11:26:50 srv_cs1 kernel: [] ? wake_up_atomic_t+0x30/0x30
Dec 1 11:26:50 srv_cs1 kernel: [] __cv_wait_io+0x18/0x20 [spl]
Dec 1 11:26:50 srv_cs1 kernel: [] zio_wait+0x10b/0x1f0 [zfs]
Dec 1 11:26:50 srv_cs1 kernel: [] dsl_pool_sync+0xbf/0x440 [zfs]
Dec 1 11:26:50 srv_cs1 kernel: [] spa_sync+0x388/0xb50 [zfs]
Dec 1 11:26:50 srv_cs1 kernel: [] ? autoremove_wake_function+0x2b/0x40
Dec 1 11:26:50 srv_cs1 kernel: [] txg_sync_thread+0x3c5/0x620 [zfs]
Dec 1 11:26:50 srv_cs1 kernel: [] ? txg_init+0x280/0x280 [zfs]
Dec 1 11:26:50 srv_cs1 kernel: [] thread_generic_wrapper+0x71/0x80 [spl]
Dec 1 11:26:50 srv_cs1 kernel: [] ? __thread_exit+0x20/0x20 [spl]
Dec 1 11:26:50 srv_cs1 kernel: [] kthread+0xcf/0xe0
Dec 1 11:26:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:26:50 srv_cs1 kernel: [] ret_from_fork+0x58/0x90
Dec 1 11:26:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:30:50 srv_cs1 kernel: INFO: task txg_sync:17368 blocked for more than 120 seconds.
Dec 1 11:30:50 srv_cs1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 1 11:30:50 srv_cs1 kernel: txg_sync D 0000000000000001 0 17368 2 0x00000080
Dec 1 11:30:50 srv_cs1 kernel: ffff885e43b6bac0 0000000000000046 ffff885e41c03980 ffff885e43b6bfd8
Dec 1 11:30:50 srv_cs1 kernel: ffff885e43b6bfd8 ffff885e43b6bfd8 ffff885e41c03980 ffff88beff9f4780
Dec 1 11:30:50 srv_cs1 kernel: 0000000000000000 7fffffffffffffff ffff88c8fde6c8d0 0000000000000001
Dec 1 11:30:50 srv_cs1 kernel: Call Trace:
Dec 1 11:30:50 srv_cs1 kernel: [] schedule+0x29/0x70
Dec 1 11:30:50 srv_cs1 kernel: [] schedule_timeout+0x209/0x2d0
Dec 1 11:30:50 srv_cs1 kernel: [] ? default_wake_function+0x12/0x20
Dec 1 11:30:50 srv_cs1 kernel: [] ? __wake_up_common+0x58/0x90
Dec 1 11:30:50 srv_cs1 kernel: [] ? read_tsc+0x9/0x10
Dec 1 11:30:50 srv_cs1 kernel: [] io_schedule_timeout+0xae/0x130
Dec 1 11:30:50 srv_cs1 kernel: [] ? prepare_to_wait_exclusive+0x56/0x90
Dec 1 11:30:50 srv_cs1 kernel: [] io_schedule+0x18/0x20
Dec 1 11:30:50 srv_cs1 kernel: [] cv_wait_common+0xa7/0x130 [spl]
Dec 1 11:30:50 srv_cs1 kernel: [] ? wake_up_atomic_t+0x30/0x30
Dec 1 11:30:50 srv_cs1 kernel: [] __cv_wait_io+0x18/0x20 [spl]
Dec 1 11:30:50 srv_cs1 kernel: [] zio_wait+0x10b/0x1f0 [zfs]
Dec 1 11:30:50 srv_cs1 kernel: [] dsl_pool_sync+0xbf/0x440 [zfs]
Dec 1 11:30:50 srv_cs1 kernel: [] spa_sync+0x388/0xb50 [zfs]
Dec 1 11:30:50 srv_cs1 kernel: [] ? autoremove_wake_function+0x2b/0x40
Dec 1 11:30:50 srv_cs1 kernel: [] txg_sync_thread+0x3c5/0x620 [zfs]
Dec 1 11:30:50 srv_cs1 kernel: [] ? txg_init+0x280/0x280 [zfs]
Dec 1 11:30:50 srv_cs1 kernel: [] thread_generic_wrapper+0x71/0x80 [spl]
Dec 1 11:30:50 srv_cs1 kernel: [] ? __thread_exit+0x20/0x20 [spl]
Dec 1 11:30:50 srv_cs1 kernel: [] kthread+0xcf/0xe0
Dec 1 11:30:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:30:50 srv_cs1 kernel: [] ret_from_fork+0x58/0x90
Dec 1 11:30:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:34:50 srv_cs1 kernel: INFO: task txg_sync:17368 blocked for more than 120 seconds.
Dec 1 11:34:50 srv_cs1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 1 11:34:50 srv_cs1 kernel: txg_sync D 0000000000000001 0 17368 2 0x00000080
Dec 1 11:34:50 srv_cs1 kernel: ffff885e43b6bac0 0000000000000046 ffff885e41c03980 ffff885e43b6bfd8
Dec 1 11:34:50 srv_cs1 kernel: ffff885e43b6bfd8 ffff885e43b6bfd8 ffff885e41c03980 ffff885eff954780
Dec 1 11:34:50 srv_cs1 kernel: 0000000000000000 7fffffffffffffff ffff88b322117440 0000000000000001
Dec 1 11:34:50 srv_cs1 kernel: Call Trace:
Dec 1 11:34:50 srv_cs1 kernel: [] schedule+0x29/0x70
Dec 1 11:34:50 srv_cs1 kernel: [] schedule_timeout+0x209/0x2d0
Dec 1 11:34:50 srv_cs1 kernel: [] ? default_wake_function+0x12/0x20
Dec 1 11:34:50 srv_cs1 kernel: [] ? __wake_up_common+0x58/0x90
Dec 1 11:34:50 srv_cs1 kernel: [] ? read_tsc+0x9/0x10
Dec 1 11:34:50 srv_cs1 kernel: [] io_schedule_timeout+0xae/0x130
Dec 1 11:34:50 srv_cs1 kernel: [] ? prepare_to_wait_exclusive+0x56/0x90
Dec 1 11:34:50 srv_cs1 kernel: [] io_schedule+0x18/0x20
Dec 1 11:34:50 srv_cs1 kernel: [] cv_wait_common+0xa7/0x130 [spl]
Dec 1 11:34:50 srv_cs1 kernel: [] ? wake_up_atomic_t+0x30/0x30
Dec 1 11:34:50 srv_cs1 kernel: [] __cv_wait_io+0x18/0x20 [spl]
Dec 1 11:34:50 srv_cs1 kernel: [] zio_wait+0x10b/0x1f0 [zfs]
Dec 1 11:34:50 srv_cs1 kernel: [] dsl_pool_sync+0xbf/0x440 [zfs]
Dec 1 11:34:50 srv_cs1 kernel: [] spa_sync+0x388/0xb50 [zfs]
Dec 1 11:34:50 srv_cs1 kernel: [] txg_sync_thread+0x3c5/0x620 [zfs]
Dec 1 11:34:50 srv_cs1 kernel: [] ? txg_init+0x280/0x280 [zfs]
Dec 1 11:34:50 srv_cs1 kernel: [] thread_generic_wrapper+0x71/0x80 [spl]
Dec 1 11:34:50 srv_cs1 kernel: [] ? __thread_exit+0x20/0x20 [spl]
Dec 1 11:34:50 srv_cs1 kernel: [] kthread+0xcf/0xe0
Dec 1 11:34:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
Dec 1 11:34:50 srv_cs1 kernel: [] ret_from_fork+0x58/0x90
Dec 1 11:34:50 srv_cs1 kernel: [] ? kthread_create_on_node+0x140/0x140
I am using RHEL 7.2 with kmod-zfs-0.6.5.8.
[root@srv_cs1 ~]# rpm -qa | grep zfs
zfs-release-1-3.el7.centos.noarch
libzfs2-0.6.5.8-1.el7.centos.x86_64
zfs-0.6.5.8-1.el7.centos.x86_64
kmod-zfs-0.6.5.8-1.el7.centos.x86_64
[root@srv_cs1 ~]# rpm -qa | grep zfs
zfs-release-1-3.el7.centos.noarch
libzfs2-0.6.5.8-1.el7.centos.x86_64
zfs-0.6.5.8-1.el7.centos.x86_64
kmod-zfs-0.6.5.8-1.el7.centos.x86_64
$ uname -a
Linux srv_cs1 3.10.0-327.el7.x86_64 #1 SMP Thu Oct 29 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux
The text was updated successfully, but these errors were encountered: