zpool replace fails when new dev name is same device - replugged external USB drive using /dev/sdX names cannot be detached/reattached

[zenaan]

System information

Distribution Name | Debian
Distribution Version | 9.5 Stretch
Linux Kernel | 4.9.0-8-amd64
Architecture | amd64
ZFS Version | 0.6.5.9-5
SPL Version | 0.6.5.9-1

Describe the problem you're observing

"External" e.g. USB drive is detached, reattached, gets new /dev/sdX name, and now we cannot cause zfs to reconfigure its device name for this pool (which remains in suspended state) without rebooting!

When oldName and newName refer to the same device, the following ought succeed, but does not:

zpool replace oldName newName

Also, "zpool import newname" ought "know" that the newname is the same as the old name, and automatically "fix up" the pool needing that device.

Additional feature: "zpool import|create name" on Linux ought give a simple little NOTE or WARN that /dev/sdX names can change and that /dev/disk/by-id/ names might be preferable.

See also the following heavy-weight feature requests/ issues (which is still lighter weight than rebooting):
#5242 feature request: allow unloading a suspended pool from memory without exporting first
#2023 feature request: allow unloading a suspended pool from memory without exporting first
#2878 zpool destroy fails on zpool with suspended IO
#6649 "zpool export " on a faulted zpool hangs and blocks other zpool commands after that
zfs-discuss: disconnected/reconnected drive gets new /dev/sdX name, zpool fails to detect or handle the problem...

EDIT: Drat, an even older issue finally appeared, which appears to be the same (dupe alert, and apologies - perhaps this one will help future USB newbie googlers):
Feature: zpool online to rename a disk #3242

And finally see also:
#2076 RAIDZ1: unable to replace a drive with itself

Describe how to reproduce the problem

Create single (USB) drive pool using /dev/sdX name, e.g. sdd, with default failmode=wait.

zpool create usbtank sdd

Physically unplug the USB drive.

Read/write to the pool (this process hangs) e.g.:

echo blah > /usbtank/test.txt

Plug in a different USB drive to make sure the zpool drive gets a different driver letter (can also achieve this randomly, and/ or sometimes just by re-plugging the removed drive, and/ or in a VM).

Plug in the (removed) zpool drive, check it has -different- dev name e.g. sde.

Both of the following styles should work, but do not:

zpool replace -f usbtank sdd sde
zpool replace -f usbtank sdd usb-ST...016742-0:0

Instead an error produced as follows:

invalid vdev specification
the following errors must be manually repaired:
/dev/sde1 is part of active pool 'usbtank'

Workaround: reboot.

Workaround: unplug/ replug USB devices until the old drive has the same device name as it used to have, then run the following (once, I had to run clear twice, perhaps due to the drive being slow to start (not sure):

zfs clear usbtank

Tidy up - fix the device name: AFTER you've got your /dev/sdX pool working "normally" again (by a workaround above), you can fix the device name this way:

zpool export usbtank
zpool import -d /dev/disk/by-id/ usbtank

And of course in future, always add "-d /dev/disk/by-id" to zpool create.

Include any warning/errors/backtraces from the system logs

[zenaan]

See also:

Feature: zpool online to rename a disk #3242

RFC: generic device removal #9129

[stale]

This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.