Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zpool import read-only #1863

Closed
behlendorf opened this issue Nov 13, 2013 · 8 comments
Closed

zpool import read-only #1863

behlendorf opened this issue Nov 13, 2013 · 8 comments
Milestone

Comments

@behlendorf
Copy link
Contributor

Observed when importing a pool read-only, v0.6.2-97_g09d672d.

VERIFY(zio->io_type != ZIO_TYPE_WRITE || spa_writeable(spa)) failed
SPLError: 9415:0:(zio.c:2613:zio_vdev_io_start()) SPL PANIC
SPL: Showing stack for process 9415
Pid: 9415, comm: z_null_iss/0 Tainted: P           ----------------   2.6.32-220.1chaos.2012010958.16kstack.ch5.x86_64 #1
Call Trace:
 [<ffffffffa03f25a7>] ? spl_debug_dumpstack+0x27/0x40 [spl]
 [<ffffffffa03f3d51>] ? spl_debug_bug+0x81/0xd0 [spl]
 [<ffffffffa059bc81>] ? zio_vdev_io_start+0x2d1/0x2e0 [zfs]
 [<ffffffffa059b5f3>] ? zio_execute+0xb3/0x140 [zfs]
 [<ffffffffa03fc858>] ? taskq_thread+0x218/0x4b0 [spl]
 [<ffffffff814ee400>] ? thread_return+0x4e/0x77e
 [<ffffffff8105fa50>] ? default_wake_function+0x0/0x20
 [<ffffffffa03fc640>] ? taskq_thread+0x0/0x4b0 [spl]
 [<ffffffff81090886>] ? kthread+0x96/0xa0
 [<ffffffff8100c14a>] ? child_rip+0xa/0x20
 [<ffffffff810907f0>] ? kthread+0x0/0xa0
 [<ffffffff8100c140>] ? child_rip+0x0/0x20
@behlendorf
Copy link
Contributor Author

This is reproducible in 0.6.2 so it's not a regression. It appears to occur because it's attempting to initialize a cache device and obviously that's not possible when the pool is readonly. I'm surprised this wasn't noticed before.

behlendorf added a commit to behlendorf/zfs that referenced this issue Nov 13, 2013
When a pool is imported read-only avoid validating any hot spare
and l2arc vdevs.  The validation process can result in the vdev
being reinitialized and a new label being written.  Since the pool
must never be written to the unexpected write IO correctly triggers
a VERIFY in the IO pipeline.

  zpool import -o readonly <pool>
  ...
  VERIFY(zio->io_type != ZIO_TYPE_WRITE || spa_writeable(spa)) failed
  SPLError: 9415:0:(zio.c:2613:zio_vdev_io_start()) SPL PANIC

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1863
@behlendorf
Copy link
Contributor Author

0904c2b appears to work correctly although I'm not 100% sure we can skip all of this validation. This should be perhaps be more targeted. I also don't see why this issue wouldn't impact Illumos/FreeBSD so we should attempt to reproduce it there. @csiden I've checked the Illumos tracker and I don't see anything like this, have you heard of any similar reports?

FransUrbo pushed a commit to FransUrbo/zfs that referenced this issue Nov 16, 2013
When a pool is imported read-only avoid validating any hot spare
and l2arc vdevs.  The validation process can result in the vdev
being reinitialized and a new label being written.  Since the pool
must never be written to the unexpected write IO correctly triggers
a VERIFY in the IO pipeline.

  zpool import -o readonly <pool>
  ...
  VERIFY(zio->io_type != ZIO_TYPE_WRITE || spa_writeable(spa)) failed
  SPLError: 9415:0:(zio.c:2613:zio_vdev_io_start()) SPL PANIC

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1863
@b333z
Copy link
Contributor

b333z commented Dec 7, 2013

I was able to replicate that import panic when readonly with cache vdev on FreeBSD 9.2 RELEASE amd64:

# zfs create tank/test
# zfs create -V 100M tank/test/m1
# zfs create -V 100M tank/test/m2
# zfs create -V 100M tank/test/c1
# zpool create test mirror /dev/zvol/tank/test/m1 /dev/zvol/tank/test/m2
# zpool export test
# zpool import -d /dev/zvol/tank/test/ test
# zpool export test
# zpool import -o readonly=on -d /dev/zvol/tank/test/ test
# zpool export test
# zpool import -d /dev/zvol/tank/test/ test
# zpool add test cache /dev/zvol/tank/test/c1
# zpool export test
# zpool import -o readonly=on -d /dev/zvol/tank/test/ test

Results in this panic:
screenshot_b43_2013-12-07_23 41 28

Have my first nightly running on an OpenIndiana that I setup a while back, so if that works I'll try the same there.

@csiden
Copy link

csiden commented Dec 7, 2013

@behlendorf I tried out the steps @b333z mentioned and was able to reproduce the issue on illumos, so it looks like it's a problem for everyone.

@behlendorf
Copy link
Contributor Author

@csiden Thanks for checking, any thoughts on the fix 0904c2b.

@csiden
Copy link

csiden commented Dec 9, 2013

@behlendorf I asked George to take a look at it and he said he's almost ready with a better fix (he's running regression tests now). I'll post an update when his fix is ready (he says it should be small and simple).

@csiden
Copy link

csiden commented Dec 11, 2013

George's fix for this is in illumos now: illumos/illumos-gate@973c78e

I tested out @b333z's steps with this fix and was no longer able to reproduce the bug.

@behlendorf
Copy link
Contributor Author

@csiden Thanks for the link, I like George's fix much better we'll merge it shortly too.

unya pushed a commit to unya/zfs that referenced this issue Dec 13, 2013
4121 vdev_label_init should treat request as succeeded when pool
     is read only
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Saso Kiselkov <skiselkov.ml@gmail.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/4121
  illumos/illumos-gate@973c78e

Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#1863
ryao pushed a commit to ryao/zfs that referenced this issue Apr 9, 2014
4121 vdev_label_init should treat request as succeeded when pool
     is read only
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Saso Kiselkov <skiselkov.ml@gmail.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/4121
  illumos/illumos-gate@973c78e

Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#1863
behlendorf added a commit to behlendorf/zfs that referenced this issue Jun 9, 2015
When a pool is imported read-only avoid validating any hot spare
and l2arc vdevs.  The validation process can result in the vdev
being reinitialized and a new label being written.  Since the pool
must never be written to the unexpected write IO correctly triggers
a VERIFY in the IO pipeline.

  zpool import -o readonly <pool>
  ...
  VERIFY(zio->io_type != ZIO_TYPE_WRITE || spa_writeable(spa)) failed
  SPLError: 9415:0:(zio.c:2613:zio_vdev_io_start()) SPL PANIC

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1863
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants