Prevent race in blkptr_verify against device removal

[pcd1193182]

Motivation and Context

When we check the vdev of the blkptr in zfs_blkptr_verify, we can run into a race condition where that vdev is temporarily unavailable. This happens when a device removal operation and the old vdev_t has been removed from the array, but the new indirect vdev has not yet been inserted.

Description

We hold the spa_config_lock while doing our sensitive verification. To ensure that we don't deadlock, we only grab the lock if we don't have config_writer held. In addition, I had to const the tags of the refcounts and the spa_config_lock arguments.

How Has This Been Tested?

zfs-test, zloop

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the ZFS on Linux code style requirements.
• I have updated the documentation accordingly.
• I have read the contributing document.
• I have added tests to cover my changes.
• All new and existing tests passed.
• All commit messages are properly formatted and contain Signed-off-by.

[sdimitro]

From a correctness standpoint this makes sense to me. I just have a nit and two questions below:
[1] Do you have a stack trace or anything from the original issue that we add in the description of the PR or something, so it is at least a little bit searchable.
[2] Do we know if that change has any performance implication due to any introduced lock contention between zio_reads() that try to grab the lock under different circumstances (e.g. some reads are blocked because they want to grab that lock as writers and wait for all the reads that hold the lock as readers to exit the critical section)...or that's not even possible?

[pcd1193182]

@sdimitro 1 is done, 2 I haven't tested, but it shouldn't be much of an issue because we grab this config lock else elsewhere in the IO path. This is the lowest level config lock, and it shouldn't have much contention on it.