Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Avoid vq_lock drop in vdev_queue_aggregate(). #12297

Merged
merged 1 commit into from
Aug 17, 2021
Merged

Avoid vq_lock drop in vdev_queue_aggregate(). #12297

merged 1 commit into from
Aug 17, 2021

Conversation

amotin
Copy link
Member

@amotin amotin commented Jun 29, 2021

vq_lock is already too congested for two more operations per I/O.
Instead of dropping and reacquiring it inside vdev_queue_aggregate()
delegate the zio_vdev_io_bypass() and zio_execute() calls for parent
I/Os to callers, that drop the lock any way to execute the new I/O.

How Has This Been Tested?

On 80-thread FreeBSD system doing random 4KB ZVOLs writes profiler shows reduction vq_lock spinning time by 15%.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
  • Documentation (a change to man pages or other documentation)

Checklist:

@amotin
Avoid vq_lock drop in vdev_queue_aggregate().
d53b0cc 
vq_lock is already too congested for two more operations per I/O.
Instead of dropping and reacquiring it inside vdev_queue_aggregate()
delegate the zio_vdev_io_bypass() and zio_execute() calls for parent
I/Os to callers, that drop the lock any way to execute the new I/O.

Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Sponsored-By: iXsystems, Inc.
@amotin amotin added Type: Performance Performance improvement or performance problem Status: Code Review Needed Ready for review and testing labels Jun 29, 2021
bwatkinson
bwatkinson approved these changes Jun 29, 2021
View reviewed changes
Copy link
Contributor

@bwatkinson bwatkinson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amotin, this is awesome! I have actually been looking at the vq_lock myself for the O_DIRECT work PR #10018. I have been working on reducing lock contention as well on the vq_lock.

@behlendorf behlendorf added Status: Accepted Ready to integrate (reviewed, tested) and removed Status: Code Review Needed Ready for review and testing labels Jul 26, 2021
@mmaybee mmaybee merged commit 7f9d9e6 into openzfs:master Aug 17, 2021
@amotin amotin deleted the vq_lock_reenter branch August 24, 2021 20:17
behlendorf pushed a commit to behlendorf/zfs that referenced this pull request Aug 24, 2021
@amotin @behlendorf
Avoid vq_lock drop in vdev_queue_aggregate()
8c207f1 
vq_lock is already too congested for two more operations per I/O.
Instead of dropping and reacquiring it inside vdev_queue_aggregate()
delegate the zio_vdev_io_bypass() and zio_execute() calls for parent
I/Os to callers, that drop the lock any way to execute the new I/O.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Mark Maybee <mark.maybee@delphix.com>
Reviewed-by: Brian Atkinson <batkinson@lanl.gov>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Closes openzfs#12297
behlendorf pushed a commit that referenced this pull request Aug 31, 2021
@amotin @behlendorf
Avoid vq_lock drop in vdev_queue_aggregate()
c066ab1 
vq_lock is already too congested for two more operations per I/O.
Instead of dropping and reacquiring it inside vdev_queue_aggregate()
delegate the zio_vdev_io_bypass() and zio_execute() calls for parent
I/Os to callers, that drop the lock any way to execute the new I/O.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Mark Maybee <mark.maybee@delphix.com>
Reviewed-by: Brian Atkinson <batkinson@lanl.gov>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Closes #12297
tonyhutter pushed a commit to tonyhutter/zfs that referenced this pull request Sep 15, 2021
@amotin @tonyhutter
Avoid vq_lock drop in vdev_queue_aggregate()
c600f06 
vq_lock is already too congested for two more operations per I/O.
Instead of dropping and reacquiring it inside vdev_queue_aggregate()
delegate the zio_vdev_io_bypass() and zio_execute() calls for parent
I/Os to callers, that drop the lock any way to execute the new I/O.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Mark Maybee <mark.maybee@delphix.com>
Reviewed-by: Brian Atkinson <batkinson@lanl.gov>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Closes openzfs#12297
rincebrain pushed a commit to rincebrain/zfs that referenced this pull request Sep 22, 2021
@amotin @rincebrain
Avoid vq_lock drop in vdev_queue_aggregate()
c473df4 
vq_lock is already too congested for two more operations per I/O.
Instead of dropping and reacquiring it inside vdev_queue_aggregate()
delegate the zio_vdev_io_bypass() and zio_execute() calls for parent
I/Os to callers, that drop the lock any way to execute the new I/O.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Mark Maybee <mark.maybee@delphix.com>
Reviewed-by: Brian Atkinson <batkinson@lanl.gov>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Sponsored-By: iXsystems, Inc.
Closes openzfs#12297
datacore-rm pushed a commit to DataCoreSoftware/openzfs that referenced this pull request Apr 1, 2022
@amotin @datacore-rm
Avoid vq_lock drop in vdev_queue_aggregate()
7dd7baf 
vq_lock is already too congested for two more operations per I/O.
Instead of dropping and reacquiring it inside vdev_queue_aggregate()
delegate the zio_vdev_io_bypass() and zio_execute() calls for parent
I/Os to callers, that drop the lock any way to execute the new I/O.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Mark Maybee <mark.maybee@delphix.com>
Reviewed-by: Brian Atkinson <batkinson@lanl.gov>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Sponsored-By: iXsystems, Inc.
Closes openzfs#12297
datacore-rm pushed a commit to DataCoreSoftware/openzfs that referenced this pull request Apr 1, 2022
@amotin @datacore-rm
Avoid vq_lock drop in vdev_queue_aggregate()
dab40b1 
vq_lock is already too congested for two more operations per I/O.
Instead of dropping and reacquiring it inside vdev_queue_aggregate()
delegate the zio_vdev_io_bypass() and zio_execute() calls for parent
I/Os to callers, that drop the lock any way to execute the new I/O.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Mark Maybee <mark.maybee@delphix.com>
Reviewed-by: Brian Atkinson <batkinson@lanl.gov>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Sponsored-By: iXsystems, Inc.
Closes openzfs#12297
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@behlendorf behlendorf behlendorf approved these changes

@bwatkinson bwatkinson bwatkinson approved these changes

@mmaybee mmaybee mmaybee approved these changes

Assignees

@mmaybee mmaybee

Labels
Status: Accepted Ready to integrate (reviewed, tested) Type: Performance Performance improvement or performance problem
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@amotin @behlendorf @bwatkinson @mmaybee