Always use an overlapped structure when calling DeviceIoControl #30
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This change fixes deadlocks occurring when communicating with the WNBD driver using DeviceIoControl[1][2]. We're setting FILE_FLAG_OVERLAPPED when opening the WNBD device in order to be able to submit multiple requests simultaneously. The Windows docs state that in this case, an OVERLAPED structure must be passed to functions such as ReadFile or WriteFile without explicitly mentioning DeviceIoControl. Apparently DeviceIoControl can hang if we don't pass an overlapped structure, especially when having a high number of concurrent requests. This is likely related to the event wait involved. We're going to update the libwnbd *Ioctl* functions, adding an optional LPOVERLAPPED argument. When not provided by the caller, we're initalizing an internal OVERLAPED structure and wait for the operation on the behalf of the caller, mimicking the Windows behavior. We're trying to re-use overlapped structures and events as much as possible, especially in the IO path. The IO dispatch loop will reuse the same events but we can't do much about "WnbdSendResponse" since that will be called by libwnbd consumers from arbitrary threads. We're adding "WnbdSendResponseEx" though, letting the caller provide an overlapped structure. Another advantage of adding overlapped parameters is that we allow libwnbd consumers to perform any *Ioctl* call asynchronously. [1] profiler counters: http://paste.openstack.org/raw/797772/ [2] wnbd / ceph counters: http://paste.openstack.org/raw/799072/ Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
|
Acked-by: Alin Gabriel Serdean aserdean@cloudbasesolutions.com |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change fixes deadlocks occurring when communicating with the
WNBD driver using DeviceIoControl[1][2].
We're setting FILE_FLAG_OVERLAPPED when opening the WNBD device
in order to be able to submit multiple requests simultaneously.
The Windows docs state that in this case, an OVERLAPED structure
must be passed to functions such as ReadFile or WriteFile without
explicitly mentioning DeviceIoControl.
Apparently DeviceIoControl can hang if we don't pass an overlapped
structure, especially when having a high number of concurrent
requests. This is likely related to the event wait involved.
We're going to update the libwnbd Ioctl functions, adding an
optional LPOVERLAPPED argument. When not provided by the caller,
we're initalizing an internal OVERLAPED structure and wait for
the operation on the behalf of the caller, mimicking the Windows
behavior.
We're trying to re-use overlapped structures and events as much
as possible, especially in the IO path. The IO dispatch loop
will reuse the same events but we can't do much about
"WnbdSendResponse" since that will be called by libwnbd consumers
from arbitrary threads. We're adding "WnbdSendResponseEx" though,
letting the caller provide an overlapped structure.
Another advantage of adding overlapped parameters is that we allow
libwnbd consumers to perform any Ioctl call asynchronously.
[1] profiler counters: http://paste.openstack.org/raw/797772/
[2] wnbd / ceph counters: http://paste.openstack.org/raw/799072/
Signed-off-by: Lucian Petrut lpetrut@cloudbasesolutions.com