libusb_set_interface_alt_setting returns kIOUSBPipeStalled on macOS #838
Comments
|
FYI, I can reproduce this with pyusb, so the issue is not specific to dfu-util. raises (I tried adding a |
|
Please help to post the usb descriptor of the device and the debug log. lsusb under macOS: (just install using homebrew: "brew install lsusb", if you use homebrew). Looks like the device FW have a problem to handle libusb_set_interface_alt_setting (libusb_device_handle *dev_handle, int interface_number, int alternate_setting). A simple fix will be changing the code to libusb_claim_interface (libusb_device_handle *dev_handle, int interface_number), if the device has no alternative settings other than the default 0. Ref: |
|
Hi @mcuee, thanks for the quick response. I did some more extensive testing, to see how different devices behave when I try to execute the Python script in comment #2. I list the controller if I was able to find that info for these products. Devices that respond with a access denied error (most likely because the USB interface is already claimed by a driver in the macOS kernel)
Devices that work fine
Devices that give a "Pipe is stalled" error (errno 32)
The issue I have is with the Pytrack and Pysense while in DFU mode as described on the above URL. The relevant line in the Note that I observed this behaviour with the Prolific USB-to-serial cable only connected on the USB side; not console is attached on the DB9 side. Suspicion[Edit] My first suspicion is to suspect an issue with the Prolific PL2303 controller. However, that may be jumping to conclusions. The observation is that these two extension boards (whose controller is unclear to me), while in DFU mode, behave the same as a Prolific USB-to-serial cable when the later is not connected to a device. Why it is behaving as such is unclear. It is also unclear when exactly it is behaving like this. I can reproduce this result with both boards on macOS 10.14.6 and 10.11.6. On the other hand, in the reports from dfu-util, people found that it did work on earlier versions on macOS and/or earlier versions of libusb, and that it also works on Windows and Linux. So my first question would be if someone can reproduce the above results. |
|
Good discussion here: So you can try the workaround, just claim the interface and not set alternative setting. Apparent the device FW is broken in dealing with this standard USB request. From Tormod Volden :
|
|
A STALL packet generally means that the transfer is not supported by the device. In this case, it is a standard USB request on the Control Endpoint (Control transfer). So this is clearly a device problem. But the work-around should be simple, just do not issue the request if it is not needed. |
|
The request seems to be supported, as it works fine on Linux and Windows, and on macOS with older libusb. It could be something special about Freek's device, but Gijs' findings lead me to believe there is something with the libusb darwin backend. |
|
Good point. The device does seem to support the request. On the other hand, since it is specific to come devices, it is possible that device does not support the request twice in a row. Previously I have devices with Microchip PIC USB FW which does not correctly set the toggle bit and it will work one time with libusb and next time it will fail. The work-around is to issue libusb_reset_device. |
|
I pointed to exactly these two changes in the dfu-util ticket and asked them to check if they were the cause, but there hasn't been anyone following up. Is the request sent twice in a row? Is it implicitly sent by another function? Issuing libusb_reset_device can be delicate in the scope of DFU since it can have its own significance for the device, like going to DFU mode or back to RunTime mode. |
|
Let's copy the report from dfu-util Ticket 86 for easier reference. This output is with dfu-util 0.9 and libusb 1.0.20 (suppsed to be true for 1.0.21/1.0.22 as well). |
|
Failed log with 1.0.24. |
|
Original report. |
|
The possible device reset (darwin_reset_device) inside libusb_set_interface_alt_setting is a particuliarity of the darwin backend. Why is it needed on this platform? I wish this was explained in the source. Apparently it seems to work out on libusb 1.0.20, but I guess it is the repercussions of this reset that causes something to fail in 1.0.23. This failure ought to be fixed, but at the same time it would be better if a series of libusb operations would create as much as possible the same USB requests on all platforms, and not add something as significant as USB resets on one platform. |
|
Unfortunately libusb_reset_device bahavior is really platform dependant. Old reference in June 2011: It is still true now that WinUSB backend does not support reset. I believe Linux behaviour is still the same as before. BTW, I have no issues running the pyusb test code for a few devices under the new Mac Mini M1. So this is device specific. @hjelmn --> please help to look into this. Thanks. |
|
I did some more tests with pyusb, basically: For now, three confirmations:
This is tested on macOS 10.14.6 (I wanted to test on macOS 10.11 too, but have not yet found the time). For those who like to reproduce: Install libusb:
Repeat steps 3-7 for each version of libusb you like to test. Run the following test scriptFor libusb 1.0.22, this produces: For libusb 1.0.23, this produces: |
|
Trying to narrow it down:
So I suspect that commit 065e586 contains the root cause. One key change in this code is the following in Before commit: After commit: So the return value was changed from Now, I don't claim to understand the code enough to grasp what |
|
Just letting you know that I was able to do a firmware upgrade with dfu-util, using the fix in PR #843. |
|
The reset is required macOS per the IOKit documentation. The PR is the appropriate fix. |
|
If you feel confident that this is indeed the correct behaviour, I certainly would appreciate it if the PR would be merged, and perhaps not take too long before releasing a new patch version (1.0.25). To give an indication of the number of affected users, these are the reports for dfu-util with pycom devices: pycom-micropython-sigfox#422, Pycom forum thread#5182, thread#5452, thread#5739, thread#6008, thread#6715, thread#6722. Surely the problem is more widespread than just pycom devices (and perhaps dfu-util). For example, the OP of dfu-util#86 mentioned a different device, a Tomu. |
|
Nathan, do you have an URL to this documentation, if it is public? Freek, thanks a lot for bisecting and finding the regression. So to understand this correctly, there is actually a PIPE error from SetAlternateInterface() and device reset, but now the endpoints are found so libusb_set_interface_alt_setting() will return success even if the requested alternate interface was not set? |
|
@tormodvolden Looks like that is true. The device apprarently does not support SetAlternateInterface() properly and the device gets reset. The following code has not been changed from 1.0.22 to 1.0.23. But it is ineed strange that macOS IOkit will require device reset to recover from a pipe stall error. On the other hand, this does have a side effect. Interestingly prior to 1.0.23, the reset actually does not reset as per Issue #455. So the side effect of the code does not really manifest. Anyway, probably dfu-util can just remove the call as it is really not necessary. DFU is anyway mainly deal with control endpoint. |
|
@hjelmn Assuming the reset is indeed as per IOkit documentation (sounds strange), then there is a side effect. So we can probably check the current alternative setting and do nothing (most of the cases) if it is already at the correct alternative setting. Majority the device should only have the defaul alt setting 0. And for those which needs to be set a different alt settings, then the device should anyway supports the call propely. This is not necessary in Linux and Windows WinUSB backend, as there is no side effect of reset. |
|
The other thing is probably to add a debug message here to warn the user that Set Alternative Interface call has failed and device will reset. |
|
@mcuee, the device seems to support libusb_set_interface_alt_setting() since the function returns success on Linux and Windows. But for what I know it can be the backends are hiding a failure there also. And someone with the device can probably confirm that there is no reset on Linux or Windows - the libusb platform code there has no reset in the alternate setting function but I don't know if the kernel would issue a reset as part of the alternate setting ioctl. |
|
I already worked on a patch to avoid unnecessary alternate settings in the dfu-util, but that wouldn't help towards solving the underlying libusb issue. IMO the libusb library should not do any check if it is necessary or not but pass on the request from the program. If the device doesn't support it, the error should be propagated back to the program. Normally I would think the program should then perform the reset if it wants to, but if is this is platform specific as it seems for macOS (I haven't found it documented yet) then I understand the need for "magic" in the platform code, to save the program from dealing with this. |
|
All: thanks for taking this up. It's very much appreciated. However, the real crux is if the device should be reset by libusb upon failure, or if that is the prerogative of the calling software.
I'm leaning a bit towards the "remove As to the question if IOKit documentation demands such a reset, I can't find it, but I only searched the web for methods with the same name, not looked into the actual included library. The documentation I found is sparse or even non-existent (The question why Apple publishes non-existent documentation is another topic ;) ). There is another, seemingly related, |
|
@macfreek Yeah. The documentation is poor and I can't remember where I read that it would need to be reset in this case. The code is working with the proposed PR which should get things working again. I will try to determine if is safe to remove the reset at another time. |
|
Yup. Really bad move on Apple's part. Especially given 10.9's usb stack had a number of regressions. Had to guess where they screwed up. |
|
As for whether we need to have 1.0.25 release to address the anomalies for 1.0.24, I think it is a good idea since this macOS issue seems to affect quite some users. Linux side -- this may affect some users and the fix has been merged. Windows side -- not so sure if #844 has a quick fix or not. I will check with Chris. |
Perhaps you read this line in IOUSBFamily USB.h: |
darwin_set_interface_altsetting no longer returns a pipe stalled error [kIOUSBPipeStalled/LIBUSB_ERROR_PIPE] if the interface is succesfully reset. Closes: libusb#838 Signed-off-by: Freek Dijkstra
Revert unintended change of commit 065e586. darwin_set_interface_altsetting no longer returns a pipe stalled error [kIOUSBPipeStalled/LIBUSB_ERROR_PIPE] if the interface is succesfully reset. Closes: libusb#838 Signed-off-by: Freek Dijkstra
If this is the reason behind the device_reset(), it is maybe enough to call darwin_clear_halt(). I don't have a Pycom board to test on, but I tried on a PL2303 USB-serial adapter (which was reported by macfreek to behave the same in this regard) on Linux after unbinding the kernel driver from the USB interface. Indeed there is a PIPE error, and CLEAR_FEATURE requests to each of the three endpoints in the interface are sent by the kernel, and there is no error from libusb. If I try to set an alternate interface different than zero (not available on this interface) using pyusb, there will be an error but it seems pyusb is doing some checks and will not send an invalid SET_INTERFACE USB request. |
|
I agree that a clear halt will be the normal solution to deal with pipe stall. However, in this case, we are not so sure why last time IOkit documentation was saying that reset is needed, so we have to keep the current reset code (magic code) as of now. We probably have to deal with potential improvement in another time. |
|
Yes, so having a reference to this IOkit documentation would be good (I tried long searching for it on the free internet), with a comment in the code. Further, the code should probably special-case exclusive access error and return appropriate error instead of resetting the device in that case. And actually only do device reset (or clear halt) if there is a PIPE error. Anyway +1 on getting macfreek's regression fix in first. |
Yeah. That line is familiar. For some reason ClearPipeStallBothEnds was insufficient. At the time DeviceReset was likely what we used for reset and it was (and still is) essentially a no-op. We now use re-enumerate for reset. It is worth trying with ClearPipeStallBothEnds on each pipe and see if that works now. I can't check if the darwin kernel is already doing it without looking at the IOUSBFamily source code (which is not available for any recent version). |
|
Wow, that line is 12 years old. I will have to dig into an old Mac OS X version (10.5 or 10.6) for Xcode to find the documentation I was reading. Probably was still using a PowerBook G4 so probably 10.5. This is likely the relevant version: https://opensource.apple.com/source/IOUSBFamily/IOUSBFamily-349.4.3/ |
|
Just to add some OS-agnostic info, clearing a pipe stall is only applicable to bulk/interrupt endpoints. A pipe stall in the case of a failed set interface request would be returned by the control pipe, which never needs to be cleared. As an observer and non-Mac user, it makes no sense to me why any action would need to be taken if a set interface request failed. |
|
A successful SET_INTERFACE request will also clear all endpoints of the interface. So the Linux kernel will do that if the request fails with PIPE error and would otherwise have been a NOP (only one alternate interface). If there are multiple alternate interfaces the PIPE error will be returned. I assume the darwin backend tried to do something similar, but instead of going through the endpoints it was easier to call device_reset() which basically did the same (at some point in time). And the logic for single vs multiple alternate interfaces was not implemented. |
This affects the entire device, which a set interface call should not do. This is unfortunate, even if it is the status quo. |
|
@dickens I agree. I wish I could remember what the reasoning was but 12 years is an eternity in software. The reset was not present in libusb 0.1.12 so I specifically added it when implementing the libusb 1.0 backend. It was undoubtedly to work around some issue seen during development. If there was some sort of bug in that version of IOUSBFamily it is likely long gone (since at least 10.9.0 given the IOUSBFamily rewrite). |
|
For reference for when the reset appeared: |
darwin_set_interface_altsetting() no longer returns the status of the underlying SetAlternateInterface(), but instead the result of a subsequent GetNumEndpoints() call. This reverts to the behaviour prior to commit 065e586. Since darwin_set_interface_altsetting() resets the interface after SetAlternateInterface(), one of the consequences is that if SetAlternateInterface() returns a kIOUSBPipeStalled error, and the interface is successfully reset, darwin_set_interface_altsetting() now returns LIBUSB_SUCCESS instead of LIBUSB_ERROR_PIPE. Closes: libusb#838 Signed-off-by: Freek Dijkstra
|
Wow, that is really the earlier days of libusb-1.0 with Daniel Drake as admin. I just created a new ticket #847 to capture potential improvement in the future. |
|
@mcuee Yup. Been doing this for awhile. Remember discussing the direction to take for libusb 1.0 with Johannes before he handed it off to Daniel. This project is ~ 20 years old now! |
|
Indeed this project is about 21 years old with the first release version of libusb 0.1.0 on 2000-02-16. I only started with libusb, libusb-win32 and pyusb in Oct 2005 because of the involvement in pickit-devel. Now interestingly I am one of the admins of all three projects even though I am not a developer. Time really flies. |
Hi, thanks for making libusb available!
I'm using dfu-util to upgrade the firmware of a development board over USB. For some reasons, that fails on macOS, with the first reports from March 2020. It was suggested that a macOS security update triggered it, but I have not seen anything more clear than that. (Upstream tickets: dfu-util and pycom.)
Tracing down this behaviour, the root cause seems that a call to
SetAlternateInterface()indarwin_set_interface_altsetting()(thelibusb_set_interface_alt_setting()for macOS) returns akIOUSBPipeStalled("pipe is stalled") error. This is repeatable behaviour.My question: is this known or expected behaviour? And how should either libusb or dfu-util handle this?
My apologies for not being more specific, I'm an end-user of this library, so am not sure about the intended behaviour.
Note that the documentation for
SetAlternateInterfacethat I could find, did not specifically mentionkIOUSBPipeStalled. OnlykIOReturnSuccess,kIOReturnNoDevice, andkIOReturnNotOpen.Given the suggestion that this behaviour changed after a security update, I verified on an older macOS system. I can reproduce it on 10.14.6 (Mojave) and 10.11.6 (El Capitan), so I am not yet convinced that this was a trigger.
The text was updated successfully, but these errors were encountered: