Slow listing of usb devices -- and potential libusb improvement to make netlink runtime selectable #237
Labels
Comments
|
Hello, |
|
Yes, and that libusb-1.0 hotplug mechanism is unusable too. As wrote I think the whole libusb is slow due to asynchronous code. So hotplug (as async too) cannot help here. Here is testing code: And here is combined output from dmesg and above testing code: As you can see there is just 0.3s window (from 2732.336276 to 2732.654600) to detect and immediatelly open usb device "Nokia USB ROM". And this cannot be done by hotplug or other asynchronous waiting. Busy waiting needs to be used for such action. Above libusb-1.0 hotplug test code detected just usb device "Nokia N900 (Update mode)" and it was later then 1s, which is slow as hell and absolutelly unusable. Note that libusb 0.1 does not have this problem and can dectect & open usb device "Nokia USB ROM" in that 0.3s window. |
|
I thiink you need to provide more detail what you want to do. And it is good to provide the debug log (set LIBUSB_DEBUG=4 and then run your libusb program) to see where is the delay. And you can even provide the debug log of your libusb-0.1 program as well (set USB_DEBUG=255 and run your libusb-0.1 program). |
|
BTW, what is your Linux version? As far as I know, majority of the Linux distro use libusb-compat-0.1 which is based on libusb-1.0. |
|
BTW, I think this is better discussed in the mailing list. You need to subscribe to the mailing list in order to post, please go to this URL. When your previously posted, it did not make to the mailing list. Even though the other user forwarded it to the mailing list, it does not have the details. Once you provide more details, I think there will be more meaningful discussions in the mailing list. |
|
Here is combined log from dmesg and above testing hotplug code. It was started with env LIBUSB_DEBUG=4 and against libusb-1.0.20 compiled from tarball: As you can see libusb-1.0 detected some device after "Nokia USB ROM" was connected, but it is slow and just received detached=1 event... So it is unusable... And about libusb 0.1, I'm using real libusb 0.1, not libusb-compat around libusb-1.0. |
|
Here is libusb 0.1 code which use busy waiting: It detect "Nokia USB ROM" usb device without any problem. Here is combined output from dmesg and this libusb 0.1 code: As you can see libusb 0.1 (real!, not compat wrapper) is working fine and can detect usb devices in time. libusb-1.0 either hotplug code or similar busy waiting loop is slow and so is unusable... |
|
Can you post the debug log for libusb 0.1 as well? Thanks. You can set USB_DEBUG=255 for that purpose. |
|
Setting this as Linux for now. |
|
I do not know why you need that log as libusb 0.1 is working and such busy waiting log is long, but here is (again combined with dmesg): |
|
@mcuee Thats all? Or do you need anything else? |
|
Yes that is all. I am thinking that your specific use case may be which libusb under Linux may not be able to support. I could be wrong. Maybe it is better you use the mailing list to tell a bit more details what is your use case and then probably there is a way out. |
|
Use case (as already wrote): start communication with device at correct time. If computer does not negotiate device protocol in 0.3s time window, then device disconnect from usb. I already sent email to ML at least two times and other people forwarded it to ML again. |
|
You need to subscribe to the mailing list in order to post. Anyway, your
description for the use case is not that clear, what do you want to achieve
in the end? If you provide more details and ask in the mailing list, maybe
there are ways to sort out (could be even non-libusb solutions).
|
|
What I need to achieve? Communicate with that device ("Nokia USB ROM ") which is on usb bus just for 0.3s. |
|
Communicate with the device for what?
|
|
Communicate with the device for what?
Eg: Software update?
|
|
Short: Query various parameters, change boot settings or load own boot code. But reason for device communication is irrelevant in this bug report. libusb 1.0 does not even detect usb devices! |
|
Having all these details may help. Anyway, I can sense your frustration. I'll shut up for now to see if others can help you better. |
|
any progress on this? I just read this discussion and the thread on maillist and... Can't say for @pali, but I already have too big incoming traffic from many other maillists to subscribe to one more just to track one single issue. Well, yes, I understand that it is "this is opensource, baby, so patches or gtfo" opinion exists, but libusb is too important system package on GNU/Linux these days to just ignore this issue. This exact issue brakes use-case of flashing very mush devices (not only nokia, but also some mediatek ones, and even more). And standing to "non-libusb-way" is bad option too, imho. So, can you, please, re-implement the blocking method like it was in libusb0 for this very cases? |
|
I'm the Arch Linux maintainer of a package affected by this bug and developed by @pali. The fact that the package is affected by this bug makes the whole packaging difficult and error-prone because libusb0 is not shipped by the distribution anymore. Could you please look into this issue that has been open for such a long time? And, as for the suggestion about the non-libusb solution, libusb has been employed for the means of flashing devices ever since I remember, even by the old, Nokia-made flashers. |
|
@pali are you using the udev or netlink option for hotplug? If using udev, what happens if you switch to netlink? |
|
I'm using code from comment #237 (comment) As you can see I have not specified udev or netlink in libusb_hotplug_register_callback() call. |
|
Pali, as I said I got a working test result with Fedora libusb, but after checking the dmesg I get below, so apparently my time slot window is longer than yours, it is about 2.83 seconds, I do not know why the difference happens, my kernel is 4.13.0-rc6+: |
|
Time slot is not longer, look at output again: It is just 0.3s before device leave and disconnects. In log is missing line |
|
@pali this is a compile-time option, not something you specify at runtime. Looking at the log debug log again I see udev messages. Try this again after compiling libusb with the --disable-udev option when running the configure script. |
|
Here is the output of my test result with pali's test code: OS: Fedora rawhide, mainline kernel 4.13.0-rc6+ |
|
@pali, I've just recalled that I at one point had libusb-compat before I replaced it with libusb0 and successfully flashed the device. I guess it was a Nokia N950, though, not N900. Should I try with an N900 + libusb-compat to try to reproduce it on my side? |
|
Reopen this issue. PRs are welcome to address this issue. |
|
@mcuee Application developer which use library X is not interested what is in library X. If you show me how to update example from #237 (comment) to not use that udev I can try it. |
No need to change your code. Just try to build libusb with '--disable-udev' option. |
|
Seems that you misunderstood, I'm not building libusb. I'm building application and I'm using libusb which is installed in the target system. |
Please just give it a try. If that works you have a workaround, you can ship your application with a libusb library with the custom build option. You can call it another name as well. I tend to think that is a better workaround than shipping with the legacy libusb-0.1. |
Which major linux distribution is doing this? Please no kidding. This is ridiculous advice. Have you already spoke with linux package maintainers with this? Why on the earth then I should switch from fully working libusb-0.1 to libusb-1.0 which requires 1) special compilation, 2) requires bundling of whole libusb-1.0 code into my application and 3) modifying whole build system to handle this crap? |
workaround? No! After 7 years nobody is interested in workaround, but in resolving problem. |
|
Anyway, the other idea is to change your code to use the following API to see if that helps. |
|
Could you show example similar to #237 (comment) ? |
Sorry but you have to try yourself. I am not a programmer myself. Or maybe @benzea or @tormodvolden can chime in to see if that is a viable option or not, and suggest the modification needed if it is potentially viable. |
|
@pali |
|
I just do not understand you, so could you be please so kind and write me a list of words which are like ridiculous? And what has your last post in common with libusb issue here that libusb is slow in listing of usb devices? I'm feeling that you do not like this bug report, trying to be off-topic and now trying to convince me to that I'm like an idiot here. So back to the issue, that new API seems to be also unusable as it is linux-only? Why? |
|
I am refering to your comment above. You used the word
|
udev is also Linux only. Do you have issues under other OS like macOS or Windows? |
I can understand your frustration, but comments like this will get you nowhere, when you are talking to an admin of a project. To be clear, I am not trying to be off-topic. And I have no idea why you think in that way. Anyway stop this.kind of off-topic comments. |
|
Another thing, consider this as a limitation of libusb and libusb-compat-01 under Linux now. Please refer to the last point in libusb FAQ. Again PRs are welcome to enhance libusb project for this kind of use case. |
To be honest, almost none of them ship libusb-0.1 these days as well 🤷 And yes, I also personally don't like the idea of bundling (as well as it is "bad way policy" in many distros. |
By the way, are they? I mean: if I'll spent a significant amount of time to dig into this, and will made a PR with such low-level API, what is the probability for it to be merged (especially in the cases that: 1) I'm not a professional programmer (so code will need a qualified review, and I may require some advices), 2) if it will happen at all, it can take about a few months, or even half-a-year, as I'm too busy at my jobs ATM)? |
FYI Debian/Ubuntu still ships legacy libusb-0.1. I am just providing a potential work-around. As of now there is no solution so that I am pointing out potential work-around. I can unserstand that you do not like the work-around though. Therefore I am mentioning another potential work-around of using libusb_wrap_sys_device() from the application side. But someone needs to try it out. |
Yes, PRs are welcome.
Fair enough. Again,the issue will still be there without people wokring on it. Right now the No 1 issue with libusb project is the lack of developers/contributors. |
So, my thought is that this is a bit of a fringe use case. However, it is valid, so the question is how one should deal with it in the future. Adding a more low level API could be a way to fix this. The other way to fix this is creating a udev rule that does the basic stuff. So, I would say:
Note that if the API from 1. exists, you could implement it yourself, by watching |
I tend to think that libusb_wrap_sys_device() is potentially the candidate here. It is mostly used by Android but can be used for Linux as well. Wrap a platform-specific system device handle and obtain a libusb device handle for the underlying device. The handle allows you to use libusb to perform I/O on the device in question. Call libusb_set_option() with the parameters (NULL, LIBUSB_OPTION_NO_DEVICE_DISCOVERY) before libusb_init() if you want to skip enumeration of USB devices. In particular, this might be needed on Android if you don't have authority to access USB devices in general. On Linux, the system device handle must be a valid file descriptor opened on the device node. The system device handle must remain open until libusb_close() is called. The system device handle will not be closed by libusb_close(). |
Aha, that was what I was looking for but didn't find! |
|
So looks like I am not a porgrammer so that I can not give you the code modification needed for the test codes, but you can refer to the libusb example codes here. Or here (for Android) If you need more guidance on the code modification, @benzea or @tormodvolden may be able to help you better. Here is the test run under Ubuntu 20.04. |
|
I can tell you a couple things about major Linux distributions: 1) They don't care much about old exotic hardware, and 2) they increasingly ship bundles of applications with custom-built libraries ("snap", "flatpak", etc) instead of using one global library version. However, we are open for contributions to make libusb work better also in this case. It has been suggested to make netlink runtime selectable. But I don't know if anyone is interested enough to implement this. libusb_wrap_sys_device() can be very useful, for instance in combination with udev scripts. I have used it to open devices directly without going through enumeration, and such programs can launch, open a device and complete a USB transfer all within a few tens of milliseconds. |
mcuee
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Sent to ML, but after 8 months without any answer: https://sourceforge.net/p/libusb/mailman/message/34985373/
The text was updated successfully, but these errors were encountered: