on Android USB 3.2, the example capture constantly report "bulk_transfer returned error, error: Out of memory"
#4091
Labels
Comments
|
Hi @GucciPrada |
|
Hi,@matkatz thanks for your reply. I just read that similar issue you mentioned. For additional information, |
|
I tried to reproduce this issue on Pixel 2 (Android 9), Samsung S8 (Android 9) and FireFly RK3399 (Android 8.1) with no success. |
Thank you! That is quite informative. What about the USB 3.0 cable or adapter you used? any advice or recommendation? |
|
I'm not sure what type my cable is it but you can probably test your cable with realsense-viewer over Windows/Linux |
|
OK, Thank you! I tested my cable with realsense-viewer on Macbook. I'll test it on Windows later. |
|
I tried with my new Android Phone, Honor V20 (another brand of Huawei), the device was enumerated as usb2.1 device but actually the Phone is declared to support USB 3.1 Gen1. And this time, the same 'out-of-memory' problem happened with firmware 5.11.4 & librealsense 2.22.0 or 2.21.0. Here is the log while running firmware 5.11.4+librealsense 2.21.0: Click me to expandAnd I then tried to use firmware 5.11.4 + librealsense 2.19.2, it can work correctly, delivering 640x480 RGBD@30Hz, although the device was enumverated as usb 2.1 device. Here is the log: Click me to expand |
|
Encountered this issue as well, using D435 + Pixel3 and D435 + Samsung S7 (same results). Both devices are not rooted, and are detected as usb 210. Firmware 5.11.04, librealsense 2.22. Tried both For Next, I tried building the AAR and created my own app. Same result for both devices. Let me know if you want me to try something else. See sample log below, if it helps: capture app logs for D435 + S7 |
|
Thanks for the inputs! |
|
There is a change on USB transfer method on v2.21.0 that might cause this issue. |
|
Happy to share that your branch works! I have tested it on S7 + D435, no errors in the log. Will test it on the Pixel3 on Monday when I have the device with me. It worked with all three streams (color, depth, infrared) turned on simultaneously at low resolution and 15 fps each. Thanks so much for the fix! |
|
Thanks! I'll give it a try and report the results at here. |
|
I tried librealsense 2.20.0 + firmware 5.11.04 on Honor V20 android phone, the example Click me to expandhowever, I am not sure whether the third to last line in the above log is critical or could be ignored. |
|
And the example app Click me to expand |
|
Hi @matkatz , I've checked the cable and the phone using both a windows tool named 'ChipGenius' and the MacOS X System Report App on MacBook Pro. On windows, I plugged my Honor V20 android phone into the USB 3.2 type-c port of the PC laptop, using the same c-to-c cable that I used to connect the android phone with the D415 camera. And the ChipGenius give me a USB detection result of the Honor V20 phone as follow: As you can see, the phone + the cable is recognized as USB 3.1 device. on MacBook Pro (Retina, 13-inch, Early 2015 version), I got almost the same result:
So why it's Added: |
|
Thanks for the feedback! I'm testing the library on 4 different platforms:
On all of those systems the camera(D415 FW 5.11.4.0) is identified as USB 3.2. I'm still not capable to reproduce the memory issue on those platforms, currently I'm working on both directions:
|
|
I tried to build the example
You could see that the frames were fitfully updated, the screen got stuck for a while and got updated later again, off and on repeatedly. The warning log seems to be similar to issue #3612. But after switching librealsense to v2.20.0, and build the |
|
I was finally able to reproduce the issue and able to see where is it coming from. To solve that I changed the read buffer size from default hard coded size to a value that is determined according to the current stream resolution. Reducing the buffer size allows to configure and stream lower resolutions in case of “Out of memory” error. I also tried to reassemble the frame from few smaller USB payloads but that didn’t work for me (I think it requires isochronous endpoint which is not supported by Android). BTW, a reboot to the device that the issue was reproduced on (Pixel 2 - Android 9), eliminated the issue. I will keep looking into this but first I want to apply the fix if it's mitigates the issue. Once again I will ask you to check my fix before I push it to the development branch. This is the branch: android-read-length-fix Thanks in advance! |
|
That's great! I'd like to try it out later in this day |
|
The Logs on first run, with D435 already attached to USB portLogs when D435 is pulled and re-attached to the USB port |
|
I got a similar result to the one from @dreifelix. I was running the example app The critical error in logcat: Click to expand the detailed log |
|
Thanks again for your help! @dreifelix, do you see the same behavior with the Pixel? did you try to reboot the phone? I also want to share with you the reason for trying to solve this issue and not revert the backend changes. This is the performance of v2.23.0 (when it does work (: ) While this is v2.20.0 results: I'll keep digging... |
|
Rebooting helped! :) In my S7, I am able to stream up to 60 fps when running 2 streams at a time, and up to 30 fps when running all 3 streams simultaneously, at the lowest resolution (424x240 for color, 480x270 for infrared and depth). At 640x480 resolution, only infrared works, up to 30 fps. I noticed the performance improvement as well. I wasn't able to reach the above frame rates in the As you noted, I am unable to run any of the streams at 6 fps at any resolution, but that's fine :) I will try it on the Pixel 3 when I get the chance - i'm just borrowing the device from another project team. Looking forward to have your changes merged in the next release. Thanks so much for looking into this! |
|
Thanks @dreifelix @GucciPrada, I guess reboot didn't help with the P20? |
|
Hi @matkatz I've never tried to reboot the phone. I'll try it today. |
My device is Odroid N2 running android 9 and I am using SDK v 2.30. I have pasted logs on separate issue thread #5328 |
|
@matkatz @RealSenseCustomerSupport @dorodnic @RealSense-Customer-Engineering I have written custom android code to recover the device using hardware_reset API provided by librealsense SDK but it keeps on trying to recover device and I have also tried it with and without Odroid's solution to reset the USB hub but without much success as even after reboot most times at least one the stream is stuck. The code for this class is as below. Following this microsoft/Azure-Kinect-Sensor-SDK#485 , https://stackoverflow.com/questions/23005708/kernel-crashes-due-to-oom-error-usb-submit-urb , https://discuss.aerospike.com/t/how-to-tune-the-linux-kernel-for-memory-performance/4195 I increased usbfs_memory_mb value. Following https://www.bo-yang.net/2015/03/30/debug-kernel-space-memory-leak I looked for any memory leaks. The plots and logs are attached below. Part of dmesg from android during error is attached below. The logcat messages generated by librs is attached below. I have asked this forum multiple times but haven't heard anything promising. Given the issue was first reported almost 6-7 months back and has not been resolved yet, I hope someone soon takes a look at this issue and gets this resolved. |
|
@alowenst01 please take a look. |
|
@alowenst01 @matkatz Was isochronous data transfer ever tried for Android? I found a repo where someone implemented the same for Android. https://github.com/Peter-St/Android-UVC-Camera |
|
UPDATE
Basically currently my code is just getting the depth and colour frame with just some post processing filter and nothing else. Top command shows the CPU usage to be around 40-45%. Earlier I was also drawing the eachb colour frame to UI and usually the app would crash in minutes. It means although the data transfer is the culprit and the memory leak is not in the SDK itself but in the way how the RSUSB is implemented. When the CPU is not stressed then RSUSB can work fine but when it is stressed the USB request fail and then I believe the method of cancelling USB request, that should cancel the filled URB queues held with the linux kernel is not working properly releasing the queue and thus causes Out of memory error. As expected in such case hardware reset or even custom USB controller reset is not able to recover and only option is to reboot the entire system. |
|
@kafan1986 Glad to see 720p 15fps work for you. We did see some stability issue with HD or FHD resolution due to Android platform limitation. So please use the working configuration as workaround at this point. Thanks! |
Actually I posted a little earlier. Although the camera was working but the colour frame was stuck, which I did not know at that point of time. Currently at 720p the device works from 4-7 hrs before it eventually faces the colour frame stuck or out of memory error. Happens in 1 - 1.5 hrs when set up at 1080p. |
|
@kafan1986 So previously what you said "I am running streaming depth and colour frame on android for last 36+ hours at 720p 15fps without any crash" is not always what you get, right? What format did you set for color frame? Could you please try YUYV format to see if any improvement? |
@RealSenseCustomerSupport At 720p colour frame stream this eventually happens every 5-7 hours and at 1080p it happens within 1-2 hours. In all above cases, the only confirmed way to recover is to reboot the entire system. "Resetting" device through official API or even resetting USB controller on the system does not guarantee recovery. Also, in the meanwhile can you update the status of the team's internal status on issue: (DSO-13539) - [Android] Camera disconnected after streaming some duration with Android Camera Sample |
|
@matkatz @RealSenseCustomerSupport @RealSense-Customer-Engineering @dorodnic Sorry for the long silence. I changed my GitHub name (used to be @xtrawurst). We have tried the latest 2.32.1 librealsense and we still get frequent loss of connection on the Samsung Galaxy Tab S4 running Android 9. Interestingly we sometimes get a more stable connection using the "RS Camera" Android app instead of our capturing app, even if we use the same configuration (resolutions and frame rate). Would it be possible to open-source the RS Camera app code so the community can see how it performs configuration etc? Update: This seems to be a cable issue! Using a proper Thunderbolt 3 cable instead of anything labeled "USB" (even USB 3.2 compliant cables) makes all the difference. I couldn't reproduce any of the above problems when using a Thunderbolt 3 cable. Update 2: Actually it seems to be more subtle than that. While I haven't found a Thunderbolt cable that doesn't work, I have also found a USB 3.1 Gen2 compliant C-to-C cable that does work: https://www.amazon.com/AmazonBasics-Double-Braided-Nylon-Type-C/dp/B07D7RZ1VS |
The RS Camera app code is already present in the github. The camera app that builds the SDK is the official RS camera app code. I have tested two approach to create a pipeline of depth and RGB data. Although there should not be any difference in either approach A or B but after months of testing. I can say approach A is OK, only if you are not doing much on the android platform. Once the CPU cores are stressed even a little, maybe due to some parallel processing, it crashes with all sort of USB data transfer issues. The 2nd approach (B) is the one used by the RS camera app. With this I can run my system somewhat stable at (720p colour + depth) at 15 fps stable for 7-8 hours before the crash and I need to reboot the system. I can not use thunderbolt cable as I am using the depth camera with single board computer (SBC) running android and it only has USB type A ports. Also, the thunderbolt cable will probably increase my setup cost given my future use case. I have used the below cable (10 gbps) for my use case and another USB 3.0 (5 gbps) cable and both of them provided same stability and none resolved issue in entirety. |
Closed
|
@GucciPrada Any questions about this issue? @kafan1986 Did you try color using YUYV format to see if any improvement? Thanks! |
|
Due to the problem and possible solutions mentioned in this issue, I finally decided turn to Linux, give up trying android. |
I am using YUYV and the error is still there. At lower frame rate i.e. 15 FPS and 720p color + 720p depth. The error occurs after every 6-7 hours and then I need to restart the entire system. |
|
@kafan1986 Did you try to lower the resolution such as 640x480 to see if any improvement? Thanks! |
I am already using lower resolution of 720p rather than 1080p. Any lower resolution will impact my other deep learning models. |
|
@GucciPrada Sorry for the inconvenience on android platform. If lower resolution on android can't be accepted for you, then switch to Linux will be good choice. @kafan1986 Sorry for the inconvenience. For the issues on android platform, our engineering team did investigations including the analysis of USB trace, however we didn't find clear clue to resolve it from librealsense. It might need the investigation from android host which is out of our scope. Sorry for that. |
|
@GucciPrada Any other questions about this? Thanks! |
|
Nope, I close this issue for now. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Issue Description
short version:
I ran the wrappers/android/examples/capture app on a Android Phone(has a usb 3.1 compatible type-c female interface) with D415 plugged in. But there was only black screen on the UI, which means the text "Connect to a Realsense Camera" was gone but no video data rendered. And at the same time, I found in the logcat of Android studio there reported constantly a plenty of warning messages:
the full log was something as follow:
Click me to expand
My Question is: Has anybody ever seen this problem before? What should I do to fix this?
Longer version:
My setup:
"Android Phone Huawei P20 + USB 3.1 Gen 2 C Male to C Male cable + Realsense D415".
I've smiply checked the cable, it is compliant with USB 3.1 Gen 2 10/20Gbps 5A 100W.
I've also checked the Huawei P20's type c interface by plugging it into my Macbook and checking the 'USB 3.0 Bus' section of System Report of the MacBook. It showed the Speed is up to 5Gbps of "Huawei P20+ the c-to-c cable" combination. I guess that means either Huawei P20 or my Macbook has a USB 3.1 Gen 1 or USB 3.0 interface.
What I did and what I saw:
I downloaded the librealsense-2.22.0.aar and librealsense-2.22.0.zip from dl.bintray.com.
After that, I import them into the android project of wrappers/android, making both examples/native-example and examples/capture Modules to depend on them.
Then I modified the examples/native-example's source code to enumerate the device, the sensors and the stream profiles of each sensor. And save the enumerated result to a log file: usb3.2-P20-enumerate-log.txt
In a word, I could tell that the highest profiles (1280x720xZ16x30Hz Depth stream and the 1920x1080xRGB8x30Hz Color stream) are available.
In the next step, I build and ran examples/capture on the Huawei P20. After granting the USB permission the capture app, the screen was black and the message
“27981-28475/com.intel.realsense.capture W/librs: bulk_transfer returned error, endpoint: �, error: Out of memory” was flooding the Logcat.
The text was updated successfully, but these errors were encountered: