Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Android 11+ Too many receivers, total of 1000, registered for pid #1136

Closed
atrope opened this issue Feb 19, 2024 · 21 comments
Closed

Android 11+ Too many receivers, total of 1000, registered for pid #1136

atrope opened this issue Feb 19, 2024 · 21 comments
Assignees
@brianquinlan
Labels
package:cronet_http type-bug Incorrect behavior (everything from a crash to more subtle misbehavior)

Comments

@atrope
Copy link

atrope commented Feb 19, 2024 •
edited
Loading

Hi,

I wanted to report an issue we've encountered with the cronet_http package in Flutter. We've successfully implemented file downloads using cronet_http within Isolates, and everything functions as anticipated during testing. However, upon deploying our application to production, we've started experiencing crashes that we're unable to replicate in our development environment.

This issue appears to be specific to devices running Android 11 and newer. I've attached the stack trace for reference.

Thank you for your assistance.
suamusica.suamusicaapp_issue_b977b22a18970e5a5cfaacac9f8100f6_crash_session_65D35FC702CE0001166B748B6186F256_DNE_1_v2_stacktrace.txt
@atrope atrope added package:cronet_http type-bug Incorrect behavior (everything from a crash to more subtle misbehavior) labels Feb 19, 2024
@atrope atrope changed the title Android 12+ Too many receivers, total of 1000, registered for pid Android 11+ Too many receivers, total of 1000, registered for pid Feb 19, 2024
@atrope
Copy link
Author

atrope commented Feb 20, 2024

@brianquinlan do you have any ideia what it could be?

@brianquinlan
Copy link
Collaborator

brianquinlan commented Feb 20, 2024 •
edited
Loading

I've done some investigation but didn't find anything obvious.

Could you tell me more about how your code is structured? Do you create many CronetClients? Do you close them? Do they share one CronetEngine? Do you make large numbers of concurrent requests (e.g. to download images)? etc?

@brianquinlan
Copy link
Collaborator

Here is the failing code: https://chromium.googlesource.com/chromium/src/+/0f71a543910198ee5b4104ea0066494a3fcba309/net/android/java/src/org/chromium/net/ProxyChangeListener.java#231

@atrope
Copy link
Author

atrope commented Feb 20, 2024

Hey @brianquinlan,
We've been utilizing a single CronetEngine for each download, operating within separate isolates for every download, without closing any of them previously. I'll address this oversight. Although our system supports a high volume of downloads, we maintain a concurrency limit of three, ensuring that a new CronetEngine is initiated only at the start of a download. Consequently, at any given moment, there should be no more than three CronetEngines active within the app.

Do you believe that simply terminating the older engines could resolve the issue? I'm struggling to understand how this adjustment could impact the specific concern you raised.

@brianquinlan
Copy link
Collaborator

Hey @atrope

Do you believe that simply terminating the older engines could resolve the issue? I'm struggling to understand how this adjustment could impact the specific concern you raised.

I'm not sure. The stack trace that you sent showed that the exception occurred while Cronet was registering a BroadCastReceiver for the Proxy.PROXY_CHANGE_ACTION.

My hypothesis is that the unregister code isn't being triggered, which eventually causes over a 1000 BroadcastReceivers to be registered. This may be happening because the CronetEngine is not being closed.

I'm going to look at the cronet_http code and see if I can find any places were it is not free resources when it could.

@brianquinlan
Copy link
Collaborator

FYI, I have a feature in the works that will make it easier to close CronetEngine if it is associated with one CronetClient:
#1137

@atrope
Copy link
Author

atrope commented Feb 21, 2024

Nice, we are now closing so when this is released we will test

@atrope
Copy link
Author

atrope commented Feb 22, 2024 •
edited
Loading

@brianquinlan
Here a minimal to reproduce.
https://github.com/SuaMusica/cronet_http_test

Just open the app and click the "Download all" button on the bottom.

it will eventually crash with said errors. if it does not crash, just change the for from 1000 to 1200.
Here it is crashing when it gets to 200~500 downloads.

PS: Just tried your branch and it also crashed

  cronet_http:
    git:
      url: https://github.com/dart-lang/http
      ref: brianquinlan-release_engine
      path: pkgs/cronet_http



E/AndroidRuntime(12507): FATAL EXCEPTION: CronetInit
E/AndroidRuntime(12507): Process: com.example.cronet_test, PID: 12507
E/AndroidRuntime(12507): m.ea: Native stack trace:
E/AndroidRuntime(12507):
E/AndroidRuntime(12507): 	at org.chromium.base.JniAndroid.handleException(:com.google.android.gms.dynamite_cronetdynamite@240515038@24.05.15 (190400-0):10)
E/AndroidRuntime(12507): 	at J.N.M6Dz0nZ5(Native Method)
E/AndroidRuntime(12507): 	at m.io.run(:com.google.android.gms.dynamite_cronetdynamite@240515038@24.05.15 (190400-0):13)
E/AndroidRuntime(12507): 	at android.os.Handler.handleCallback(Handler.java:958)
E/AndroidRuntime(12507): 	at android.os.Handler.dispatchMessage(Handler.java:99)
E/AndroidRuntime(12507): 	at android.os.Looper.loopOnce(Looper.java:230)
E/AndroidRuntime(12507): 	at android.os.Looper.loop(Looper.java:319)
E/AndroidRuntime(12507): 	at android.os.HandlerThread.run(HandlerThread.java:67)
E/AndroidRuntime(12507): Caused by: java.lang.IllegalStateException: Too many receivers, total of 1000, registered for pid: 12507, callerPackage: com.example.cronet_test
E/AndroidRuntime(12507): 	at android.os.Parcel.createExceptionOrNull(Parcel.java:3077)
E/AndroidRuntime(12507): 	at android.os.Parcel.createException(Parcel.java:3053)
E/AndroidRuntime(12507): 	at android.os.Parcel.readException(Parcel.java:3036)
E/AndroidRuntime(12507): 	at android.os.Parcel.readException(Parcel.java:2978)
E/AndroidRuntime(12507): 	at android.app.IActivityManager$Stub$Proxy.registerReceiverWithFeature(IActivityManager.java:6137)
E/AndroidRuntime(12507): 	at android.app.ContextImpl.registerReceiverInternal(ContextImpl.java:1913)
E/AndroidRuntime(12507): 	at android.app.ContextImpl.registerReceiver(ContextImpl.java:1860)
E/AndroidRuntime(12507): 	at android.content.ContextWrapper.registerReceiver(ContextWrapper.java:791)
E/AndroidRuntime(12507): 	at m.dw.c(:com.google.android.gms.dynamite_cronetdynamite@240515038@24.05.15 (190400-0):7)
E/AndroidRuntime(12507): 	at m.dw.a(:com.google.android.gms.dynamite_cronetdynamite@240515038@24.05.15 (190400-0):2)
E/AndroidRuntime(12507): 	at org.chromium.net.ProxyChangeListener.start(:com.google.android.gms.dynamite_cronetdynamite@240515038@24.05.15 (190400-0):56)
E/AndroidRuntime(12507): 	... 7 more
E/AndroidRuntime(12507): Caused by: android.os.RemoteException: Remote stack trace:
E/AndroidRuntime(12507): 	at com.android.server.am.ActivityManagerService.registerReceiverWithFeature(ActivityManagerService.java:16672)
E/AndroidRuntime(12507): 	at android.app.IActivityManager$Stub.onTransact$registerReceiverWithFeature$(IActivityManager.java:11530)
E/AndroidRuntime(12507): 	at android.app.IActivityManager$Stub.onTransact(IActivityManager.java:2928)
E/AndroidRuntime(12507): 	at com.android.server.am.ActivityManagerService.onTransact(ActivityManagerService.java:3195)
E/AndroidRuntime(12507): 	at android.os.Binder.execTransactInternal(Binder.java:1375)
E/AndroidRuntime(12507):
I/Process (12507): Sending signal. PID: 12507 SIG: 9
Lost connection to device.

Exited.

@brianquinlan
Copy link
Collaborator

brianquinlan commented Feb 22, 2024 •
edited
Loading

@atrope

Did you modify your code to release the engine i.e. https://github.com/SuaMusica/cronet_http_test/pull/1/files

It did not crash for me with the loop running 2000 times.

@atrope
Copy link
Author

atrope commented Feb 22, 2024

Hello @brianquinlan, by setting closeEngine to true, I was able to download 1500 files. I believe this should resolve the issue.

Thank you!

@brianquinlan
Copy link
Collaborator

I'll close this issue, thanks for all the help debugging!

@atrope
Copy link
Author

atrope commented Mar 1, 2024

@brianquinlan Now that we fixed this one I can see another error coming from using cronet in isolates.

They all Happen in the foreground do you happen to know what could be? I am not able to reproduce. Do I Open a new issue for this one?
Here is one stacktrace:

Thanks!

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 8480 >>> suamusica.suamusicaapp <<<

backtrace:
  #00  pc 0x0000000000006fc0  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libdartjni.so (getMethodID+217) (BuildId: 8c8d5a374f130640e74311fc4f467ef4d837e03a)
  #01  pc 0x00000000007e6674  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #02  pc 0x00000000009d60dc  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #03  pc 0x00000000009d5fcc  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #04  pc 0x00000000009d5f6c  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #05  pc 0x0000000000873748  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #06  pc 0x00000000009d4894  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #07  pc 0x0000000001341b10  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #08  pc 0x000000000148aee0  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #09  pc 0x0000000001341a28  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #10  pc 0x0000000001341624  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #11  pc 0x00000000013415c4  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #12  pc 0x0000000000e29fac  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #13  pc 0x000000000148d7d8  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #14  pc 0x00000000007f320c  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #15  pc 0x00000000007f309c  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #16  pc 0x00000000007f11cc  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #17  pc 0x00000000007f1b04  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #18  pc 0x000000000148dd84  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #19  pc 0x00000000007f042c  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #20  pc 0x000000000148d7d8  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #21  pc 0x00000000007f320c  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #22  pc 0x00000000007f309c  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #23  pc 0x00000000007f11cc  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #24  pc 0x00000000007f1b04  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #25  pc 0x000000000083e588  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #26  pc 0x00000000007f0c78  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #27  pc 0x00000000007f0b84  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #28  pc 0x00000000007f0b54  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #29  pc 0x000000000082f37c  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #30  pc 0x000000000082f32c  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #31  pc 0x00000000007e8638  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libapp.so (BuildId: 7bf2d0b48605b0329eaf42acdd2cc262)
  #32  pc 0x00000000005ff7ab  /data/app/~~eE_N_EGiaH8mhBg8Xedp3A==/suamusica.suamusicaapp-c6iLAP0x1GikkpjemcM_TA==/lib/arm/libflutter.so (BuildId: 40d62f1a02b6396f56425b70354081510133280d)

@atrope
Copy link
Author

atrope commented Mar 1, 2024

@flutter-symbolizer-bot #1136 (comment)

@atrope atrope mentioned this issue Mar 6, 2024
Closed
@techouse
Copy link

techouse commented May 9, 2024 •
edited
Loading

I've just noticed this in Crashlytics.

  • cronet_http version: 1.2.0
  • Flutter version: 3.19.6
  • Android version: 12
  • device: Moto G Pure (XT2163DL) (happened only on this device 2x)

I have set closeEngine: true.

Stack trace

Fatal Exception: m.ia: Native stack trace:

       at org.chromium.base.JniAndroid.handleException(:com.google.android.gms.dynamite_cronetdynamite@241518027@24.15.18 (150300-0):10)
       at androidx.compose.foundation.text.selection.TextSelectionColors.M6Dz0nZ5(TextSelectionColors.kt)
       at m.mo.run(:com.google.android.gms.dynamite_cronetdynamite@241518027@24.15.18 (150300-0):13)
       at android.os.Handler.handleCallback(Handler.java:978)
       at android.os.Handler.dispatchMessage(Handler.java:104)
       at android.os.Looper.loopOnce(Looper.java:238)
       at android.os.Looper.loop(Looper.java:357)
       at android.os.HandlerThread.run(HandlerThread.java:85)

Caused by java.lang.IllegalStateException: Too many receivers, total of 1000, registered for pid: 15433, callerPackage: com.example.app
       at android.os.Parcel.createExceptionOrNull(Parcel.java:2434)
       at android.os.Parcel.createException(Parcel.java:2410)
       at android.os.Parcel.readException(Parcel.java:2393)
       at android.os.Parcel.readException(Parcel.java:2335)
       at android.app.IActivityManager$Stub$Proxy.registerReceiverWithFeature(IActivityManager.java:6092)
       at android.app.ContextImpl.registerReceiverInternal(ContextImpl.java:1781)
       at android.app.ContextImpl.registerReceiver(ContextImpl.java:1742)
       at android.content.ContextWrapper.registerReceiver(ContextWrapper.java:745)
       at m.hu.c(:com.google.android.gms.dynamite_cronetdynamite@241518027@24.15.18 (150300-0):7)
       at org.chromium.net.ProxyChangeListener.start(:com.google.android.gms.dynamite_cronetdynamite@241518027@24.15.18 (150300-0):42)
       at androidx.compose.foundation.text.selection.TextSelectionColors.M6Dz0nZ5(TextSelectionColors.kt)
       at m.mo.run(:com.google.android.gms.dynamite_cronetdynamite@241518027@24.15.18 (150300-0):13)
       at android.os.Handler.handleCallback(Handler.java:978)
       at android.os.Handler.dispatchMessage(Handler.java:104)
       at android.os.Looper.loopOnce(Looper.java:238)
       at android.os.Looper.loop(Looper.java:357)
       at android.os.HandlerThread.run(HandlerThread.java:85)

Caused by android.os.RemoteException: Remote stack trace:
	at com.android.server.am.ActivityManagerService.registerReceiverWithFeature(ActivityManagerService.java:12850)
	at android.app.IActivityManager$Stub.onTransact(IActivityManager.java:2485)
	at com.android.server.am.ActivityManagerService.onTransact(ActivityManagerService.java:2603)
	at android.os.Binder.execTransactInternal(Binder.java:1192)
	at android.os.Binder.execTransact(Binder.java:1151)

@brianquinlan
Copy link
Collaborator

Are you making a large number of Clients?

@techouse
Copy link

techouse commented May 9, 2024

Between 4 and 7.

@brianquinlan
Copy link
Collaborator

@techouse Can you produce a minimal reproduction of this issue?

@techouse
Copy link

techouse commented May 9, 2024

I can't as I haven't encountered it myself. I just saw one customer having this issue 2x on that device via Crashlytics.

Maybe it has to do with the fact that I declare the http.Client as a getter inside a Provider. I'll try switching to a late final http.Client instead. 🤷 However, this was never an issue with the IOClient.

@brianquinlan
Copy link
Collaborator

brianquinlan commented May 9, 2024 •
edited
Loading

I can't as I haven't encountered it myself. I just saw one customer having this issue 2x on that device via Crashlytics.

Maybe it has to do with the fact that I declare the http.Client as a getter inside a Provider. I'll try switching to a late final http.Client instead. 🤷

Do you create a new Client in the provider or reuse the same one?

However, this was never an issue with the IOClient.

CronetClient is listening for changes to the android proxy. IOClient doesn't so changing the proxy settings will cause IOClient to enter a bad state.

@techouse
Copy link

techouse commented May 9, 2024

Do you create a new Client in the provider or reuse the same one?

I create it in the Provider, however, I only have like 4 of these Providers alive at any time. I should probably create it statically once and then pass it around, right?

@brianquinlan
Copy link
Collaborator

I create it in the Provider, however, I only have like 4 of these Providers alive at any time. I should probably create it statically once and then pass it around, right?

Yes, that would probably be best.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@brianquinlan brianquinlan

Labels
package:cronet_http type-bug Incorrect behavior (everything from a crash to more subtle misbehavior)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@techouse @atrope @brianquinlan