Android: java.lang.ArrayIndexOutOfBoundsException thrown by AblyInstanceStore's setPaginatedResult method

[QuintinWillison]

We've had a customer report that a small subset of their app users have experienced this crash.

Information provided via Firebase Crashlytics...

{code}
Fatal Exception: java.lang.ArrayIndexOutOfBoundsException
src.length=11 srcPos=0 dst.length=25 dstPos=0 length=12
java.lang.System.arraycopy (System.java)
com.android.internal.util.GrowingArrayUtils.insert (GrowingArrayUtils.java:141)
android.util.LongSparseArray.put (LongSparseArray.java:218)
io.ably.flutter.plugin.AblyInstanceStore.setPaginatedResult (AblyInstanceStore.java:105)
io.ably.flutter.plugin.AblyMethodCallHandler$3.onSuccess (AblyMethodCallHandler.java:280)
io.ably.flutter.plugin.AblyMethodCallHandler$3.onSuccess (AblyMethodCallHandler.java:277)
io.ably.lib.http.BasePaginatedQuery$ResultRequest$1.onSuccess (BasePaginatedQuery.java:215)
io.ably.lib.http.BasePaginatedQuery$ResultRequest$1.onSuccess (BasePaginatedQuery.java:212)
io.ably.lib.http.HttpScheduler$AsyncRequest.setResult (HttpScheduler.java:321)
io.ably.lib.http.HttpScheduler$AblyRequestWithFallback.run (HttpScheduler.java:206)
java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1167)
java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:641)
java.lang.Thread.run (Thread.java:919)
{code}
From the crash log for a Nokia 5.4 device running Android 10.

Other potentially salient details:

• 6 instances of this crash seen across 14,600 installations of the app
• OS: 50% running Android 9
• Devices: 33% Samsung, 33% Xiaomi, 17% Vivo, 17% HMD Global
• Device states: 50% background

Also see [this internal Ably Support Team Slack thread|https://ably-real-time.slack.com/archives/C8SPU4589/p1643257377074100].

[QuintinWillison]

Very similar profile of issue encountered here: airbnb/lottie-android#452

My working theory is that we're manipulating the LongSparseArray instance from multiple threads, when it's not implemented to be safe for concurrent use.

Our AblyInstanceStore class needs to be improved. The solution might be 'as simple as' adding synchronized semantics to instance methods, however I would like to see analysis of where this is being used across the codebase and clearer definition of the API offered by this internal class including constraints and limitations implied by that (to be explicitly stated).

[QuintinWillison]

I've also now been given access to more information from Crashlytics. Across a two month period, we've seen 443 crash events across 316 users, so a healthy sample. 😁

• 98% of those were in device state 'background'
• There is no specific version of Android OS (5 thru 11 seen)
• There is no specific device make (list includes Google, Sony, Samsung, etc..)