Increased crash rate in iOS 26

[jkmathew]

Description

Hi everyone,

We are seeing an increase in crashes in our iOS app, which seems to have increased after the release of iOS 26.x. We believe the crash is originating from the Abseil library, but the crash reports are confusing.

Here is what we are seeing:

• Symbolicated Crash Report: The symbolicated stack trace points to a function named ilmIsNull. However, we do not have any symbol with this name anywhere in our app.

EXC_BAD_ACCESS: Exception 1, Code 1, Subcode 4611686018427435728 >
KERN_INVALID_ADDRESS at 0x400000000000bad0.
  libobjc.A           0x191c41b3c  objc_release
  MyApp               0x1066f8a04  ilmIsNull
  MyApp               0x1066d3620  ilmIsNull
  MyApp               0x1066d3130  ilmIsNull
  MyApp               0x1066d25f0  ilmIsNull
  MyApp               0x106729988  ilmIsNull
  Foundation          0x19231d250  __NSINDEXSET_IS_CALLING_OUT_TO_A_BOOL_BLOCK__
  Foundation          0x19231ce40  -[NSBlockOperation main]
  Foundation          0x19231cf70  __NSOPERATION_IS_INVOKING_MAIN__
  Foundation          0x19231ca10  -[NSOperation start]
  Foundation          0x19231cdd8  __NSOPERATIONQUEUE_IS_STARTING_AN_OPERATION__
  Foundation          0x19231c694  __NSOQSchedule_f
  libdispatch         0x1cc61644c  _dispatch_block_async_invoke2
  libdispatch         0x1cc6207e8  _dispatch_client_callout
  libdispatch         0x1cc60b660  _dispatch_continuation_pop
  libdispatch         0x1cc60acd4  _dispatch_async_redirect_invoke
  libdispatch         0x1cc618f44  _dispatch_root_queue_drain
  libdispatch         0x1cc6196f8  _dispatch_worker_thread2
  libsystem_pthread   0x1efa7f378  _pthread_wqthread

• Unsymbolicated Crash Report: When we look at the unsymbolicated trace, we see this function: _ZN66GMSx_AbslFlagDefaultGenForprotobuf_enable_debug_string_safe_format3GenEPv

EXC_BAD_ACCESS: Exception 1, Code 1, Subcode 4611686018427435728 >
KERN_INVALID_ADDRESS at 0x400000000000bad0.
  libobjc.A           0x191c41b3c  objc_release_x0
  MyApp               0x1066f8a08  _ZN66GMSx_AbslFlagDefaultGenForprotobuf_enable_debug_string_safe_format3GenEPv
  MyApp               0x1066d3624  _ZN66GMSx_AbslFlagDefaultGenForprotobuf_enable_debug_string_safe_format3GenEPv
  MyApp               0x1066d3134  _ZN66GMSx_AbslFlagDefaultGenForprotobuf_enable_debug_string_safe_format3GenEPv
  MyApp               0x1066d25f4  _ZN66GMSx_AbslFlagDefaultGenForprotobuf_enable_debug_string_safe_format3GenEPv
  MyApp               0x10672998c  _ZN66GMSx_AbslFlagDefaultGenForprotobuf_enable_debug_string_safe_format3GenEPv
  Foundation          0x19231d254  <redacted>
  Foundation          0x19231ce44  <redacted>
  Foundation          0x19231cf74  <redacted>
  Foundation          0x19231ca14  <redacted>
  Foundation          0x19231cddc  <redacted>
  Foundation          0x19231c698  <redacted>
  libdispatch         0x1cc616450  <redacted>
  libdispatch         0x1cc6207ec  <redacted>
  libdispatch         0x1cc60b664  <redacted>
  libdispatch         0x1cc60acd8  <redacted>
  libdispatch         0x1cc618f48  <redacted>
  libdispatch         0x1cc6196fc  <redacted>
  libsystem_pthread   0x1efa7f37c  _pthread_wqthread

We strongly suspect the crash is in Abseil because the _ZN66GMSx_AbslFlagDefaultGen prefix suggests it's part of the Abseil library, which is a dependency of Firebase SDK, and we were able to find the source code for this exact function in the public Abseil repository.

Our main questions are:

What could be the reason for this crash?

Why is the crash report being incorrectly symbolicated as ilmIsNull?

What is the best way to solve this?

Any help or insights would be appreciated. Thanks!

Reproducing the issue

This is hard to reproduce, happening randomly in production.

Firebase SDK Version

11.2.0

Xcode Version

16.1,26.0.1

Installation Method

Swift Package Manager

Firebase Product(s)

Performance, Remote Config, DynamicLinks

Targeted Platforms

iOS

Relevant Log Output

If using Swift Package Manager, the project's Package.resolved

Expand Package.resolved snippet

{
  "originHash" : "9a06f7c038e1ac752815335b888ab332e5b9d0375981275b84b647d822ca0066",
  "pins" : [
    {
      "identity" : "abseil-cpp-binary",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/google/abseil-cpp-binary.git",
      "state" : {
        "revision" : "194a6706acbd25e4ef639bcaddea16e8758a3e27",
        "version" : "1.2024011602.0"
      }
    },
    {
      "identity" : "app-check",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/google/app-check.git",
      "state" : {
        "revision" : "61b85103a1aeed8218f17c794687781505fbbef5",
        "version" : "11.2.0"
      }
    },
    {
      "identity" : "firebase-ios-sdk",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/firebase/firebase-ios-sdk.git",
      "state" : {
        "revision" : "1fc52ab0e172e7c5a961f975a76c2611f4f22852",
        "version" : "11.2.0"
      }
    },
    {
      "identity" : "google-tag-manager-ios-sdk",
      "kind" : "remoteSourceControl",
      "location" : "git@github.com:googleanalytics/google-tag-manager-ios-sdk.git",
      "state" : {
        "revision" : "484de314452681ad059200bcda01253ee930b0bb",
        "version" : "8.0.0"
      }
    },
    {
      "identity" : "googleappmeasurement",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/google/GoogleAppMeasurement.git",
      "state" : {
        "revision" : "07a2f57d147d2bf368a0d2dcb5579ff082d9e44f",
        "version" : "11.1.0"
      }
    },
    {
      "identity" : "googledatatransport",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/google/GoogleDataTransport.git",
      "state" : {
        "revision" : "617af071af9aa1d6a091d59a202910ac482128f9",
        "version" : "10.1.0"
      }
    },
    {
      "identity" : "googlesignin-ios",
      "kind" : "remoteSourceControl",
      "location" : "git@github.com:google/GoogleSignIn-iOS.git",
      "state" : {
        "revision" : "65fb3f1aa6ffbfdc79c4e22178a55cd91561f5e9",
        "version" : "8.0.0"
      }
    },
    {
      "identity" : "googleutilities",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/google/GoogleUtilities.git",
      "state" : {
        "revision" : "53156c7ec267db846e6b64c9f4c4e31ba4cf75eb",
        "version" : "8.0.2"
      }
    },
    {
      "identity" : "grpc-binary",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/google/grpc-binary.git",
      "state" : {
        "revision" : "f56d8fc3162de9a498377c7b6cea43431f4f5083",
        "version" : "1.65.1"
      }
    },
    {
      "identity" : "gtm-session-fetcher",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/google/gtm-session-fetcher.git",
      "state" : {
        "revision" : "a2ab612cb980066ee56d90d60d8462992c07f24b",
        "version" : "3.5.0"
      }
    },
    {
      "identity" : "gtmappauth",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/google/GTMAppAuth.git",
      "state" : {
        "revision" : "5d7d66f647400952b1758b230e019b07c0b4b22a",
        "version" : "4.1.1"
      }
    },
    {
      "identity" : "interop-ios-for-google-sdks",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/google/interop-ios-for-google-sdks.git",
      "state" : {
        "revision" : "2d12673670417654f08f5f90fdd62926dc3a2648",
        "version" : "100.0.0"
      }
    },
    {
      "identity" : "ios-maps-sdk",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/googlemaps/ios-maps-sdk.git",
      "state" : {
        "revision" : "3786e5028d868482ef88140117d33efd60828300",
        "version" : "9.1.1"
      }
    },
    {
      "identity" : "leveldb",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/firebase/leveldb.git",
      "state" : {
        "revision" : "a0bc79961d7be727d258d33d5a6b2f1023270ba1",
        "version" : "1.22.5"
      }
    },
    {
      "identity" : "nanopb",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/firebase/nanopb.git",
      "state" : {
        "revision" : "b7e1104502eca3a213b46303391ca4d3bc8ddec1",
        "version" : "2.30910.0"
      }
    },
    {
      "identity" : "promises",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/google/promises.git",
      "state" : {
        "revision" : "540318ecedd63d883069ae7f1ed811a2df00b6ac",
        "version" : "2.4.0"
      }
    },
    {
      "identity" : "swift-protobuf",
      "kind" : "remoteSourceControl",
      "location" : "https://github.com/apple/swift-protobuf.git",
      "state" : {
        "revision" : "d72aed98f8253ec1aa9ea1141e28150f408cf17f",
        "version" : "1.29.0"
      }
    }
    ...
    ...
    ...
  ],
  "version" : 3
}

If using CocoaPods, the project's Podfile.lock

Expand Podfile.lock snippet

Replace this line with the contents of your Podfile.lock!

[google-oss-bot]

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.

[backslash-f]

We also notice a spike in crashes using Firebase 12.5.0. Crash reports from Xcode/Organizer point to firebase::firestore::util::ObjcThrowHandler(firebase::firestore::util::ExceptionType, char const*, char const*, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<ch... + 284.

A stack trace sample (they all look the same, with no real useful info (to me at least)):

Exception Type:  EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Triggered by Thread:  20

Last Exception Backtrace:
0   CoreFoundation                	0x198c0c7cc __exceptionPreprocess + 164 (NSException.m:249)
1   libobjc.A.dylib               	0x195edf2e4 objc_exception_throw + 88 (objc-exception.mm:356)
2   Foundation                    	0x197fdca78 -[NSAssertionHandler handleFailureInFunction:file:lineNumber:description:] + 272 (NSException.m:261)
3                       	0x101267158 firebase::firestore::util::ObjcThrowHandler(firebase::firestore::util::ExceptionType, char const*, char const*, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<ch... + 284
4                       	0x101266b68 firebase::firestore::util::Throw(firebase::firestore::util::ExceptionType, char const*, char const*, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) + 20
5                       	0x1012b2c18 firebase::firestore::util::internal::FailAssertion(char const*, char const*, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) + 32
6                       	0x1012b2cb8 firebase::firestore::util::internal::FailAssertion(char const*, char const*, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, char const*) + 160
7                       	0x1012e6c80 firebase::firestore::local::LevelDbTransaction::Commit() + 656
8                       	0x1012ddae8 firebase::firestore::local::LevelDbPersistence::RunInternal(absl::lts_20240722::string_view, std::__1::function<void ()>) + 200
9                       	0x1012f34f0 firebase::firestore::local::LocalStore::CollectGarbage(firebase::firestore::local::LruGarbageCollector*) + 108
10                      	0x10128797c std::__1::__function::__func<firebase::firestore::core::FirestoreClient::ScheduleLruGarbageCollection()::$_0, std::__1::allocator<firebase::firestore::core::FirestoreClient::ScheduleLruGarbageColle... + 48
11                      	0x10122b624 firebase::firestore::util::AsyncQueue::ExecuteBlocking(std::__1::function<void ()> const&) + 68
12                      	0x10134be70 firebase::firestore::util::Task::ExecuteAndRelease() + 188
13  libdispatch.dylib             	0x1a08e20d0 _dispatch_client_callout + 20 (object.m:576)
14  libdispatch.dylib             	0x1a08e5580 _dispatch_continuation_pop + 596 (queue.c:325)
15  libdispatch.dylib             	0x1a08f953c _dispatch_source_latch_and_call + 420 (source.c:596)
16  libdispatch.dylib             	0x1a08f8104 _dispatch_source_invoke + 836 (source.c:961)
17  libdispatch.dylib             	0x1a08e9560 _dispatch_lane_serial_drain + 368 (queue.c:3934)
18  libdispatch.dylib             	0x1a08ea1e0 _dispatch_lane_invoke + 380 (queue.c:4025)
19  libdispatch.dylib             	0x1a08f5258 _dispatch_root_queue_drain_deferred_wlh + 288 (queue.c:7193)
20  libdispatch.dylib             	0x1a08f4aa4 _dispatch_workloop_worker_thread + 540 (queue.c:6787)
21  libsystem_pthread.dylib       	0x2210efc7c _pthread_wqthread + 288 (pthread.c:2696)
22  libsystem_pthread.dylib       	0x2210ec488 start_wqthread + 8 (:-1)

[cherylEnkidu]

Hi @jkmathew, thank you for reporting this. I understand this error is really disruptive. Based on the error log, the crash appears to be caused by database corruption, likely due to the LevelDB database on the user’s device becoming corrupted during a garbage collection process.
Since we can’t currently reproduce the issue, I’d like to gather a bit more information to narrow down the investigation. In your observations, how does SDK version 11.2.0 behave on iOS 26.x? Specifically, does 11.2.0 show significantly better stability or performance compared to 12.5.0 on iOS 26.x?

[jkmathew]

@cherylEnkidu
We are still using 11.2.0. Did not update to 12.x yet. We are trying to do that. I will share more details later.

[cherylEnkidu]

Hi @jkmathew ,

The debug logs commented here don’t appear to be related to the issue described. For both issues, could you provide additional logs if you find more information?
Also, we couldn’t find any function named ilmIsNull in our SDK or its dependencies. Could you double-check whether this function is actually coming from the Firestore SDK?

[jkmathew]

Hey @cherylEnkidu
That is the problem with this stacktrace. Even if the symbolicated crash log shows ilmIsNull, when I check unsymbolicated logs, it shows _ZN66GMSx_AbslFlagDefaultGenForprotobuf_enable_debug_string_safe_format3GenEPv. I think this is coming from Abseil. We dont have any dependency on Abseil directly or transitively other than firebase/Google sdks. That is the reason I suspect this is from some firebase/Google sdks.