Random Realm crashes

[sync-by-unito]

How frequently does the bug occur?

Sometimes

Description

We just rarely receive crash reports in the Crashlytics

Stacktrace & log output

The app was in the background for 20 minutes and then received some time to run in the background or was moving to the active state and crashed. One log also had Reading failed: Premature end of input and Connection[1]: Connection closed due to error errors but another one didn't. Not sure if important but also realm.refresh() method was called on two different Sync Realms. Overall, looks like network connection, obviously, becomes broken after app spends too much time in the background and then the app may crash when becoming active:

Crashed: Thread
0  libsystem_kernel.dylib         0x7964 __pthread_kill + 8
1  libsystem_pthread.dylib        0x7378 pthread_kill + 268
2  libsystem_c.dylib              0x1ff50 abort + 164
3  Divtracker                     0x293238 std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::operator<<<std::__1::char_traits<char> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char) + 203116
4  Divtracker                     0x2933a8 realm::util::terminate_internal(std::__1::basic_stringstream<char, std::__1::char_traits<char>, std::__1::allocator<char> >&) + 203484
5  Divtracker                     0x293520 realm::util::terminate(char const*, char const*, long, std::initializer_list<realm::util::Printable>&&) + 203860
6  Divtracker                     0x3160d8 realm::util::network::Service::BasicStreamOps<realm::util::network::ssl::Stream>::BufferedReadOperBase::advance() + 739340
7  Divtracker                     0x3159e4 void realm::util::network::Service::BasicStreamOps<realm::util::network::ssl::Stream>::async_buffered_read<std::__1::function<void (std::__1::error_code, unsigned long)> >(realm::util::network::ssl::Stream&, char*, unsigned long, int, realm::util::network::ReadAheadBuffer&, std::__1::function<void (std::__1::error_code, unsigned long)>) + 737560
8  Divtracker                     0x310cb8 realm::util::websocket::(anonymous namespace)::EZSocketImpl::async_read(char*, unsigned long, std::__1::function<void (std::__1::error_code, unsigned long)>) + 717804
9  Divtracker                     0x32200c (anonymous namespace)::WebSocket::frame_reader_loop() + 788288
10 Divtracker                     0x3162dc void realm::util::network::Service::AsyncOper::do_recycle_and_execute<std::__1::function<void (std::__1::error_code, unsigned long)>, std::__1::error_code&, unsigned long&>(bool, std::__1::function<void (std::__1::error_code, unsigned long)>&, std::__1::error_code&, unsigned long&) + 739856
11 Divtracker                     0x315df0 realm::util::network::Service::BasicStreamOps<realm::util::network::ssl::Stream>::BufferedReadOper<std::__1::function<void (std::__1::error_code, unsigned long)> >::recycle_and_execute() + 738596
12 Divtracker                     0x318f8c realm::util::network::Service::Impl::run() + 751296
13 Divtracker                     0x2b3d94 realm::sync::Client::run() + 337096
14 Divtracker                     0x41a8bc void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, realm::_impl::SyncClient::SyncClient(std::__1::unique_ptr<realm::util::Logger, std::__1::default_delete<realm::util::Logger> >, realm::SyncClientConfig const&, std::__1::weak_ptr<realm::SyncManager const>)::'lambda0'()> >(void*) + 548316
15 libsystem_pthread.dylib        0x19a4 _pthread_start + 148
16 libsystem_pthread.dylib        0xea0 thread_start + 8

The app tried to open Realm synchronously and crashed, not sure if important but there was also remote function call in parallel:

Crashed: RealmManager.realm
0  libsystem_kernel.dylib         0x7b78 __pthread_kill + 8
1  libsystem_pthread.dylib        0x73bc pthread_kill + 268
2  libsystem_c.dylib              0x2051c abort + 168
3  Divtracker                     0x293238 std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::operator<<<std::__1::char_traits<char> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char) + 203116
4  Divtracker                     0x2933a8 realm::util::terminate_internal(std::__1::basic_stringstream<char, std::__1::char_traits<char>, std::__1::allocator<char> >&) + 203484
5  Divtracker                     0x293520 realm::util::terminate(char const*, char const*, long, std::initializer_list<realm::util::Printable>&&) + 203860
6  Divtracker                     0x149078 realm::Cluster::init(realm::MemRef) + 48532
7  Divtracker                     0x157794 realm::ClusterTree::create_root_from_parent(realm::ArrayParent*, unsigned long) + 107696
8  Divtracker                     0x157b5c realm::ClusterTree::init_from_parent() + 108664
9  Divtracker                     0x2567e8 realm::Table::init(unsigned long, realm::ArrayParent*, unsigned long, bool, bool) + 469164
10 Divtracker                     0x17c12c realm::Group::create_table_accessor(unsigned long) + 257608
11 Divtracker                     0x17be14 realm::Group::do_get_table(unsigned long) + 256816
12 Divtracker                     0x36a3f8 realm::ObjectSchema::ObjectSchema(realm::Group const&, realm::StringData, realm::TableKey) + 22420
13 Divtracker                     0x372fa0 realm::ObjectStore::schema_from_group(realm::Group const&) + 58172
14 Divtracker                     0x3bfae4 realm::Realm::read_schema_from_group_if_needed() + 176132
15 Divtracker                     0x3bf7f0 realm::Realm::Realm(realm::Realm::Config, realm::util::Optional<realm::VersionID>, std::__1::shared_ptr<realm::_impl::RealmCoordinator>, realm::Realm::MakeSharedTag) + 175376
16 Divtracker                     0x3405c8 realm::Realm::make_shared_realm(realm::Realm::Config, realm::util::Optional<realm::VersionID>, std::__1::shared_ptr<realm::_impl::RealmCoordinator>) + 912636
17 Divtracker                     0x33faec realm::_impl::RealmCoordinator::do_get_realm(realm::Realm::Config, std::__1::shared_ptr<realm::Realm>&, realm::util::Optional<realm::VersionID>, realm::util::CheckedUniqueLock&) + 909856
18 Divtracker                     0x33f7a4 realm::_impl::RealmCoordinator::get_realm(realm::Realm::Config, realm::util::Optional<realm::VersionID>) + 909016
19 Divtracker                     0x3c05c4 realm::Realm::get_shared_realm(realm::Realm::Config) + 178916
20 Divtracker                     0xa2a4bc +[RLMRealm realmWithConfiguration:queue:error:] + 869 (shared_ptr.h:869)
21 Divtracker                     0xab76ec Realm.init(configuration:queue:) + 1580652 (<compiler-generated>:1580652)
22 Divtracker                     0xce75c closure #1 in static RealmManager.syncOpenRealm(configuration:) + 2019 (RealmManager.swift:2019)
23 Divtracker                     0xe33b0 partial apply for closure #1 in static RealmManager.syncOpenRealm(configuration:) + 4309595056 (<compiler-generated>:4309595056)
24 Divtracker                     0xb972dc MapSink.on(_:) + 43 (Map.swift:43)
25 Divtracker                     0xb974a0 protocol witness for ObserverType.on(_:) in conformance MapSink<A, B> + 2497568 (<compiler-generated>:2497568)
26 Divtracker                     0xba48f8 ObserveOnSink.run(_:_:) + 133 (ObserveOn.swift:133)
27 Divtracker                     0xba5b10 partial apply for thunk for @escaping @callee_guaranteed (@unowned @callee_guaranteed () -> ()) -> () + 2556560 (<compiler-generated>:2556560)
28 Divtracker                     0xbb312c partial apply for thunk for @escaping @callee_guaranteed (@in_guaranteed A, @unowned @callee_guaranteed @substituted <A> (@in_guaranteed A) -> () for <A>) -> () + 2611372 (<compiler-generated>:2611372)
29 Divtracker                     0xbb2870 closure #1 in RecursiveImmediateScheduler.schedule(_:) + 2609136 (<compiler-generated>:2609136)
30 Divtracker                     0xc12718 partial apply for closure #1 in DispatchQueueConfiguration.schedule<A>(_:action:) + 90 (DispatchQueueScheduler.swift:90)
31 Divtracker                     0xa01a8 thunk for @escaping @callee_guaranteed () -> () + 4309320104 (<compiler-generated>:4309320104)
32 libdispatch.dylib              0x1e68 _dispatch_call_block_and_release + 32
33 libdispatch.dylib              0x3a2c _dispatch_client_callout + 20
34 libdispatch.dylib              0xb124 _dispatch_lane_serial_drain + 668
35 libdispatch.dylib              0xbcb4 _dispatch_lane_invoke + 444
36 libdispatch.dylib              0x16500 _dispatch_workloop_worker_thread + 648
37 libsystem_pthread.dylib        0x10bc _pthread_wqthread + 288
38 libsystem_pthread.dylib        0xe5c start_wqthread + 8

Can you reproduce the bug?

Not yet

Reproduction Steps

Unknown but for the first crash log it looks related to network connection timeout after leaving from the background

Version

10.25.0 and previous since I have continuous crash reports for 90 days

What SDK flavour are you using?

MongoDB Realm (i.e. Sync, auth, functions)

Are you using encryption?

No, not using encryption

Platform OS and version(s)

iPadOS 15.4.0, iPadOS 15.3.1, iOS 15.4.1, ...

Build environment

ProductName:	macOS
ProductVersion:	12.3.1
BuildVersion:	21E258

/Applications/Xcode.app/Contents/Developer
Xcode 13.3.1
Build version 13E500a

/usr/local/bin/pod
1.11.3
Realm (10.25.0)
RealmSwift (10.25.0)

/bin/bash
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin21)

/usr/local/bin/carthage
0.38.0
(not in use here)

/usr/bin/git
git version 2.32.0 (Apple Git-132)

[ironage]

@anton-plebanovich thanks for reporting this! It looks like an assertion failure, is there an assertion message captured from stderr on these devices that you can share? Getting a reproduction case for this would be ideal, but I understand that may be difficult to obtain.

Guessing about the network thread sent to the background sounds promising. Maybe it is an edge case that we have not handled correctly; possibly ETIMEDOUT from ::recv().

[sync-by-unito]

➤ James Stone commented:

[~daniel.tabacaru@mongodb.com] do you have capacity to investigate this?

This issue might just go away when we remove the custom websocket implementation as well.

[anton-plebanovich]

@ironage Sadly, there is only crash_info_entry_0: abort() called entry attached to the crash report. I doubt I can have easy to reproduce steps because we have a lot of users and they for sure background the app often but amount of crashes is few.

[sync-by-unito]

➤ Daniel Tabacaru commented:

[~james.stone@mongodb.com] I don't have capacity at the moment. I could investigate it at a later point if it's not too urgent.

[sync-by-unito]

➤ Ian Ward commented:

Hi. Going through old support tickets this quarter - has there been any more progress on this ticket?

[sync-by-unito]

➤ michael-wb commented:

We had a customer ticket come in HELP-38176 that was closed since the customer never got back to us when requesting debug info to confirm that this is a related task. I had started looking into this - it seems to be a failure that occurs when SSL encryption is enabled, but it is difficult to reproduce.

I have been primarily focused on getting Platform Networking across the finish line and will get back to looking at this TR afterwards.