Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iOS crash due to default.realm.lock': open() failed: Operation not permitted Path #7874

Closed
haozhutw opened this issue Jul 6, 2022 · 17 comments
Assignees

Comments

@haozhutw
Copy link

haozhutw commented Jul 6, 2022

How frequently does the bug occur?

Sometimes

Description

Expected
The crash shouldn't happen.

Actual result
The crash happens during app startup process. We did a database migration in function application:didFinishLaunchingWithOptions in AppDelegate.m

Stacktrace & log output

Fatal Exception: RLMException
/var/mobile/Containers/Data/Application/8025F2F4-E299-4025-AE69-A441B9D27B48/Documents/RealmDB/default.realm.lock: Unable to open a realm at path '/var/mobile/Containers/Data/Application/8025F2F4-E299-4025-AE69-A441B9D27B48/Documents/RealmDB/default.realm.lock': open() failed: Operation not permitted Path: /var/mobile/Containers/Data/Application/8025F2F4-E299-4025-AE69-A441B9D27B48/Documents/RealmDB/default.realm.lock Exception backtrace: 0 MyAppName 0x0000000104ad8734 _ZNK5realm5Table10find_firstINS_4util8OptionalINS_8ObjectIdEEEEENS_6ObjKeyENS_6ColKeyET_ + 106252 1 MyAppName 0x00000001049b55b0 _ZNK5realm19ArrayFixedBytesNullINS_4UUIDELi16EE15find_first_nullEmm + 124732 2 MyAppName 0x00000001049b864c _ZNK5realm19ArrayFixedBytesNullINS_4UUIDELi16EE15find_first_nullEmm + 137176 3 MyAppName 0x00000001049bc8e4 _ZNK5realm19ArrayFixedBytesNullINS_4UUIDELi16EE15find_first_nullEmm + 154224 4 MyAppName 0x0000000104b9d41c _ZNK5realm5Table10find_firstINS_4util8OptionalINS_8ObjectIdEEEEENS_6ObjKeyENS_6ColKeyET_ + 912372 5 MyAppName 0x0000000104b9c548 _ZNK5realm5Table10find_firstINS_4util8OptionalINS_8ObjectIdEEEEENS_6ObjKeyENS_6ColKeyET_ + 908576 6 MyAppName 0x0000000104b9c3a0 _ZNK5realm5Table10find_firstINS_4util8OptionalINS_8ObjectIdEEEEENS_6ObjKeyENS_6ColKeyET_ + 908152 7 MyAppName 0x0000000104beaf40 _ZN5realm7Results8index_ofINS_4util8OptionalINS_4UUIDEEEEEmRKT_ + 73892 8 MyAppName 0x00000001048aadf4 _Z23RLMAddNotificationBlockI10RLMResultsEP20RLMNotificationTokenPT_U13block_pointerFvP11objc_objectP19RLMCollectionChangeP7NSErrorEPU28objcproto17OS_dispatch_queue8NSObject + 665224 9 MyAppName 0x00000001048aaa2c _Z23RLMAddNotificationBlockI10RLMResultsEP20RLMNotificationTokenPT_U13block_pointerFvP11objc_objectP19RLMCollectionChangeP7NSErrorEPU28ob


Fatal Exception: RLMException
0  CoreFoundation                 0x12586c __exceptionPreprocess
1  libobjc.A.dylib                0x6c50 objc_exception_throw
2  MyAppName                      0x39f9f0 RLMSetErrorOrThrow(NSError*, NSError* __autoreleasing*) + 380 (RLMUtil.mm:380)
3  MyAppName                      0x382404 RLMRealmTranslateException(NSError* __autoreleasing*) + 327 (RLMRealm.mm:327)
4  MyAppName                      0x3838ec translateSharedGroupOpenException(NSError* __autoreleasing*) + 357 (RLMRealm.mm:357)
5  MyAppName                      0x383830 +[RLMRealm realmWithConfiguration:queue:error:] + 445 (RLMRealm.mm:445)
6  MyAppName                      0x382a2c +[RLMRealm realmWithConfiguration:error:] + 362 (RLMRealm.mm:362)
7  MyAppName                      0x381974 +[RLMRealm defaultRealm] + 197 (RLMRealm.mm:197)
8  MyAppName                      0x843c -[AppDelegate migrateRealmDB] + 151 (AppDelegate.m:151)
9  MyAppName                      0x7d54 -[AppDelegate application:didFinishLaunchingWithOptions:] + 28 (AppDelegate.m:28)
10 UIKitCore                      0xbc5228 -[UIApplication _handleDelegateCallbacksWithOptions:isSuspended:restoreState:]
11 UIKitCore                      0xbc7290 -[UIApplication _callInitializationDelegatesWithActions:forCanvas:payload:fromOriginatingProcess:]
12 UIKitCore                      0xbcccec -[UIApplication _runWithMainScene:transitionContext:completion:]
13 UIKitCore                      0x222c74 -[_UISceneLifecycleMultiplexer completeApplicationLaunchWithFBSScene:transitionContext:]
14 UIKitCore                      0x790f9c _UIScenePerformActionsWithLifecycleActionMask
15 UIKitCore                      0x22380c __101-[_UISceneLifecycleMultiplexer _evalTransitionToSettings:fromSettings:forceExit:withTransitionStore:]_block_invoke
16 UIKitCore                      0x2232cc -[_UISceneLifecycleMultiplexer _performBlock:withApplicationOfDeactivationReasons:fromReasons:]
17 UIKitCore                      0x22361c -[_UISceneLifecycleMultiplexer _evalTransitionToSettings:fromSettings:forceExit:withTransitionStore:]
18 UIKitCore                      0x222e58 -[_UISceneLifecycleMultiplexer uiScene:transitionedFromState:withTransitionContext:]
19 UIKitCore                      0x22b3a4 __186-[_UIWindowSceneFBSSceneTransitionContextDrivenLifecycleSettingsDiffAction _performActionsForUIScene:withUpdatedFBSScene:settingsDiff:fromSettings:transitionContext:lifecycleActionType:]_block_invoke
20 UIKitCore                      0x69d60c +[BSAnimationSettings(UIKit) tryAnimatingWithSettings:actions:completion:]
21 UIKitCore                      0x7a96c4 _UISceneSettingsDiffActionPerformChangesWithTransitionContext
22 UIKitCore                      0x22b09c -[_UIWindowSceneFBSSceneTransitionContextDrivenLifecycleSettingsDiffAction _performActionsForUIScene:withUpdatedFBSScene:settingsDiff:fromSettings:transitionContext:lifecycleActionType:]
23 UIKitCore                      0x525a0 __64-[UIScene scene:didUpdateWithDiff:transitionContext:completion:]_block_invoke
24 UIKitCore                      0x50f14 -[UIScene _emitSceneSettingsUpdateResponseForCompletion:afterSceneUpdateWork:]
25 UIKitCore                      0x521c8 -[UIScene scene:didUpdateWithDiff:transitionContext:completion:]
26 UIKitCore                      0xbcae8c -[UIApplication workspace:didCreateScene:withTransitionContext:completion:]
27 UIKitCore                      0x6c6e38 -[UIApplicationSceneClientAgent scene:didInitializeWithEvent:completion:]
28 FrontBoardServices             0xa3bc -[FBSScene _callOutQueue_agent_didCreateWithTransitionContext:completion:]
29 FrontBoardServices             0x35d04 __94-[FBSWorkspaceScenesClient createWithSceneID:groupID:parameters:transitionContext:completion:]_block_invoke.200
30 FrontBoardServices             0x194a0 -[FBSWorkspace _calloutQueue_executeCalloutFromSource:withBlock:]
31 FrontBoardServices             0x359c8 __94-[FBSWorkspaceScenesClient createWithSceneID:groupID:parameters:transitionContext:completion:]_block_invoke
32 libdispatch.dylib              0x3db0 _dispatch_client_callout
33 libdispatch.dylib              0x7738 _dispatch_block_invoke_direct
34 FrontBoardServices             0x5e250 __FBSSERIALQUEUE_IS_CALLING_OUT_TO_A_BLOCK__
35 FrontBoardServices             0x5dee0 -[FBSSerialQueue _targetQueue_performNextIfPossible]
36 FrontBoardServices             0x5e434 -[FBSSerialQueue _performNextFromRunLoopSource]
37 CoreFoundation                 0xa176c __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__
38 CoreFoundation                 0xa1668 __CFRunLoopDoSource0
39 CoreFoundation                 0xa0960 __CFRunLoopDoSources0
40 CoreFoundation                 0x9aa8c __CFRunLoopRun
41 CoreFoundation                 0x9a21c CFRunLoopRunSpecific
42 GraphicsServices               0x3784 GSEventRunModal
43 UIKitCore                      0xbc8ee8 -[UIApplication _run]
44 UIKitCore                      0xbce75c UIApplicationMain
45 MyAppName                      0x8c24 main + 14 (main.m:14)
46 libdyld.dylib                  0x16b0 start

Can you reproduce the bug?

Not yet

Reproduction Steps

This crash is found on the Firebase console, I can't reproduce it. According to the Firebase, it happens more than 600 times and affects more than 300 users. I tried solutions in other issues below:

#3308 (comment)
#5938 (comment)
#1260 (comment)
#1874 (comment)
#1571 (comment)

And I even looked up the docs https://docs.mongodb.com/realm-legacy/docs/swift/latest/index.html#using-realm-with-background-app-refresh, but after updating my code I still see this crash happens on the production environment.

Here are my code about migration

- (void)migrateRealmDB {
  NSString *kRealmFileName = @"default.realm";
  NSString *kRealmLockFileName = @"default.realm.lock";
  NSString *kRealmManagementFolder = @"default.realm.management";
  
  NSString *documentFolder = [NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) firstObject];
  NSString *dbfolder = [documentFolder stringByAppendingPathComponent:@"RealmDB"];
  NSString *newRealmPath = [dbfolder stringByAppendingPathComponent:kRealmFileName];
  BOOL isDirectory = YES;
  BOOL folderExists = [[NSFileManager defaultManager] fileExistsAtPath:dbfolder isDirectory:&isDirectory];
  if (!folderExists) {
    BOOL res = [[NSFileManager defaultManager] createDirectoryAtPath:dbfolder
                              withIntermediateDirectories:YES
                                               attributes:@{ NSFileProtectionKey: NSFileProtectionNone }
                                                    error:nil];
    if (res) {
      // create folder successfully
      folderExists = YES;
      NSString *defaultRealmPath = [documentFolder stringByAppendingPathComponent:kRealmFileName];
      BOOL realmExists = [[NSFileManager defaultManager] fileExistsAtPath:defaultRealmPath];
      if (realmExists) {
        // need to move realm to RealmDB folder
        NSString *defaultRealmLock = [documentFolder stringByAppendingPathComponent:kRealmLockFileName];
        NSString *defaultRealmFolder = [documentFolder stringByAppendingPathComponent:kRealmManagementFolder];
        
        NSString *newRealmLock = [dbfolder stringByAppendingPathComponent:kRealmLockFileName];
        NSString *newRealmFolder = [dbfolder stringByAppendingPathComponent:kRealmManagementFolder];
        
        [[NSFileManager defaultManager] moveItemAtPath:defaultRealmPath toPath:newRealmPath error:nil];
        [[NSFileManager defaultManager] moveItemAtPath:defaultRealmFolder toPath:newRealmFolder error:nil];
        [[NSFileManager defaultManager] moveItemAtPath:defaultRealmLock toPath:newRealmLock error:nil];
      }
    }
  }
  // migrate database
  RLMRealmConfiguration *config = [RLMRealmConfiguration defaultConfiguration];
  // Set the new schema version. This must be greater than the previously used
  // version (if you've never set a schema version before, the version is 0).
  config.schemaVersion = 1;
  if (folderExists) {
    config.fileURL = [NSURL URLWithString:newRealmPath];
  }

  // Set the block which will be called automatically when opening a Realm with a
  // schema version lower than the one set above
  config.migrationBlock = ^(RLMMigration *migration, uint64_t oldSchemaVersion) {
      // We haven’t migrated anything yet, so oldSchemaVersion == 0
      if (oldSchemaVersion < 1) {
          // Nothing to do!
          // Realm will automatically detect new properties and removed properties
          // And will update the schema on disk automatically
      }
  };

  // Tell Realm to use this new configuration object for the default Realm
  [RLMRealmConfiguration setDefaultConfiguration:config];

  // Now that we've told Realm how to handle the schema change, opening the file
  // will automatically perform the migration
  [RLMRealm defaultRealm];
}

I created a sub directory in document folder, move the default.realm file and default.realm.lock file to this sub directory and set its attribution to NSFileProtectionNone, but still seeing this crash on the production environment.

Version

Realm 10.7.2

What SDK flavour are you using?

Local Database only

Are you using encryption?

No, not using encryption

Platform OS and version(s)

iOS 14

Build environment

Xcode version: 13.1
Dependency manager and version: CocoaPods 1.11.3

@dianaafanador3
Copy link
Collaborator

Hi @haozhutw it seems like we are not able to access the file and because default.realm.lock is the first one we check, it is the one that throws.
This seems very difficult to reproduce and I've seen some other issue where this happens, I don't think this is related to the migration, but it happens on migrateRealmDB because it is the first time opening the realm on the app lifecycle.

Have you seen any increase of crashes in any particular iOS version?.

One change we can make it is to always set up the file protection for the folder.
So Instead of checking if the folder exist before setting the file protection

if (!folderExists) {
    BOOL res = [[NSFileManager defaultManager] createDirectoryAtPath:dbfolder
                              withIntermediateDirectories:YES
                                               attributes:@{ NSFileProtectionKey: NSFileProtectionNone }
                                                    error:nil];

Do it before opening the realm using

NSString *folderPath = [[config.fileURL absoluteString] stringByDeletingLastPathComponent];
    NSError *error;
    [[NSFileManager defaultManager] setAttributes:@{ NSFileProtectionKey:
                                                         NSFileProtectionNone } ofItemAtPath:folderPath error:&error];

making sure it is executed every time

@sync-by-unito sync-by-unito bot added the Waiting-For-Reporter Waiting for more information from the reporter before we can proceed label Jul 7, 2022
@haozhutw
Copy link
Author

haozhutw commented Jul 7, 2022

Hi @haozhutw it seems like we are not able to access the file and because default.realm.lock is the first one we check, it is the one that throws. This seems very difficult to reproduce and I've seen some other issue where this happens, I don't think this is related to the migration, but it happens on migrateRealmDB because it is the first time opening the realm on the app lifecycle.

Have you seen any increase of crashes in any particular iOS version?.

One change we can make it is to always set up the file protection for the folder. So Instead of checking if the folder exist before setting the file protection

if (!folderExists) {
    BOOL res = [[NSFileManager defaultManager] createDirectoryAtPath:dbfolder
                              withIntermediateDirectories:YES
                                               attributes:@{ NSFileProtectionKey: NSFileProtectionNone }
                                                    error:nil];

Do it before opening the realm using

NSString *folderPath = [[config.fileURL absoluteString] stringByDeletingLastPathComponent];
    NSError *error;
    [[NSFileManager defaultManager] setAttributes:@{ NSFileProtectionKey:
                                                         NSFileProtectionNone } ofItemAtPath:folderPath error:&error];

making sure it is executed every time

@dianaafanador3 Thank you for the advice! I'll try it out.

@github-actions github-actions bot added Needs-Attention Reporter has responded. Review comment. and removed Waiting-For-Reporter Waiting for more information from the reporter before we can proceed labels Jul 7, 2022
@haozhutw
Copy link
Author

haozhutw commented Jul 7, 2022

@dianaafanador3 This crash happens on iOS 12 and above versions, most of them happen on iOS 15 because most of the users are using the latest iOS system. I have updated my code logic, hope this issue can be fixed after my app goes live next time.

@dianaafanador3
Copy link
Collaborator

@haozhutw I'll leave this parked for a few months, let us know if after the update this keeps happening.

@sync-by-unito sync-by-unito bot removed the Needs-Attention Reporter has responded. Review comment. label Jul 7, 2022
@haozhutw
Copy link
Author

haozhutw commented Aug 8, 2022

Hi @dianaafanador3, I found another crash after publishing a new version of my app. Here is the crash log

Fatal Exception: RLMException
/var/mobile/Containers/Data/Application/25AD1DE0-8943-4C1C-9DE7-
C6BA3A7B24E0/Documents/RealmDB/default.realm.management/access_control.write.mx: Unable to open a realm at path 
'/var/mobile/Containers/Data/Application/25AD1DE0-8943-4C1C-9DE7-
C6BA3A7B24E0/Documents/RealmDB/default.realm.management/access_control.write.mx': open() failed: Operation not 
permitted Path: /var/mobile/Containers/Data/Application/25AD1DE0-8943-4C1C-9DE7-
C6BA3A7B24E0/Documents/RealmDB/default.realm.management/access_control.write.mx Exception backtrace: 0 

MyAppName 0x0000000101430b4c _ZNK5realm5Table10find_firstINS_4util8OptionalINS_8ObjectIdEEEEENS_6ObjKeyENS_6ColKeyET_ + 106252 1 
MyAppName 0x000000010130fb7c _ZNK5realm19ArrayFixedBytesNullINS_4UUIDELi16EE15find_first_nullEmm + 133360 2
MyAppName 0x000000010130dbe4 _ZNK5realm19ArrayFixedBytesNullINS_4UUIDELi16EE15find_first_nullEmm + 125272 3
MyAppName 0x0000000101310a64 _ZNK5realm19ArrayFixedBytesNullINS_4UUIDELi16EE15find_first_nullEmm + 137176 4  
MyAppName 0x0000000101314cfc _ZNK5realm19ArrayFixedBytesNullINS_4UUIDELi16EE15find_first_nullEmm + 154224 5 
MyAppName 0x00000001014f5834 _ZNK5realm5Table10find_firstINS_4util8OptionalINS_8ObjectIdEEEEENS_6ObjKeyENS_6ColKeyET_ + 912372 6 
MyAppName 0x00000001014f4960 _ZNK5realm5Table10find_firstINS_4util8OptionalINS_8ObjectIdEEEEENS_6ObjKeyENS_6ColKeyET_ + 908576 7 
MyAppName 0x00000001014f47b8 _ZNK5realm5Table10find_firstINS_4util8OptionalINS_8ObjectIdEEEEENS_6ObjKeyENS_6ColKeyET_ + 908152 8 
MyAppName 0x0000000101543358 _ZN5realm7Results8index_ofINS_4util8OptionalINS_4UUIDEEEEEmRKT_ + 73892 9 
MyAppName 0x00000001012035c0 _Z23RLMAddNotificationBlockI10RLMResultsEP20RLMNotificationTokenPT_U13block_pointerFvP11objc_objectP19RLMCollectionChangeP7NSErrorEPU28objcproto17OS_dis

I did change the file protection to NSFileProtectionNone every time before opening the database, no way to fix this? really upset.

@haozhutw
Copy link
Author

haozhutw commented Aug 8, 2022

I set file protection to NSFileProtectionNone to every realm file, including default.realm, default.realm.lock, default.realm.management and the dbfolder, I think that should be OK. I want to know if default.realm.management should be excluded or Could you indicate what I'm doing wrongly? Thank you so much!

@leemaguire
Copy link
Contributor

Hi @haozhutw you should just set NSFileProtectionNone on the parent folder for all these files.

Just a couple of questions:

  • Also do you have App Groups enabled? i.e is there an app extension which also tries to use the Realm file at the same time?
  • Do you know if the app is put to the background when the crash occurs?

@sync-by-unito sync-by-unito bot added the Waiting-For-Reporter Waiting for more information from the reporter before we can proceed label Aug 19, 2022
@haozhutw
Copy link
Author

Hi @haozhutw you should just set NSFileProtectionNone on the parent folder for all these files.

Just a couple of questions:

  • Also do you have App Groups enabled? i.e is there an app extension which also tries to use the Realm file at the same time?
  • Do you know if the app is put to the background when the crash occurs?

Thanks for your reply. I don't have App Groups enabled and no app extension. This crash is reported by Firebase and from the report it happens on foreground, here is the screenshot.
Screen Shot 2022-08-19 at 18 20 58

@github-actions github-actions bot added Needs-Attention Reporter has responded. Review comment. and removed Waiting-For-Reporter Waiting for more information from the reporter before we can proceed labels Aug 19, 2022
@leemaguire
Copy link
Contributor

@haozhutw can you update to the latest version of Realm and see if that helps.

@sync-by-unito sync-by-unito bot added Waiting-For-Reporter Waiting for more information from the reporter before we can proceed and removed Needs-Attention Reporter has responded. Review comment. labels Aug 25, 2022
@sync-by-unito sync-by-unito bot added the More-information-needed More information is needed to progress. The issue will close automatically in 2 weeks. label Sep 8, 2022
@benlacey
Copy link

benlacey commented Oct 3, 2022

Just in case it's useful for anyone else with this issue, we tracked it down in our app to the MFi App Launch feature, where if the user powered on their iOS device but didn't unlock it, when the MFi Bluetooth device was powered on and the app launched (underneath the locked screen) any access to files in the app's documents folder would fail with the same error seen in this report. The Crashlytics issue in our case also showed the app as not being in the background, but this isn't totally true - it's underneath the lock screen, not in the background ;)

So this could occur if the app makes use of any feature that could launch the app before the user has ever unlocked the device (geofencing maybe?)

Our initial workaround is checking UIApplication.shared.isProtectedDataAvailable before deciding whether to use the database or not.

@haozhutw
Copy link
Author

Still encounter this issue on the production environment

Fatal Exception: RLMException
/var/mobile/Containers/Data/Application/EA078871-274D-49EA-81C6-CDCF275CB0AF/Documents/RealmDB/default.realm.management/access_control.control.mx: Unable to open a realm at path '/var/mobile/Containers/Data/Application/EA078871-274D-49EA-81C6-CDCF275CB0AF/Documents/RealmDB/default.realm.management/access_control.control.mx': open() failed: Operation not permitted Path: /var/mobile/Containers/Data/Application/EA078871-274D-49EA-81C6-CDCF275CB0AF/Documents/RealmDB/default.realm.management/access_control.control.mx Exception backtrace: 0 

@github-actions github-actions bot added Needs-Attention Reporter has responded. Review comment. and removed Waiting-For-Reporter Waiting for more information from the reporter before we can proceed labels Oct 12, 2022
@sync-by-unito sync-by-unito bot removed the Needs-Attention Reporter has responded. Review comment. label Oct 21, 2022
@bmunkholm
Copy link
Contributor

@haozhutw There is nothing that points to a bug in Realm as far as we can tell. This appears to be a problem in your application that happens to occur when you open a Realm.
If you can provide a short repro-case that shows a problem in Realm we will be happy to look at it again. Until then I'll close this for now.
Thanks!

@nalexn
Copy link

nalexn commented Nov 21, 2022

@bmunkholm one use case was provided by @benlacey , in our case, this crash happens when the app receives a push notification after the device was rebooted but not yet unlocked. Realm is stored in a shared app group's directory and is accessed by a push notification service extension. As a fallback, even the attempt to boot an empty in-memory realm fails as well, because it still tries to write the lock file on disk at /tmp/in-memory-db.lock

@leemaguire
Copy link
Contributor

@nalexn did you try change the folder protection permissions in your App Group directory?

@nalexn
Copy link

nalexn commented Nov 30, 2022

@leemaguire I did, though I don't set it to none as recommended and use completeUntilFirstUserAuthentication for privacy reasons. I consider changing protection to none a hack full of compromises, which, in fact, does not work for in-memory realms as I noted above, where tmp folder is protected as well and you cannot even change the protection to none.

@leemaguire
Copy link
Contributor

@nalexn thanks for the details, I'll bring this topic back up for discussion with the team to see if anything can be done.

@sync-by-unito sync-by-unito bot removed the More-information-needed More information is needed to progress. The issue will close automatically in 2 weeks. label Dec 6, 2022
@leemaguire
Copy link
Contributor

I'm closing this issue as I have submitted a feature request for #7874 (comment) in Realm Core.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 18, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

6 participants