New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

It should be possible to debug an application using encrypted realms #1885

Closed
bdash opened this Issue May 8, 2015 · 20 comments

Comments

Projects
None yet
@bdash
Contributor

bdash commented May 8, 2015

We disabled debugging of applications using encrypted realms to address a frequent LLDB deadlock that would occur (issue #1625). We should address the underlying issue that causes the hang and reinstate the ability to debug applications that use encrypted realms. This will likely require changes in core.

@jondal

This comment has been minimized.

Show comment
Hide comment
@jondal

jondal Jul 3, 2015

So how would one go about debugging an app using an encrypted Realm database?

jondal commented Jul 3, 2015

So how would one go about debugging an app using an encrypted Realm database?

@jpsim

This comment has been minimized.

Show comment
Hide comment
@jpsim

jpsim Jul 3, 2015

Contributor

You should disable encryption, without modifying any of your code, by setting the following environment variable: REALM_DISABLE_ENCRYPTION=YES. You can read more about this in our docs: https://realm.io/docs/objc/latest/#debugging-encrypted-realms

Contributor

jpsim commented Jul 3, 2015

You should disable encryption, without modifying any of your code, by setting the following environment variable: REALM_DISABLE_ENCRYPTION=YES. You can read more about this in our docs: https://realm.io/docs/objc/latest/#debugging-encrypted-realms

@marchyman

This comment has been minimized.

Show comment
Hide comment
@marchyman

marchyman Jul 3, 2015

Which works, but... if the problem you are trying to debug has anything to do with the content of an encrypted database you are back to adding NSLog statements with the added hassle of not being able to run your program within Xcode.

marchyman commented Jul 3, 2015

Which works, but... if the problem you are trying to debug has anything to do with the content of an encrypted database you are back to adding NSLog statements with the added hassle of not being able to run your program within Xcode.

@segiddins

This comment has been minimized.

Show comment
Hide comment
@segiddins

segiddins Jul 3, 2015

Contributor

@marchyman we understand that's far from ideal, hence why adding support for debugging encrypted realms is a P1 (priority one) for us at the moment.

Contributor

segiddins commented Jul 3, 2015

@marchyman we understand that's far from ideal, hence why adding support for debugging encrypted realms is a P1 (priority one) for us at the moment.

yusuga added a commit to yusuga/YSRealmStore that referenced this issue Jul 25, 2015

Support Realm (0.93.2)
Abolished the XCTest of encryption.
See Also: realm/realm-cocoa#1885
@natanrolnik

This comment has been minimized.

Show comment
Hide comment
@natanrolnik

natanrolnik Jul 26, 2015

Contributor

+1 for this adding the support for debugging encrypted realms.

In my case, I have a pre-populated realm file, that is copied in the first launch... And as I'm using Swift 2, using Realm 0.92 doesn't work due to some error in RLMAccessor.mm.

Any suggestions on what should I do?

Contributor

natanrolnik commented Jul 26, 2015

+1 for this adding the support for debugging encrypted realms.

In my case, I have a pre-populated realm file, that is copied in the first launch... And as I'm using Swift 2, using Realm 0.92 doesn't work due to some error in RLMAccessor.mm.

Any suggestions on what should I do?

@bmunkholm bmunkholm added P2 and removed P1 labels Sep 1, 2015

@narup

This comment has been minimized.

Show comment
Hide comment
@narup

narup Sep 30, 2015

Can I migrate from un-encrypted to encrypted version later? If I decide to use encryption once debugging support is enabled

narup commented Sep 30, 2015

Can I migrate from un-encrypted to encrypted version later? If I decide to use encryption once debugging support is enabled

@jpsim

This comment has been minimized.

Show comment
Hide comment
@jpsim

jpsim Sep 30, 2015

Contributor

Yes, you can specify a new encryption key as part of -[RLMRealm writeCopyToPath:encryptionKey:error:].

Contributor

jpsim commented Sep 30, 2015

Yes, you can specify a new encryption key as part of -[RLMRealm writeCopyToPath:encryptionKey:error:].

@jondal

This comment has been minimized.

Show comment
Hide comment
@jondal

jondal Oct 1, 2015

May I kindly suggest that this be upgraded to P1 again?

jondal commented Oct 1, 2015

May I kindly suggest that this be upgraded to P1 again?

@jpsim

This comment has been minimized.

Show comment
Hide comment
@jpsim

jpsim Oct 1, 2015

Contributor

May I kindly suggest that this be upgraded to P1 again?

This issue is essentially equivalent to "rewrite how encryption works", which is certainly not a top priority for us at the moment. To debug encrypted Realms during development, you can disable encryption without changing any of your code by setting the REALM_DISABLE_ENCRYPTION environment variable, as described in our docs:

Attaching an LLDB session to a process using an encrypted Realm is not currently supported. In some cases, you may work around this by setting REALM_DISABLE_ENCRYPTION=YES in your environment; this variable forces all encrypted API methods to work without encryption, allowing you to debug without having to change which Realm APIs you call in your app code. For obvious security reasons, this workaround will not work if you try to access an existing .realm file that was previously encrypted (this will result in an exception: File::AccessError), but it might be useful in cases where your app creates & accesses new Realm files.

Contributor

jpsim commented Oct 1, 2015

May I kindly suggest that this be upgraded to P1 again?

This issue is essentially equivalent to "rewrite how encryption works", which is certainly not a top priority for us at the moment. To debug encrypted Realms during development, you can disable encryption without changing any of your code by setting the REALM_DISABLE_ENCRYPTION environment variable, as described in our docs:

Attaching an LLDB session to a process using an encrypted Realm is not currently supported. In some cases, you may work around this by setting REALM_DISABLE_ENCRYPTION=YES in your environment; this variable forces all encrypted API methods to work without encryption, allowing you to debug without having to change which Realm APIs you call in your app code. For obvious security reasons, this workaround will not work if you try to access an existing .realm file that was previously encrypted (this will result in an exception: File::AccessError), but it might be useful in cases where your app creates & accesses new Realm files.

@zpapp

This comment has been minimized.

Show comment
Hide comment
@zpapp

zpapp Oct 2, 2015

This is very disappointing. I think this is very close to saying that Realm on iOS doesn't support encryption. Even if I decide to put up with this major inconvenience and encrypt my database, I have to wonder what kind of ripple effect this will have on encryption as a feature. How many Realm users will decide that using encryption is not worth the trouble, how the decreased usage will lower the priority of any future bug related to encryption, while at the same time increasing the chance that I will be the first one to run into such bugs...

IMHO this should be the first thing you mention in the "Encryption" section of the documentation, not the WatchOS radar. (Edit: As a side note, the phrase "attaching an LLDB session to a process..." obfuscates the problem a bit, even though I am sure it's technically accurate. "You won't be able to run your app from Xcode" would be a more understandable statement in my opinion.)

zpapp commented Oct 2, 2015

This is very disappointing. I think this is very close to saying that Realm on iOS doesn't support encryption. Even if I decide to put up with this major inconvenience and encrypt my database, I have to wonder what kind of ripple effect this will have on encryption as a feature. How many Realm users will decide that using encryption is not worth the trouble, how the decreased usage will lower the priority of any future bug related to encryption, while at the same time increasing the chance that I will be the first one to run into such bugs...

IMHO this should be the first thing you mention in the "Encryption" section of the documentation, not the WatchOS radar. (Edit: As a side note, the phrase "attaching an LLDB session to a process..." obfuscates the problem a bit, even though I am sure it's technically accurate. "You won't be able to run your app from Xcode" would be a more understandable statement in my opinion.)

@jpsim

This comment has been minimized.

Show comment
Hide comment
@jpsim

jpsim Oct 2, 2015

Contributor

I'm sorry you're disappointed. We know this is important to many users of encryption, and maybe others can share their own tips when debugging encrypted Realms (/cc @anlaital).

The best way you can help us prioritize the issue is to help us grow our engineering team, so we can put more people on important tasks like these: https://realm.io/jobs.

Contributor

jpsim commented Oct 2, 2015

I'm sorry you're disappointed. We know this is important to many users of encryption, and maybe others can share their own tips when debugging encrypted Realms (/cc @anlaital).

The best way you can help us prioritize the issue is to help us grow our engineering team, so we can put more people on important tasks like these: https://realm.io/jobs.

@bmunkholm

This comment has been minimized.

Show comment
Hide comment
@bmunkholm

bmunkholm Oct 2, 2015

Contributor

@zpapp I can assure you that supporting encryption as a feature is a top priority for us. And we always prioritise bug fixing and support above anything else - so there is no need to worry about the encryption feature stability, although I do follow your line of thought.
Having this as a P2 does not mean that it's low priority, just that the ones in P1 are higher priority to get fixed before we tackle the current P2's. One of the factors we use when prioritising is if there are workarounds. Another factor is if it affects current users, and yet another the effort it requires to fix.
Some of the other features we have had even more requests for, and which we are working hard to complete, are null support, async queries and fine grained notifications. What's not visible from this priority list is also that fixing the debug support for encryption actually requires a lot of work in the core database team, so this reflects that teams priorities more than the Cocoa teams.
But we hear you, and will do our best to get it resolved as quickly as we can with due consideration to other requests as well.

Contributor

bmunkholm commented Oct 2, 2015

@zpapp I can assure you that supporting encryption as a feature is a top priority for us. And we always prioritise bug fixing and support above anything else - so there is no need to worry about the encryption feature stability, although I do follow your line of thought.
Having this as a P2 does not mean that it's low priority, just that the ones in P1 are higher priority to get fixed before we tackle the current P2's. One of the factors we use when prioritising is if there are workarounds. Another factor is if it affects current users, and yet another the effort it requires to fix.
Some of the other features we have had even more requests for, and which we are working hard to complete, are null support, async queries and fine grained notifications. What's not visible from this priority list is also that fixing the debug support for encryption actually requires a lot of work in the core database team, so this reflects that teams priorities more than the Cocoa teams.
But we hear you, and will do our best to get it resolved as quickly as we can with due consideration to other requests as well.

@anlaital

This comment has been minimized.

Show comment
Hide comment
@anlaital

anlaital Oct 2, 2015

As stated above, debugging encrypted Realms isn't something that's currently working. What we've done is that we simply don't use encryption in debug builds, so it's enabled only in Beta and App Store builds. It's definitely not optimal, but we haven't been able to find another solution to it so far. Encryption has been working solid for a long time now, so we don't find this approach too risky.

anlaital commented Oct 2, 2015

As stated above, debugging encrypted Realms isn't something that's currently working. What we've done is that we simply don't use encryption in debug builds, so it's enabled only in Beta and App Store builds. It's definitely not optimal, but we haven't been able to find another solution to it so far. Encryption has been working solid for a long time now, so we don't find this approach too risky.

@zpapp

This comment has been minimized.

Show comment
Hide comment
@zpapp

zpapp Oct 2, 2015

I appreciate the responses!

One of my concerns is how would I debug a problem (in my app, not with Realm) that can only be reproduced by a specific Realm file that now happens to be encrypted (I might get such a file from a user).

What's the easiest way to remove encryption from a previously encrypted database file?

My first thought was that the browser might do this, but it looks like the browser can't even open an encrypted realm. So I am guessing I'll have to write an app to do this...? Is there an inverse of the writeCopyToPath:encryptionKey: method that would allow me to wholesale remove encryption, instead of copying the objects one by one?

zpapp commented Oct 2, 2015

I appreciate the responses!

One of my concerns is how would I debug a problem (in my app, not with Realm) that can only be reproduced by a specific Realm file that now happens to be encrypted (I might get such a file from a user).

What's the easiest way to remove encryption from a previously encrypted database file?

My first thought was that the browser might do this, but it looks like the browser can't even open an encrypted realm. So I am guessing I'll have to write an app to do this...? Is there an inverse of the writeCopyToPath:encryptionKey: method that would allow me to wholesale remove encryption, instead of copying the objects one by one?

@tgoyne

This comment has been minimized.

Show comment
Hide comment
@tgoyne

tgoyne Oct 2, 2015

Member

The version of writeCopyToPath that doesn't take an encryption key writes an unencrypted copy.

Member

tgoyne commented Oct 2, 2015

The version of writeCopyToPath that doesn't take an encryption key writes an unencrypted copy.

@jpsim

This comment has been minimized.

Show comment
Hide comment
@jpsim

jpsim Oct 2, 2015

Contributor

One of my concerns is how would I debug a problem (in my app, not with Realm) that can only be reproduced by an encrypted Realm file (which I would get from a user).

Using console logs, or a remote logging tool like PonyDebugger or Antenna.

What's the easiest way to remove encryption from a previously encrypted database file?

Using writeCopyToPath: without an encryption key. Changing the encryption key for a file can be done in a similar way.

My first thought was that the browser might do this, but it looks like the browser can't even open an encrypted realm.

The Realm Browser can open encrypted Realms as of realm/realm-browser-osx#74, a build of which is awaiting App Store approval.

Contributor

jpsim commented Oct 2, 2015

One of my concerns is how would I debug a problem (in my app, not with Realm) that can only be reproduced by an encrypted Realm file (which I would get from a user).

Using console logs, or a remote logging tool like PonyDebugger or Antenna.

What's the easiest way to remove encryption from a previously encrypted database file?

Using writeCopyToPath: without an encryption key. Changing the encryption key for a file can be done in a similar way.

My first thought was that the browser might do this, but it looks like the browser can't even open an encrypted realm.

The Realm Browser can open encrypted Realms as of realm/realm-browser-osx#74, a build of which is awaiting App Store approval.

@tgoyne tgoyne removed the Blocked label Nov 17, 2015

@tgoyne

This comment has been minimized.

Show comment
Hide comment
@tgoyne

tgoyne Nov 17, 2015

Member

The required core changes have now landed in master, so now we just need a new core release to make this possible.

Member

tgoyne commented Nov 17, 2015

The required core changes have now landed in master, so now we just need a new core release to make this possible.

@zpapp

This comment has been minimized.

Show comment
Hide comment
@zpapp

zpapp Nov 18, 2015

That's great news, thanks for letting us know!

zpapp commented Nov 18, 2015

That's great news, thanks for letting us know!

@tgoyne tgoyne self-assigned this Nov 18, 2015

@tgoyne tgoyne closed this in #2874 Nov 18, 2015

@tgoyne tgoyne removed the P2 label Nov 18, 2015

@codytwinton

This comment has been minimized.

Show comment
Hide comment
@codytwinton

codytwinton Dec 17, 2015

Any update here?

codytwinton commented Dec 17, 2015

Any update here?

@bdash

This comment has been minimized.

Show comment
Hide comment
@bdash

bdash Dec 17, 2015

Contributor

This is fixed in Realm Objective-C and Swift v0.97, released earlier today.

Contributor

bdash commented Dec 17, 2015

This is fixed in Realm Objective-C and Swift v0.97, released earlier today.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment