TWRP backup doesn't work with Google "work profiles"

[angadsingh]

Device: OP6
Google "Work profile" created for work account (https://support.google.com/work/android/answer/6191949?hl=en)
TWRP/Nandroid backup from recovery throws the following error:

I:Error adding file '/data/system_ce/10' to '/data/media/0/TWRP/BACKUPS/c73c84e3/2018-06-14--06-19-03/data.ext4.win'
Error creating backup.
I:ERROR tarList for thread ID 0
Error creating backup.
createTarFork() process ended with ERROR: 255
Backup Failed. Cleaning Backup Folder.

It seems that (similar to multi-user account or parallel apps), work profiles create a new user on the device, and /data/system_ce/10 might be an artifact of my work profile (not sure). The other folder is /data/system_ce/0.

Going by XDA, TWRP doesn't support multi-user profiles. Does it not support work profiles either?

[fxsheep]

work profiles uses part of multiuser

[angadsingh]

They use android's file based encryption (FBE): https://source.android.com/security/encryption/file-based

The system_ce and system_de folders are actually Credential Encrypted (CE) storage and Device Encrypted (DE) storage as documented above.

I can't even do an adb pull on my work profile's folders:

adb: error: failed to copy '/data/system_ce/10/gZbggZAffqRWsSUW3jB7gD' to './10/gZbggZAffqRWsSUW3jB7gD': open failed: Required key not available```

[piranhaphish]

Having the same issue. Is there any way to exclude certain folders from the backup?

[Mygod]

I agree that excluding secondary users (if backup is not possible) would be great.

[andreas1288]

Please ignore that files on backup AND restore. So that the other user of profile will not be touched at all.

[lars18th]

Hi,

I use work profile as a secondary user. So, PLEASE, include:

1. The option to EXCLUDE some user defined directories & multi-user files.
2. The option to DUMP the DATA partition, like other partitions, instead of only use the TAR archieve.
3. The option to add to the backup the INTERNAL STORAGE.

With all of this we'll can do a full backup using multiuser profiles.
Thank you!

[jwgn]

Any update on this one? I backed up my phone while work profile was enabled. Now I am unable to restore: extractTarFork() process ended with ERROR: 255

[CaptainThrowback]

This should work fine now, but you would need to backup the phone with the newest version of TWRP, and then the new backup should restore properly. Any previous backups would not work as the work profile would've been encrypted.

[jwgn]

No, it didn't work. I created the backup with 3.4.0 and tried to restore it with 3.4.0. Error 255.
Turns out I had a user with ID 11, probably created by Island, which is using the work profile feature.

Cannot find key for 11
error looking up proper e4crypt policy for '//data/system_de/11/' - 1DE11
tar_extract_file(): failed to extract //data/system_de/11/ !!!

So I deleted all files and folders named "11" from /data/system_de/, /data/misc/user/ and /data/vendor_de/ manually from the backup files (with 7-Zip) and pushed them back to my phone. Then I was able to restore the backup.

Not sure if this is a corner case, but apparently TWRP did not skip this profile when creating the backup.

[CaptainThrowback]

Did you first decrypt the work profile before performing the backup? Where is the recovery log? You shouldn't have to delete or skip anything. Without logs it's impossible to determine what happened here.

[jwgn]

I did not decrypt anything before taking the backup.

It initially failed on the first file (data.f2fs.win000) with "11" files/folders in /data/misc/user/ and /data/vendor_de/.

So I deleted them and then it failed on the last file (data.f2fs.win012) with "11" files in /data/system_de/.

I was not unable to restore anything without deleting these files.

Here is the log from the second attempt https://www.dropbox.com/s/9jec7aobe02to0p/recovery%5B1%5D.7z?dl=0

[CaptainThrowback]

I did not decrypt anything before taking the backup.

It initially failed on the first file (data.f2fs.win000) with "11" files/folders in /data/misc/user/ and /data/vendor_de/.

So I deleted them and then it failed on the last file (data.f2fs.win012) with "11" files in /data/system_de/.

I was not unable to restore anything without deleting these files.

Here is the log from the second attempt https://www.dropbox.com/s/9jec7aobe02to0p/recovery%5B1%5D.7z?dl=0

I think I'm missing a log. You're going to need to walk through your entire process, and provide the logs from each step. There should be a log from the backup in the backup folder - I think that one will be the most useful. For the restore, did you format the device or do something that would've deleted the work profile prior to restoring? According to that log, there's no user 11 on the device, which explains why it couldn't restore it.

[jwgn]

I do not have any other logs. All previous logs got deleted during my ROM flashing attempts.
Yes, I formatted everything including /system, then flashed a fresh factory image, then tried to restore the data partition.
Restoring everything including all other partitions did not work, I ended up in a bootloop.

As already mentioned, I had Island (https://island.oasisfeng.com/) installed on my phone and I think this is where that user "11" came from? But it was not in use, I think the work profile was not even active when taking the backup.

[CaptainThrowback]

There is a recovery log in your backup folder. Since you were able to restore your backup, that folder/log must exist. That's the log we need to see.

[jwgn]

Yes, thanks, found it.
https://www.dropbox.com/s/vcdgxq1cbo0c8id/recovery.7z?dl=0

[noahajac]

Backup/restore operations will fail without all users decrypted. Whether they are in use or not. TWRP will automatically try to decrypt secondary profiles with the default password and/or the password provided by the user for the primary profile. I don't know what "password" is used by Island.

I'll work on adding ignore logic for non-decrypted users.

[jwgn]

Ok, but the backup operation succeeded without any errors while the restore operation failed with
error looking up proper e4crypt policy for '//data/system_de/11/' - 1DE11

Shouldn't the backup abort at this point instead of going on?
Failed to decrypt user 11

[noahajac]

Right now there is no logic to explicitly abort backups with undecrypted users. I'm not sure why it doesn't error out when backing up but does when restoring.

[Micha-Btz]

Hi,
maybe you have an idea for me, as decryption on user 10 always fails.

e4crypt_unlock_user_key 0 serial=0 token_present=0
Skipping non-key ..
Skipping non-key .
Trying user CE key /data/misc/vold/user_keys/ce/0/current
Successfully retrieved key
Determining wrapped-key support for /data
fbe.data.wrappedkey = false
Determining wrapped-key support for /data
fbe.data.wrappedkey = false
Added key 798111340 (ext4:341e9bb1c698a11d) to keyring 449853792 in process 540
Added key 972089171 (f2fs:341e9bb1c698a11d) to keyring 449853792 in process 540
Added key 243114293 (fscrypt:341e9bb1c698a11d) to keyring 449853792 in process 540
Installed ce key for user 0
User 0 Decrypted Successfully!
User 0 Decrypted Successfully
I:User 10 is not decrypted.
Attempting to decrypt FBE for user 10...
Using synthetic password method
Handle is 'a94c05821e4281f1'
Using synthetic password method
Handle is 'a94c05821e4281f1'
using secdis
gatekeeper verification failed
Using synthetic password method
Handle is 'a94c05821e4281f1'
e4crypt_unlock_user_key 10 serial=0 token_present=0
Skipping non-key ..
Skipping non-key .
Trying user CE key /data/misc/vold/user_keys/ce/10/current
Using Keymaster HAL: 4 from QTI for encryption.  Security level: TRUSTED_ENVIRONMENT, HAL: android.hardware.keymaster@4.0::IKeymasterDevice/default
Failed to read from /data/misc/vold/user_keys/ce/10/current/keymaster_key_blob
Failed to find working ce key for user 10
Couldn't read key for 10
e4crypt_unlock_user_key returned fail
Failed to decrypt user 10

The path for /data/misc/vold/user_keys/ce/0/current and /data/misc/vold/user_keys/ce/10/current
looks identical. Both contains the files encrypted_key,secdiscardable, stretching and version.
The file keymaster_key_blob doesn't exist for both.
There are hard coded paths that twrp is looking for or searches it the path at boot?
I attach the whole recovery.log.

recovery-log.txt

twrp path https://github.com/redispade/device_xiaomi_grus-twrp

Xiaomi Mi 9SE

[CaptainThrowback]

Hi,
maybe you have an idea for me, as decryption on user 10 always fails.

e4crypt_unlock_user_key 0 serial=0 token_present=0
Skipping non-key ..
Skipping non-key .
Trying user CE key /data/misc/vold/user_keys/ce/0/current
Successfully retrieved key
Determining wrapped-key support for /data
fbe.data.wrappedkey = false
Determining wrapped-key support for /data
fbe.data.wrappedkey = false
Added key 798111340 (ext4:341e9bb1c698a11d) to keyring 449853792 in process 540
Added key 972089171 (f2fs:341e9bb1c698a11d) to keyring 449853792 in process 540
Added key 243114293 (fscrypt:341e9bb1c698a11d) to keyring 449853792 in process 540
Installed ce key for user 0
User 0 Decrypted Successfully!
User 0 Decrypted Successfully
I:User 10 is not decrypted.
Attempting to decrypt FBE for user 10...
Using synthetic password method
Handle is 'a94c05821e4281f1'
Using synthetic password method
Handle is 'a94c05821e4281f1'
using secdis
gatekeeper verification failed
Using synthetic password method
Handle is 'a94c05821e4281f1'
e4crypt_unlock_user_key 10 serial=0 token_present=0
Skipping non-key ..
Skipping non-key .
Trying user CE key /data/misc/vold/user_keys/ce/10/current
Using Keymaster HAL: 4 from QTI for encryption.  Security level: TRUSTED_ENVIRONMENT, HAL: android.hardware.keymaster@4.0::IKeymasterDevice/default
Failed to read from /data/misc/vold/user_keys/ce/10/current/keymaster_key_blob
Failed to find working ce key for user 10
Couldn't read key for 10
e4crypt_unlock_user_key returned fail
Failed to decrypt user 10

The path for /data/misc/vold/user_keys/ce/0/current and /data/misc/vold/user_keys/ce/10/current
looks identical. Both contains the files encrypted_key,secdiscardable, stretching and version.
The file keymaster_key_blob doesn't exist for both.
There are hard coded paths that twrp is looking for or searches it the path at boot?
I attach the whole recovery.log.

recovery-log.txt

twrp path https://github.com/redispade/device_xiaomi_grus-twrp

Xiaomi Mi 9SE

Do you have a separate password for user 10? If so, did you attempt to decrypt using Advanced -> Decrypt Users?

[Micha-Btz]

I only have a password for user 0, then I think the password is derived from user 0.
For the work profile I use shelter app.
Advanced -> Decrypt Users? doesnt work

[KoffieNu]

I found this ticket after having this problem myself. The question below triggered something I wanted to try:

Do you have a separate password for user 10? If so, did you attempt to decrypt using Advanced -> Decrypt Users?

That got me thinking. I went into accounts and found an option 'use one lock'. I unchecked that and set the same pin code for the work profile. That enabled TWRP to unlock both profiles with the same pin. Somehow 'use one lock' and setting it once (main) is in Android 10 different from setting the same code twice.

With the 'use one lock' I guess Android 10 thinks of a lock for the work profile and sets (and unlocks) it via internal code instead of setting the same code. Advanced -> Decrypt Users doesn't work with 'use one lock' but it is even not needed when you manually set the same code on both main and work profile. (is probably needed when you set 2 different codes)

Device: Samsung Tab S5e
TWRP: 3.4.0-0 from https://build.twrp.me/twrp-3.4.0-0-gts4lvwifi.img (@luk1337 built)
Image: Lineage 17.1
Firmware: T720XXU1BTF7_CL18864194_QB32199498_REV00.zip

[Micha-Btz]

That got me thinking. I went into accounts and found an option 'use one lock'. I unchecked that and set the same pin code for the work profile. That enabled TWRP to unlock both profiles with the same pin

Great man, works on my Xiaomi Mi 9SE too
:-)

[Uelk]

That got me thinking. I went into accounts and found an option 'use one lock'. I unchecked that and set the same pin code for the work profile. That enabled TWRP to unlock both profiles with the same pin

Sadly does not work for me.

Oneplus 7 GM1903
OOS 10.03.GM57BA
TWRP 3.3.1-74

Edit: With TWRP 3.4.0-0 it works! Thanks!

[ultra-azu]

That got me thinking. I went into accounts and found an option 'use one lock'. I unchecked that and set the same pin code for the work profile. That enabled TWRP to unlock both profiles with the same pin

While it did work for the backup, after formatting Data I couldn't restore it back. I guess it's that time of the year again where I have to start everything from scratch :/ . I guess it could be worse too, at least I have a way to recover my contacts and other stuff.

[CaptainThrowback]

That got me thinking. I went into accounts and found an option 'use one lock'. I unchecked that and set the same pin code for the work profile. That enabled TWRP to unlock both profiles with the same pin

While it did work for the backup, after formatting Data I couldn't restore it back. I guess it's that time of the year again where I have to start everything from scratch :/ . I guess it could be worse too, at least I have a way to recover my contacts and other stuff.

Just create the user account in Android again first, and then the restore should work fine.

[T-vK]

Can anyone confirm this? I'm worried my backup strategy isn't sufficient.

[vukisz]

Hi having quite similar error, but for system user:

error looking up proper e4crypt policy for '//data/system_de/0/' - 1DE0
tar_extract_file(): failed to extract //data/system_de/0/ !!!
I:Unable to extract tar archive '/data/media/TWRP/BACKUPS/OldData/data.ext4.win012'

More context on this xda post.

[noahajac]

Is this after a wipe/format of data? Do all users on the backup already exist on the device?

[vukisz]

Tried wipe at first. Then did a full data format. Still same issue.
I am not quite sure how to check users from backup and create them in destination.

[noahajac]

Do you not know the user profiles you had on the original device? If you did a format you will need to boot again and recreate the users. If you only had one user (0) then just booting should be sufficient.

…

On Thu, Apr 1, 2021 at 12:08 PM vukisz ***@***.***> wrote: Tried wipe at first. Then did a full data format. Still same issue. I am not quite sure how to check users from backup and create them in destination. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1256 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD5PUSXEETPMHF6NFAI62GLTGSK7ZANCNFSM4FFOF36Q> .

[vukisz]

I am still using old device. I have not created any additional user profiles explicitly. Maybe created guest account, buy then deleted it.
How I can check user existing profiles on the original device? And how those could be recreated in twrp? Using advanced -> terminal?

[noahajac]

Post a recovery.log of both the restore on the new device and of just booting TWRP and mounting /data on the old device.

…

On Thu, Apr 1, 2021, 12:19 PM vukisz ***@***.***> wrote: I am still using old device. I have not created any additional user profiles explicitly. Maybe created guest account, buy then deleted it. How I can check user existing profiles on the original device? And how those could be recreated in twrp? Using advanced -> terminal? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1256 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD5PUSTA26SEAV6HKSPDMHDTGSMKVANCNFSM4FFOF36Q> .

[vukisz]

Sure. https://drive.google.com/file/d/1CZTV5sFlTBN8-dZ3AMb6Amw8jytfpQHl/view?usp=drivesdk
Can you access it?

[noahajac]

Did you boot system at all on the new device before attempting to restore?

…

On Thu, Apr 1, 2021 at 12:34 PM vukisz ***@***.***> wrote: Sure. https://drive.google.com/file/d/1CZTV5sFlTBN8-dZ3AMb6Amw8jytfpQHl/view?usp=drivesdk Can you access it? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1256 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD5PUSXLPPRZXHO3QFYPZ6LTGSOBRANCNFSM4FFOF36Q> .

[vukisz]

Nope, just did format, restart twrp, copied backup using twrp file manager and hit restore.
I have try booting after data format? :-)

[noahajac]

Boot first to create user 0.

…

On Thu, Apr 1, 2021, 12:40 PM vukisz ***@***.***> wrote: Nope, just did format, restart twrp, copied backup using twrp file manager and hit restore. I have try booting after data format? :-) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1256 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD5PUSUW6PE5ZMLVVD6A5PLTGSOX3ANCNFSM4FFOF36Q> .

[vukisz]

Data format + restart ends in a bootloop. Is it as expected?

[noahajac]

Actual format (the button literally says "Format Data") or a factory reset?

…

On Thu, Apr 1, 2021, 12:43 PM vukisz ***@***.***> wrote: Data format + restart ends in a bootloop. Is it as expected? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1256 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD5PUSWGKMZ2NOBMM7RUGFTTGSPD3ANCNFSM4FFOF36Q> .

[vukisz]

TWRP->Wipe-Format data-> Then full screen text:

Wipe
Format Data
Format Data will wipe all of your apps, backups, pictures, videos, media, and removes encryption on internal storage.
This cannot be undone.
Type yes to continue. Press back to cancel.

Typing yes and confirming

And this ends in:

Updating partition details...
done
Full SELinux support is present.
MTP Enabled
Formatting Data using mke2fs...
Done.
You may need to reboot recovery to be able to use / Formatting metadata using mke2fs...
data again.
Done.
Updating partition details...
...done

[noahajac]

Then no that is not expected. Is it actually bootlooping or just taking time to boot?

…

On Thu, Apr 1, 2021, 12:48 PM vukisz ***@***.***> wrote: TWRP->Wipe-Format data-> Then full screen text: Wipe Format Data Format Data will wipe all of your apps, backups, pictures, videos, media, and removes encryption on internal storage. This cannot be undone. Type yes to continue. Press back to cancel. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1256 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD5PUSV3J2GPULSHBGCLFMLTGSPUFANCNFSM4FFOF36Q> .

[vukisz]

Bootlooping:

1. Warning, that bootloader is unlocked window
2. Oneplus icon window
3. Vibrate restart and go back to 1.

Maybe I'll try flashing boot, vendor and system again

[vukisz]

Finally resolved.
Flashed all the factory images from OTA archive provided by manufacturer.
Then restoring System/Vendor/Boot and formating data was not ending in bootloop.
After that restoring data went well. And finally after a cache wipe new phone was successfully running old phones backup.
Thanks for helping @noahajac ;-)

[nordurljosahvida]

Hi all, just to be sure I'm good, backing up now with "warning: not all users decrypted" after "failed to decrypt user 10" [using work profile with a few apps in it, generated with app "Shelter" from f-droid. Backup is now proceeding after skipping the warning.

A. Is there a way to also decrypt user 10?
B. If not [and really it doesn't matter], will restore succeed without user 10 after a complete wipe / formatting of the device?

Thanks!

[EmanuelLoos]

Is there a way to also decrypt user 10?

Only after you (or someone else) read through the AOSP (Android Open Source Project) source code and implement a way to decrypt user 10 into TWRP. The AOSP source code is publicly available so theoretically it would be possible if you know Java and C++ enough and have the time. If you decide to do that please make your source code available and make a pull request so it works for others as well.

[nordurljosahvida]

Not a dev here so unfortunately I won't be able to contribute here.

About question B has anyone tried with latest TWRP?

Thanks

[Iey4iej3]

Hi all, just to be sure I'm good, backing up now with "warning: not all users decrypted" after "failed to decrypt user 10" [using work profile with a few apps in it, generated with app "Shelter" from f-droid. Backup is now proceeding after skipping the warning.

A. Is there a way to also decrypt user 10?
B. If not [and really it doesn't matter], will restore succeed without user 10 after a complete wipe / formatting of the device?

Thanks!

There is a solution: look back at #1256 (comment)

[EmanuelLoos]

There is a solution: look back at #1256 (comment)

Oh, right, I forgot about this since when I had the issue I couldn't enable "Use one lock" as my phone always got stuck at the boot animation.

[nordurljosahvida]

I found this ticket after having this problem myself. The question below triggered something I wanted to try:

Do you have a separate password for user 10? If so, did you attempt to decrypt using Advanced -> Decrypt Users?

That got me thinking. I went into accounts and found an option 'use one lock'. I unchecked that and set the same pin code for the work profile. That enabled TWRP to unlock both profiles with the same pin. Somehow 'use one lock' and setting it once (main) is in Android 10 different from setting the same code twice.

With the 'use one lock' I guess Android 10 thinks of a lock for the work profile and sets (and unlocks) it via internal code instead of setting the same code. Advanced -> Decrypt Users doesn't work with 'use one lock' but it is even not needed when you manually set the same code on both main and work profile. (is probably needed when you set 2 different codes)

Device: Samsung Tab S5e
TWRP: 3.4.0-0 from https://build.twrp.me/twrp-3.4.0-0-gts4lvwifi.img (@luk1337 built)
Image: Lineage 17.1
Firmware: T720XXU1BTF7_CL18864194_QB32199498_REV00.zip

Very useful info and discovery, however it's the exact opposite for me. I had one lock enabled already, disabled it, set the exact same password for user 10, and TWRP automatically decrypts both successfully in sequence.

OnePlus 7 Pro on Resurrection Remix 10, work profile managed by shelter

[Iey4iej3]

Very useful info and discovery, however it's the exact opposite for me. I had one lock enabled already, disabled it, set the exact same password for user 10, and TWRP automatically decrypts both successfully in sequence.

The real point is that, if you enable "use one lock", the low-level encryption keys for two users are different. That is to say, we suspect that Android use one key to compute that of the other, so in this case, you could not simply input the same key to decrypt both. That is why we need to read the code of Android to understand what is really happening.

[yshui]

Having the same problem and unfortunately I wiped data and forgot to backup my work profile beforehand.

I did some digging:

https://github.com/aosp-mirror/platform_frameworks_base/blob/c5d02da0f6553a00da6b0d833b67d3bbe87341e0/services/core/java/com/android/server/locksettings/LockSettingsService.java#L392-L397

Looks like the managed profile is using a randomly generated key, which is presumably stored somewhere?

Edit: https://github.com/aosp-mirror/platform_frameworks_base/blob/c5d02da0f6553a00da6b0d833b67d3bbe87341e0/services/core/java/com/android/server/locksettings/LockSettingsStorage.java#L449-L450

Seems to be in /data/system/.... Guess since I wiped it my data is definitely gone then. RIP.

[yonderbread]

So what exactly would be my decryption password for users 10 and 11 if my main user 0 uses a pattern to unlock?

[biboon]

Hello! Was there any progression on this?
I just had a bad update and I'm desperately trying to backup/access my Work profile (Shelter).

Seems to be in /data/system/.... Guess since I wiped it my data is definitely gone then. RIP.

I dug down to a file gatekeeper.profile.key which may be helpful but it's a binary file. Is there a way to manually mount the partition through ADB using this key file? Like TWRP would do?

Edit: formatting

[Atemu]

IME, if your work pattern is the same as your user pattern, it decrypts both.

[biboon]

Well unfortunately it seems like it does not in my case. I can decrypt user 0 with my pin code, but decrypting user 11 fails. I have a somewhat "particular" work profile though as it was actually set up by the app Shelter.

[Atemu]

@biboon mine was also set up by shelter.

What I missed is that I ticked the "Use one lock" setting under security.

[Craftplacer]

I'll share the knowledge I've gained here:

@yshui's answer is correct, Android does generate a password when "unifying" the work profile with the main user. It also generates a secret key which it uses to encrypt the password. The encrypted password is being saved to /data/system/users/{work profile user id}/gatekeeper.profile.key afaik. Android dumps the secret key into the KeyStore with alias profile_key_name_decrypt_{work profile user id}.

While I was successful finding the blob associated to the key inside /data/misc/keystore/persistent.sqlite, I have also noticed that it had a pKMblob header with a trailing zero, indicating that it was hardware generated.

So I'm not sure how to proceed from here, if I happen to get custom code to interface with my device, I might be able to extract key.