Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option for Signal install location (internal/external memory) #9015

Closed
wants to merge 1 commit into from
Closed

Option for Signal install location (internal/external memory) #9015

wants to merge 1 commit into from

Conversation

ssgromov
Copy link

@ssgromov ssgromov commented Sep 3, 2019

First time contributor checklist

Contributor checklist


Description

Following

https://developer.android.com/guide/topics/data/install-location.html

as originally pointed out at
https://community.signalusers.org/t/allow-installation-on-external-storage-location-sd-card/2638

Added option to allow the system to install Signal on the external storage.

Unfortunately, this feature request was closed automatically already twice.

My Signal database has reached some >7Gb size after which it is not possible to update the OS & apps without a

backup Signal to SD -> uninstall Signal -> update -> install Signal again -> restore from SD backup

sequence, which is very annoying.

I am aware of the issue
https://github.com/signalapp/Signal-Android/wiki/Data-loss-when-using-an-SD-Card-formatted-as-internal-memory
which I guess is likely caused by the official update overwriting the install option.

@alan-signal
Copy link
Contributor

alan-signal commented Sep 3, 2019

According to the docs

When your application is installed on the external storage:

The .apk file is saved on the external storage, but all private user data, databases, optimized .dex files, and extracted native code are saved on the internal device memory.

This suggests hardly anything will be stored on the external storage. Have you tested this PR and have evidence that contradicts that documentation?

Signal also falls under the "Applications that should NOT install on external storage" section of the docs as it has several Services.

@ssgromov
Copy link
Author

@alan-signal

The Android documentation is not exhaustive about what 'all private user data' etc. are.

Furthermore, there are two options, namely SD card is used as (1) external or (2) internal storage. I am talking right now only about Android 9 now, I haven't tested any other OS.

Whilst option (1) may imply (I haven't tested this) that private data is not movable, option (2) allows moving private data to the internal storage on the SD (I have tested this using this workaround).

There is a glitch, however: Signal must be freshly installed and moved to the SD space using the system interface before setting up the account. Otherwise, a migration of files (of an already existing account) offered by Android completes, but Signal stops recognising media that were received before the move.

And there is a workaround, too: I just made a backup to SD, uninstalled Signal, installed fresh, moved it to SD space, and then restored from the backup. That works fine for already a week.

To recap,

  • On Android 9 one can use the manifest option proposed here without changes to the code (via an override in developer options)
  • Migration cannot be done on the already set-up account, but via backup
  • Only "internal storage" SD use was tested, how this works with "external storage" SD is not known

=> I guess this request may be closed without merging.

@TheStealthReporter
Copy link

I've tested this change for several months now (using a local fork, even before this PR) on a Moto G5 first with Android 8.1 (Stock ROM) and then Android 9 (LineageOS) and an "internal storage" SD card. It worked flawlessly the entire time. I can confirm the entire data is moved, not just the .apk file.

The Moto G5 Stock ROM only allows moving data to SD card if it is mounted as "internal storage" (due to security/encryption) and if android:installLocation="auto" is set. Forcing a move to SD is not possible if one of these conditions is not satisfied.

Formatting an SD card as "internal storage" also means that the SD is almost never removed (since all data is encrypted, it is useless without this specific phone or reformatting it and loosing all data). The restarts of services should not be a practical concern in this case. It also seems like the actual behavior differs from the android documentation a bit (exact semantics might to have changed over the years).

I will test the restart of the services in a few days with following procedure (is this procedure meaningful?):

  • unmounting the SD
  • mounting the SD
  • not starting Signal
  • let somebody else send me a message
  • see what happens and report back

If there's anything else that has to be tested, I'm happy to do so. I'm not that familiar with android development, with the codebase or what services are used.

As this is a deal-breaker for me because of limited real internal storage, I really hope this PR gets merged. This is such a simple fix for a huge problem.

@FedericoCeratto
Copy link

Related to #4065 #7792 #7881 #7521 #4297 #9126 #8417 #8684 #8099

@FedericoCeratto
Copy link

Can we please merge the PR? Signal eats internal storage which can be very limited, making the application unusable on old phones.

@sovamind
Copy link

I'd like to also ask that the storage thing get fixed. I've got a Samsung Galaxy S7 which has 32GB of internal memory but a 400GB SD. It's sort of ridiculous that with Signal installed I keep running out of space and getting messages from Android that I need to free up storage immediately.

@ssgromov
Copy link
Author

I'd like to also ask that the storage thing get fixed. I've got a Samsung Galaxy S7 which has 32GB of internal memory but a 400GB SD. It's sort of ridiculous that with Signal installed I keep running out of space and getting messages from Android that I need to free up storage immediately.

@sovamind
if it is possible on your phone, you can override and enable the option to install to the external space (see my post above).

@klues
Copy link

klues commented May 22, 2020

On my phone I'm using the SD-card as external storage, so the suggestion of @ssgromov doesn't help me. Signal is using so much space on my phone that I'm about to being forced to uninstall it. However I've convinced so many of my friends to use Signal, so it would be really embarrassing to be forced to stop using Signal because of this little storage issue.

Please find a solution for saving data to external storage, I don't want to stop using Signal.

@LeeBinder
Copy link

@ssgromov , can you please detail how you backed up Signal, and how you restored your backup?

@ssgromov
Copy link
Author

@ssgromov , can you please detail how you backed up Signal, and how you restored your backup?

detailed in my post above:

And there is a workaround, too: I just made a backup to SD, uninstalled Signal, installed fresh, moved it to SD space, and then restored from the backup. That works fine for already a week.

Here's, however, again what I did :

  • make a backup (e.g. to SD or internal memory, whatever), make sure the backup will be seen by newly installed Signal
  • uninstall Signal
  • enable "Force allow apps on external" setting in Android developer options (it's there since v. 9 only, sorry)
  • install Signal afresh
  • move Signal to SD space (using Android system dialogue)
  • start Signal and restore from backup

Voila, it should work.

Please remember that I only tested it in situation when the SD card is used as internal storage extension (not as "portable", i.e. normal" SD card that can be plugged out and, e.g., be read in PC).

@LeeBinder
Copy link

Thank you @ssgromov for the detailed steps! The part I was missing was that Signal has a built-in backup, hidden under ⋮> Settings > Chats & media > Backups. For my logic that's a tool and should be right in the⋮menu ..

BTW, the "Force allow apps on external" setting in Android dev options is present in my Android 7.1.1.

Also complete uninstall of the app might not be necessary, just deletion of all its data, because the Android system dialogue only moves the data not the entire app.

@LeeBinder
Copy link

LeeBinder commented Jun 18, 2020

@ssgromov NO success here, and it might not have anything to do with that..:

  • I have Android 7.1.1
  • I left my SD formatted with stock FAT32
  • I did NOT uninstall Signal

Details:

  • enabled "Force allow apps on external" setting in Android dev options
  • ran Signal's built-in backup, hidden under ⋮> Settings > Chats & media > Backups
    (as a side note I OCR'ed this 30 digit backup passphrase and copied it into the clipboard..)
  • via Android App Manager, removed all Signal data, then changed the app's storage location from internal storage to SD card
  • to be on the safe side, rebooted my phone
  • re-opened Signal, pasted the 30 digit passcode, re-verified with SMS code and PIN

Some minutes later all my messages are back, and on my SD card I now see a hidden (and - from what I've read - encrypted) file under:

.android_secure/org.thoughtcrime.securesms-2.asec

It is only 59MB big which means it can NOT house all my Signal's data.. which my root file manager, browsing through that file's mount point, confirms:

mnt/asec/org.thoughtcrime.securesms-2.asec

only contains the base.apk, the lib folder and an empty 'lost+found' folder..

When I check with one of my root apps, I see that Signal's data path is still internal as before:

/data/user/0/org.thoughtcrime.securesms-2.asec aka /data/data/org.thoughtcrime.securesms (two path descriptions to same folder)

but nowhere in that folder or its subfolders I can find any images or videos, and the folder on my SD card android/data/org.thoughtcrime.securesms only contains two empty folders 'files' and 'cache'. Huh?

On further research¹ I found out that the folder which stores all media (with mostly untypical file extensions) is

/data/data/org.thoughtcrime.securesms/app_parts (encrypted .mms files)
   and possibly also
/data/data/org.thoughtcrime.securesms/app_captures (.jpg, .blob files)

So to verify that a attempted "data" move was really successful even if you have formatted your SD card as 'internal' (Linux FS), you have to go to /data/data/org.thoughtcrime.securesms/ and check if the folders app_parts and app_captures still exist, and if yes, if they still contain any big files, or not.
 ________________________________________________
¹
   • Signal uses too much data on storage / can't clear data with signal | #5515-312254344
   • Signal Android uses way too much internal storage | #9033-593057330

Also see (there is waayy more users like us who need this bug fixed!!):

@LeeBinder
Copy link

LeeBinder commented Jun 19, 2020

@ssgromov : unfortunately I had to edit my posting above.. if hope you have a moment to re-read it.

But before and apart from that, could you (and anybody else if you feel supportive) please share another little detail which I did not find in your comments:

Where after "moving to SD card" does your Signal app now store its "big" media data - videos, audio, images etc?

Can you please fire up your fav file manager, look for that folder on your SD card (data/data/org.thoughtcrime.securesms? android/data/org.thoughtcrime.securesms?), and inside of it look for the app_parts subfolder (with .mms files) and possibly the app_captures (.jpg, .blob files) subfolder, and share the path to and the size of the main org.thoughtcrime.securesms folder? Thank you.

@LeeBinder
Copy link

hello @ssgromov . On your internal storage, can you please go to (system root) /data/data/org.thoughtcrime.securesms/ and check if the folders app_parts and app_captures still exist, and if yes, if they still contain any big files? How big in size is your (system root) /data/data/org.thoughtcrime.securesms/ now after moving your Signal to SD card?

@LeeBinder
Copy link

LeeBinder commented Jun 20, 2020

Instructions to have Signal Data on regular external storage SD formatted as stock FAT32

!! Requirement: root via Magisk !!

Tested in Android 7.1.1 on a Samsung phone with a TouchWiz ROM

[EDIT]: not working anymore in Android 11, both for me and somebody else (ulph0, see below).

  1. root your Android by installing Magisk and Magisk Manager.
    NOTE: I cannot give any instructions here how you do this for your specific phone!
    DISCLAIMER: you do this completely at your own risk!!!!
  2. Open Magisk Manager > Settings > Mount Namespace Mode: toggle to “Global Namespace”. That will enable you to create lasting Symlinks even on your external storage FAT32 SD which should also be readable by Signal!
  3. Reboot
  4. Install FX File Explorer (free) and open it
    (Note: other root and Symlink capable file managers might work, too, or might not)
  5. In FX File Explorer’s home screen, tap onto the green button with a white # labeled “System (Root)”. Navigate to /storage > tap into your external SD card’s mount point (in my case that’s 0000-0000) > create a folder (structure) of your choice for Signal’s data. To reflect the path on the internal system, I created a directory path with three new folders analog to the same structure as on internal: data/data/org.thoughtcrime.securesms. If you prefer things simple, you could also just create 1 folder labeled e.g. “Signal Data” - that’s entirely up to you.
    !! ATTENTION: it is paramount you do this as “System (Root)” - do NOT tap onto the entry for your SD card on FX’s home screen!!!
  6. For quicker navigation, create a shortcut to this folder to FX’s home screen: ⋮> bookmark icon in the top center > label it "Signal Data on SD"
  7. Go back to FX’s home screen (back button on your phone or Home icon in FX’s menu bar’s very left) > again tap onto the green button with a white # labeled “System (Root)” > Navigate to /data/data/org.thoughtcrime.securesms
  8. Again for quicker navigation, create another shortcut to this folder to FX’s home screen: ⋮> bookmark icon in the top center > label it "Signal Data on Internal"
  9. Prepare to copy the folder app_parts: long tap it > tap onto “Copy” in the top bar. You will see a little white clipboard indicator in the top right.
  10. Go back to FX’s home screen > tap on the Icon you labeled “Signal Data on SD” which should take you to external storage data/data/org.thoughtcrime.securesms
  11. Paste the clipboard content in there which should copy the app_parts folder into there
  12. Long tap onto the app_parts folder, ⋮ > Symlink
  13. In the little pop-up window, tap onto the path displayed under “Symlink Directory” > swipe the top bar until you see the # > tap onto it > navigate to /data/data/org.thoughtcrime.securesms (FX’s home screen with the bookmarks can’t be accessed here) > OK (leave the Symlink Name as app_parts(1) > OK > OK
  14. Go back to FX’s home screen > tap on the Icon you labeled “Signal Data on Internal” > you should see your newly created Symlink app_parts(1) > rename the folder app_parts to e.g. app_parts_ > rename the Symlink app_parts(1) to app_parts
  15. Open Signal > open a conversation with some image and tap onto it > it should open maximized
  16. Navigate out of Signal. Recommended: force quit it via Android
  17. Go back into FX > home screen > “Signal Data on SD” > rename app_parts to e.g. app_parts_
  18. Re-open Signal > re-open the conversation with the image and tap onto it > it should NOT open!
  19. Go back into FX > “Signal Data on SD” (should still be open) > rename app_parts_ to app_parts
  20. Go back into Signal, tap onto the same image preview > it should again open maximized!!

If so you have done everything correctly AND it’s working as supposed on your phone.

  1. Go back into FX > home screen > “Signal Data on internal” > delete the folder app_parts_ CAREFUL: do NOT remove the Symlink app_parts .. ;)

  2. If you also have a folder app_captures in Signal Data on internal, repeat the steps for that folder!

(23. OPTIONAL AND NOT TESTED: the Global Namespace settings and the Symlinks might stick even if you uninstall Magisk again which will also unroot your Android. If you prefer not be rooted you are welcome to test this and report back)

DISCLAIMER: that’s all I can do for you. Either this works on your phone, or it doesn’t.

@ssgromov
Copy link
Author

Hi @LeeBinder ,

sorry for taking long to answer (I see a lot of updates in the meanwhile though).
Nice to see you have succeeded.

P.S. Perhaps, uninstalling Signal helped me (as it is supposed to wipe everything), then the files you've asked about (e.g. app_parts) might have landed in the right paths after moving it and restoring. Who knows... FYI, I have a not rooted phone and I do not see these anyway.

Cheers

@LeeBinder
Copy link

@ssgromov Gotcha.

Hopefully one fine day the coders will give us an easy option to choose data storage.

@ssgromov
Copy link
Author

ssgromov commented Jul 9, 2020

Hopefully one fine day the coders will give us an easy option to choose data storage.

Yes, @LeeBinder , and I think I am also missing an option of reducing picture sizes... we do not need to send 12Mpixel pictures all the time... that would also save a LOT of space. Gotta check the issues list... maybe someone proposed that already.

@ThemGo
Copy link

ThemGo commented Aug 10, 2020

@LeeBinder
Thank you very much for sharing your approach!

This is how it works with a rooted phone with SuperSU - tested in Android 6.0.1 on Sony Z3:

  1. Your phone is rooted and has SuperSU app
  2. Open SuperSU app: deactivate "Settings/Security/Mount namespace separation" resp. "Einstellungen/Sicherheit/Namensbereichsabtrenung mounten"
    Remaining steps are the same... I copied and symlinked directory /data/data/org.thoughtcrime.securesms/app_parts to /storage/<0000-0000>/datadata/signal-app_parts

Note to 4.: You will also have to install Root Add-on for FX File Explorer, otherwise the green button with a white # labeled “System (Root)” will be missing. Not necessary though to activate Pro version test phase.

It seems to be crucial to symlink only a sub directory, I tried with another app to symlink its whole /data/data/xxx directory and the app was broken after restart.
BTW app SD Maid is very useful for me to identify more crowded directories worth to be symlinked ("Storage analyzer" resp. "Speicheranalyse"). Signal was on first place - by a wide margin!

@LeeBinder
Copy link

LeeBinder commented Aug 11, 2020

hey @ThemGo , thanks for the hints with the Root Add-on for FX File Explorer and SD Maid! Glad my share helped you. BTW, re. the Root Add-on for FX File Explorer:

*** THIS ADD ON IS NO LONGER REQUIRED WITH MODERN VERSIONS OF FX FILE EXPLORER ***

(You can now enable root access directly within FX, there is no need to install this add-on. This add-on remains available only for users of old versions and very old devices. Download the latest FX here: https://play.google.com/store/apps/details?id=nextapp.fx )

@michaelzangl
Copy link

@ssgromov How did you manage to do a Backup on the SD card?

I am currently running in the same problem: Device space is running out, with signal being by far the biggest app in there. Even when removing all other apps, I would not have enough space for a backup file of signal. Signal always tries to create that file in internal storage. I cannot migrate to an other device, I cannot remove and re-install signal.

@LeeBinder
Copy link

LeeBinder commented Aug 23, 2020

@michaelzangl : https://github.com/johanw666/Signal-Android/releases

..provided you don't mind the 666..

@l3u
Copy link

l3u commented Sep 10, 2020

Please make it possible to at least have the chat data on SD storage. I really don't mind Signal to be in the phone's memory, but the data grows to several GB in no time. This is really a problem, esp. on older phones. Along with the fact that a backup can't be placed on SD storage, too …

@l3u l3u mentioned this pull request Sep 10, 2020
4 tasks
@Tmpod
Copy link

Tmpod commented Nov 4, 2020

@michaelzangl : https://github.com/johanw666/Signal-Android/releases

..provided you don't mind the 666..

Can this fork be installed over an existing stock Signal installation as an update? I want to backup my Signal data so I can upgrade my OS, but my database is ~8GB and looking at this comment, I simply have no way of doing it except to chuck it onto my external SD card.
It it were possible I would get the apk, update Signal with it and perform the backup.

@grandchild
Copy link

grandchild commented Nov 4, 2020

@Tmpod it might -- provided you are confident with the adb commandline tool. Don't attempt any overwrite install unless you know what that means. You might well lose your data.

Also, looking at that fork's storage code, I am not sure it will detect SD cards on all phones, since it relies on calling getExternalFilesDirs, which doesn't always return the actual SD card paths, but the internal storage again. So I wouldn't be too sure that it will work.

Do you have an option to upgrade to Android10 on the existing phone? It seems that for Android 10 you can now select the backup location.

@Tmpod
Copy link

Tmpod commented Nov 4, 2020

@Tmpod it might -- provided you are confident with the adb commandline tool. Don't attempt any overwrite install unless you know what that means. You might well lose your data.
Also, looking at that fork's storage code, I am not sure it will detect SD cards on all phones, since it relies on calling getExternalFilesDirs, which doesn't always return the actual SD card paths, but the internal storage again. So I wouldn't be too sure that it will work.

mmmm, I'm pretty sure I don't want to risk it then. But then again, I have no option for the backup, it simply would be too big... This backup is preventing me from upgrading my phone (still on Nougat).

Do you have an option to upgrade to Android10 on the existing phone? It seems that for Android 10 you can now select the backup location.

I am using Lineage and would like to continue doing so, but my phone (Redmi 4X, codename santoni) doesn't seem to have Lineage 17 yet (might not even get it at all, it's not a super recent model), only 16 which is Android 9.

@stale
Copy link

stale bot commented Jan 26, 2022

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@stale stale bot added the wontfix label Jan 26, 2022
@klues
Copy link

klues commented Jan 26, 2022

Yes, dear stale bot, this is still very much relevant! And much more also for Signal people like @greyson-signal @alan-signal @moxie-signal
One of the main tasks I'm currently doing on other peoples phones is to sort out and delete Signal media data because people are complaining "My phone says I don't have memory anymore!". And I have to explain to them why it's not possible to save Signal data on SD card (which I can't). This is really embarrassing and keeping me off from advertising Signal to other people, which I've done very much in the past.

@stale stale bot removed the wontfix label Jan 26, 2022
@LeeBinder
Copy link

I 100% agree with @klues Other apps can. This is NOT a biggie at all. 1422 open issues just for the Android version. Signal devs, if you don't even know how to code simple stuff, then how do you want to proceed..

@l3u
Copy link

l3u commented Jan 26, 2022

I'm pretty sure insults won't motivate the devs very much to work on this. This is for sure still a problem, esp. for older phones. But since Signal started to auto-compresses images, the amount of space used dropped significantly (in my case from about 3.5 GB to about 800 MB, keeping 6 months of messages), and thus, this problem leveled off a bit. Still, this should be kept on the to-do list.

@LeeBinder
Copy link

Even stating the obvious truth doesn't matter anymore at this point in time of this pitiful and unnecessary saga. Not knowing what I'm talking about? Then read my instructions..

@stale
Copy link

stale bot commented Mar 31, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix label Mar 31, 2022
@michaelzangl
Copy link

Keep it open

@stale stale bot removed the wontfix label Mar 31, 2022
@tl072
Copy link

tl072 commented Apr 2, 2022

I would agree about keeping this issue open. Android phones are a pain for a variety of reasons and having some flexibility for this will make them less painful.

@LeeBinder
Copy link

LeeBinder commented Apr 17, 2022

My instructions are not working anymore for me on my new Nokia 6.1 in LineageOS 18.1 (Android 11) with SD formatted as Internal Storage, aka "Adoptable Storage", above all for the sake of encryption.

[EDIT]: not working for me with SD formatted now as Mobile Storage, either.

For comparison, WhatsApp data is being written to the merged/ fused Adoptable Storage by default. That can be achieved by following fredsky2's instructions. The key is

pm grant com.whatsapp android.permission.WRITE_EXTERNAL_STORAGE

easiest via Terminal from one's computer via adb shell.

With Signal, the corresponding command

pm grant org.thoughtcrime.securesms android.permission.WRITE_EXTERNAL_STORAGE

does not have any effect because (as per error thrown),

Package org.thoughtcrime.securesms has not requested permission android.permission.WRITE_EXTERNAL_STORAGE

So this issue could be solved if Signal did ask for permission to write to external storage. The default data path can still be left as /data/data/org.thoughtcrime.securesms. But then at least those who want to change that path, would finally have the opportunity to do so. An option, rather than a doctrine.

Devs @alan-signal @alex-signal @greyson-signal, would you please at least consider, and share your ideas how this can in some way be moved forward with as little effort for you as possible?
Thank you as always

@ehrenkret-signal ehrenkret-signal deleted the branch signalapp:master April 29, 2022 21:09
@h0jeZvgoxFepBQ2C
Copy link

Please provide a way to store the backups to the SD card, it's eating all my disk space on my internal storage any my phone stops working because of it!

@ulph0
Copy link

ulph0 commented Sep 3, 2022

@LeeBinder Hi thanks yor your post above from 2 years ago. My mobile ran full lately - i don't want to delete old messages or buy a new mobile, so i tried it. Didn't work for me. In step 15 i see no images. I don't have adopable storage, but i have Android 11 :'( I am wondering that there is no way to force granting unrequested access to external storage on a rooted phone. even when signal does backups to the external storage now (for which it needs access to external storage?!)

@LeeBinder
Copy link

@ulph0 not working for me with SD formatted now as Mobile Storage, either. From what I know, that's most likely due to Android 11's scoped storage. Yes, Signal CAN access SD for backup, but scoped storage does NOT allow symlinking anymore as described above because folders in Data/Data/.. are pretty much each in their respective dedicated sandbox, so to speak. And the Signal dev team is not willing to change the path, claiming security concerns, from what I understand. What to say....

@Arkoprovo1996
Copy link

It can be a security concern, but I think there's no point saying security when it interferes with the usability of the app, right? As in if people can't use the app because it takes up space on their device, then they'll just uninstall it and not be secure anyways. I'm on Android 11 (dotOS) & apps can request access to do this. Organic Maps, Google Maps, Telegram, all do this. So, yea.

@h0jeZvgoxFepBQ2C
Copy link

/reopen

yeah please fix this, this is really shitty.

@ulph0
Copy link

ulph0 commented Dec 19, 2023

it is really sad that this still isn't possible. i want to be free to decide where my data is stored. i want to access all my messages, even with small system memory. why is this closed?

@LeeBinder
Copy link

see #9015 (comment). The beauty of open source.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet