Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow exporting a backup from Signal Desktop #522

Closed
strugee opened this issue Dec 16, 2015 · 125 comments
Closed

Allow exporting a backup from Signal Desktop #522

strugee opened this issue Dec 16, 2015 · 125 comments

Comments

@strugee
Copy link
Contributor

strugee commented Dec 16, 2015

It would be great if you could export backups from Signal Desktop in the same way you can export them from Signal for Android. I've just lost a huge amount of conversations and private keys because of signalapp/Signal-Android#4577, and I'd like to at least partially restore my data from my desktop install.

@ThoughtPhotography
Copy link

Would really like this too. My phone died, meaning all my messages will be lost even though I can see them in Signal Desktop still...

@pabloferz
Copy link

pabloferz commented Sep 20, 2016

Yeah, this would really be a life saver.

@nomeata
Copy link

nomeata commented Oct 25, 2016

I have systematic backups of my messages in Facebook, jabber, ICQ, Telegram. A good backup of my signal messages (including pictures) is direly needed!

@strugee
Copy link
Contributor Author

strugee commented Oct 25, 2016

@nomeata please don't bump issues.

Besides, you can already backup Signal messages from mobile. At least, you can in the Android app. It doesn't include pictures but I believe someone's working on that too.

@iflowfor8hours
Copy link

Actually, this depends on your use case for signal desktop. If you swap phones often (different SIMs for different countries), then your phone is the only source of truth for your message history, as signal desktop blows away your history every time you re-register a device. Being able to make a backup and restore from the desktop client would mitigate that when you return from travel. I would find it really useful, as I found it surprising the desktop deleted my history the first time I did that.

@ThoughtPhotography
Copy link

The reality is mobile devices are lost, stolen, break etc. far more often than desktop computers which means messages will be lost, but remain on the desktop version. Not being able to export/backup from Signal Desktop means it's impossible to get them back in the mobile version.

Aside from exporting and being able to merge backups from the mobile/desktop versions, a way around this would be an option to auto-backup the encrypted message database to a cloud account or send to an email account. That way messages can be recovered when a device is lost etc.

@iflowfor8hours
Copy link

iflowfor8hours commented Nov 3, 2016

This is my point exactly. I lose and change devices constantly, and would like to have a canonical source of truth of messaging, which lives on my desktop. In the current case, I can't have that without backing up my device regularly and then syncing to a cloud service somewhere and using that as my history. I think we're in agreement here.
How can I help?

@nomeata
Copy link

nomeata commented Dec 27, 2016

I have started implementing a feature for the Desktop client that would create a zip file consisting of all messages of (for now one) conversation, together with all media files. Let’s see how far I get (tomorrow).

nomeata added a commit to nomeata/Signal-Desktop that referenced this issue Dec 28, 2016
=== What works:

Exporting text and media files of the currently visible conversation to
a zip file containing an HTML file with the text, including links to the
media files, as well as the media files. The Signal Desktop CSS file is
included to achieve the same styling as in the app (but without any
controls, which do not make sense in a static settings).

Yay!

=== What does not work:

It does not yet load all messages back to the beginning of history.

The fonts are not included in the zip file, for even more proper
styling.

The conversation avatar (if present) is not included.

The code is far from being in good shape.

=== Note to reviewers

I know that the code is not a shape to be merged, but it works for me.
I am interested in cleaning it up to be mergable if the general
functionaliy is desired (i.e. I will not fix code cosmetics if in the
end the pull request is reject because the feature is not wanted.)
@nomeata
Copy link

nomeata commented Dec 28, 2016

I noticed that this is issue more about an data export (that can be imported) than a archive export (which is meant to be readable independent of Signal), so I created a separate issue for that at #1013.

@jessebett
Copy link

In issue #1017 it is mentioned that Signal Desktop stores unencrypted messages locally. Is this true? Does anyone know where these messages are stored?

My situation is as described by @ThoughtPhotography, my phone broke irreparably and I wasn't able to backup my conversations. I still have my Desktop client, with all the history. I would like to be able to download and save all my messages before I try installing signal on my new phone and linking desktop. It looks as though I'll have to uninstall/reinstall desktop to sync new phone's Signal, and I doubt my history will carry over.

@strugee
Copy link
Contributor Author

strugee commented Feb 17, 2017

@jessebett you don't have to reinstall. You can relink Signal Desktop in its settings and your history will stay.

@jessebett
Copy link

@strugee I don't want to stray too far from the thread issue, but can I relink in Desktop if I wasn't able to unlink first from Mobile? I don't see any option for that in my Desktop settings.

@strugee
Copy link
Contributor Author

strugee commented Feb 17, 2017

@jessebett yes, you can.

@nomeata
Copy link

nomeata commented Feb 17, 2017

In issue #1017 it is mentioned that Signal Desktop stores unencrypted messages locally. Is this true? >Does anyone know where these messages are stored?

Stored unencryptedly in Chrome’s IndexdDB.

@nomeata
Copy link

nomeata commented May 6, 2017

I worked some more on the export. The export-to-zip worked nicely, until my history became too large. Maybe some kind of 100MB problem, maybe a bug in zip.js.

So I refactored the code and it now supports either exporting a zip file (with the chat history and all media), or simply to an empty directory.

The code is, in “works for me state” at the GitHub clone at https://github.com/nomeata/Signal-Desktop/. I am still willing to clean this up into a mergable state, if the feature itself is deemed desirable.

@nomeata
Copy link

nomeata commented Aug 23, 2017 via email

@nomeata
Copy link

nomeata commented Aug 24, 2017

Well, how do I switch from the official version to yours?

I go to $HOME/.config/google-chrome/Default/Extensions/idof theapp/0.42.6_0 and apply the patch, as produce by git diff, using patch, and then restart the browser. It does not quite apply the translations properly, to the button to do the backup is unlabeled, but otherwise it works for me.

(I only recommend following this if you know git, patch and chrome apps in general well enough to understand the above procedure. And backup everything first properly, of course.)

@nomeata
Copy link

nomeata commented Dec 17, 2017

The “migration to signal-desktop” offered by the Chrome app does precisely what I need. Can we get that functionality also in the new stand-alone app?

@scottnonnenberg
Copy link
Contributor

@nomeata Would love to know a little bit more about your scenario. The current export functionality copies the entire app, including all encryption information. It's dangerous, because if you export, then continue using the original app as well as the new, copied app, you'll run into lots of errors.

@nomeata
Copy link

nomeata commented Dec 18, 2017

The scenario is archival. I want to get all data out in a format that I can, hopefully, read in 20 years even without the Signal software around. Maybe “backup” is the wrong word, as I do not want to restore it into Signal.

@nomeata
Copy link

nomeata commented Dec 19, 2017

Ah, as expected: The backup functionality is present in the the new stand-alone program, just not exposed via the UI. But Ctrl-Shift-I and

Whisper.Backup.exportToDirectory("/tmp")

does precisely what I need.

This is good enough for me, and if the Signal devs have good reasons not to expose that functionality in the UI, then I will not argue and happily use the developer console for my backups.

Tip: Run

for file in /path/to/the/backup/**/*.json ; do json_pp < "$file" | sponge "$file"; done

for prettier json files and (if you keep the backups in a git repo) better diffs.

@devurandom
Copy link

@nomeata Thanks! This is really useful, e.g. in case the conversation needs to be archived or be made available to other parties.

@scottnonnenberg
Copy link
Contributor

Alright, you're getting fancy. :0) I would recommend that you delete the root-level db.json, since it has sensitive encryption data in it. Also, you should know that the conversation directory names start with a timestamp representing the last activity of that contact - so your git diffs will be pretty substantial.

@nomeata
Copy link

nomeata commented Dec 19, 2017

Thanks for the heads-up! The latter can be fixed with

rename 's/^.* \((.*)\)/$1/' *

@501st-alpha1
Copy link

On my Debian GNU/Linux system, I got this when I ran that rename command:

find: warning: Unix filenames usually don't contain slashes (though pathnames do).  That means that '-name `*.s/^.* \\((.*)\\)/$1/'' will probably evaluate to false all the time on this system.  You might find the '-wholename' test more useful, or perhaps '-samefile'.  Alternatively, if you are using GNU grep, you could use 'find ... -print0 | grep -FzZ `*.s/^.* \\((.*)\\)/$1/''.
Files successfully renamed!

Despite the last line, the files weren't renamed. I couldn't figure out how to fix the rename command, so I did this instead:

ls | while read file; do new=$(echo $file | sed -e 's/^.* (\(.*\))$/\1/'); mv "$file" "$new"; done

and that worked.

@breznak
Copy link

breznak commented Feb 6, 2018

There is great news on the front of backups/restore on Android in signalapp/Signal-Android#7380
@Roghetti would you please consider also spinning a PR for the desktop version here? It would solve the problem of backups here and allow for restoring between ones PC-Android (ideally, if the PR can be changed to as much platform agnostic as possible- which it quite seems, only the GUI parts are "android-only"- the we could have all Android/iOS/PC backup&restore)

@rowanthorpe
Copy link

rowanthorpe commented Aug 31, 2021

For a while I've been using a tool I wrote to decrypt and export the sqlite/sqlcipher data from signal-desktop's db to a plaintext sql dump (which can be read as-is, or slurped into an unecrypted sqlite db for browsing, or into a sql-cipher db for importing back into Signal). Seeing this thread motivated me to put the (very small) tool in a repo and link to it here. Importantly I found that:

  • I had to compile and install a latest upstream source version of sqlcipher (even Debian Sid still provides v3 but this task requires >=v4)
  • I had to manually get the 16-byte salt from the beginning of the db-file and append that (32 chars) to the 32-byte key (64 chars) found in the config file. Any of the other suggested ways to achieve compatibility seemed to not work for me.

@ParisKellySkopelitis
Copy link

@rowanthorpe I was working on a similar tool as well! Didn't get that far though haha. In theory this would allow for iOS to android backups (via signal desktop of course).

@rowanthorpe
Copy link

@ParisKellySkopelitis I was also thinking that probably the database is the same across Desktop/Android/iOS (sqlite with sqlcipher extension) so I guess this tool should work on those too. Anyone wanting to try would need to just solve getting access to the db-file and the config-file on those platforms, and anything beyond that is just parsing and post-processing to present the data differently for people who don't know how (or just don't want) to deal with their data in SQL form. One post-processing function I would like to see is for deduplicating data while merging backups taken from Signal linked on the desktop and on the phone, so that regular backups can be taken from both without having to do the mental gymnastics to avoid overlaps or remember what was backed up where, and when...

@ankostis
Copy link

ankostis commented Sep 2, 2021

FYI, there is this module for scrapping signal-desktop's DB for URLs, part of the bigger promnesia project, which contains an SQL-query that presents Signal discussions in a sensible way adaptable to exporting needs:
https://github.com/karlicoss/promnesia/blob/master/src/promnesia/sources/signal.py

@oomek
Copy link

oomek commented Sep 9, 2021

I was forced to factory reset my phone, so I couldn't make a backup. I have all my conversations in my desktop app. The mobile app warns me that it will wipe all connected devices clean. How can I keep my messages history and still being able to use my mobile app?

Update: Since my phone number was the same the messages stayed in place once I re-linked my desktop app. All good.

@tbvdm
Copy link

tbvdm commented Sep 10, 2021

I have written sigtop which allows you to export messages, attachments and the SQLite database itself from Signal Desktop. Maybe it's useful to some.

@loxK
Copy link

loxK commented Sep 29, 2021

@tbvdm can it be imported in the mobile client ?

@tbvdm
Copy link

tbvdm commented Sep 30, 2021 via email

@v6
Copy link

v6 commented Feb 23, 2022

I'd like to see this, too, especially since "my phone dropped in the ocean" is a real thing.

@tbvdm I'll try out that sigtop utility.

@Timoses
Copy link

Timoses commented Apr 23, 2022

Well, @signal ... This is very very necessary.
I dropped my phone and it is dead now.

I will use sigtop (#522 (comment)) now to at least save my messages etc. (perhaps encrypt it myself then...).

@nettnikl
Copy link

@Timoses in case you find a solution for reimporting them to mobile, please share it, I'm sure it would help many!
(Including me, i relied on normal Android application data backups, which do work for anything but signal)

@Timoses
Copy link

Timoses commented Apr 26, 2022

@Timoses in case you find a solution for reimporting them to mobile, please share it, I'm sure it would help many! (Including me, i relied on normal Android application data backups, which do work for anything but signal)

@nettnikl I don't think import of the "backup" (more like an export) from desktop client via sigtop is possible.

@nettnikl
Copy link

@Timoses I see, sorry, i misunderstood your intention.

Well, i'll keep looking for a solution, not sure what information is missing in the data the desktop client is using in comparison to the data in the android client, but i might be able to fill in dummy data - as the interesting info is saved in desktop.

@mdczaplicki
Copy link

Bump. This is needed :)

@cloudgreen
Copy link

Bump. This is needed :)

I agree!

Chris Aderne's signal-export (https://github.com/carderne/signal-export), as mentioned above, looks very promising for exporting chats, but the Docker installation requirement (for Windows) has made me put it off for a bit... but maybe Docker installation isn't very time consuming, nor system invasive?

@shvchk
Copy link

shvchk commented May 21, 2022

Docker installation isn't very time consuming, nor system invasive

That's pretty much what Docker is about : )

@cloudgreen
Copy link

Docker installation isn't very time consuming, nor system invasive

That's pretty much what Docker is about : )

Haha, good point. :)

Docker requires enabling of the WSL 2 feature... but maybe WSL 2 doesn't add that much overhead to a Windows installation? (I like to keep my main Windows installation as lean and clean as possible.)

I really appreciate all the efforts for Signal export ❤, but it would be fantastic if we could get an even simpler method for mainstream usage.

@maltalex
Copy link

maltalex commented Aug 8, 2022

Feature requests seem to have moved to the Signal forum.
Specifically, this request can be found here - https://community.signalusers.org/t/message-history-sync-for-newly-linked-desktop-and-ipad-devices/2839

Edit: I don't like the move either, and would also want to see this feature implemented. Reacting with thumbs down because I'm pointing you to the right place to ask for this feature isn't going to help. Instead, click on the link and join the feature request in the right place.

@NiklasBr
Copy link

Data loss and unexpected behaviour is generally categorised as a bug and this issue was created well before that forum was opened. But I digress, Signal appears to never allow us to access our own data. 😞

@sorryusernameisalreadytaken

The ability to backup from signal desktop to mobile and vice versa is a very urgent task for Signal. That is my opinion.

Explicitly, also the possibility to combine backups. Why not, because you are in possession of the key....

@codethief
Copy link

Explicitly, also the possibility to combine backups.

@sorryusernameisalreadytaken While there's no official solution for merging backups (yet?), it can be done using https://github.com/bepaald/signalbackup-tools#merge

@btakita
Copy link

btakita commented Oct 6, 2023

We need a backup/restore solution that can merge multiple backups as well. I have a case where I have important history separated in multiple backups & would like to merge all of them together. Currently, I have to choose which important messages I want to keep & which to "archive". Switch between "archives" means uninstalling/reinstalling the signal app on the phone leading to more backup history fragmentation.

We need some sanity to this process...please.

@hl037
Copy link

hl037 commented Nov 17, 2023

Why whole circle quit signals just because of this non-sense. The phone should not be the true source ! Phones gets stolen, lost, and hard to get data out when it's dead (contrary to a pc where it's easy to un plug the hard drive and plug it to another). I'm coming again to this thread to see if now there is some solution, still not. So I will always say bad (true) things about signals : it's shit.

@lahma0
Copy link

lahma0 commented Jan 20, 2024

@hl037 Calling Signal shit, simply because it lacks a feature you want, is honestly ridiculous and extremely petty. Thousands, or even tens of thousands, of hours of development time have been put into this open source project, much of it by people who were offering up their time and knowledge without any compensation. Signal is a one-of-a-kind resource that has, in many cases, set the standard for how secure, encrypted messaging should be done. We should be offering our gratitude to all of the brilliant minds who put so much into this application (such as its original creator, Moxie Marlinspike), not criticizing it and calling it derogatory terms because it doesn't meet our expectations perfectly in every single way. While I would like to see the features discussed here as much as the next person, I think it's childish and immature to devolve into name calling and unproductive criticism. That's just my 2 cents though..

@MacDada
Copy link

MacDada commented Mar 1, 2024

automated-signal closed this as completed on Sep 30, 2021

Am I missing something? I don't see an option to export a backup from Signal on my Mac [?]

@mailinglists35
Copy link

automated-signal closed this as completed on Sep 30, 2021

Am I missing something? I don't see an option to export a backup from Signal on my Mac [?]

signal devs are unfriendly people who reject feature requests in their github repo, they only accept bug reports here, that's why they close feature requests issues. see #522 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests