Repository navigation
Error: Encrypted export/import is disabled #2516
Description
Activity
We'll re-enable it when it's ready for widespread use. I'm glad you were able to find a workaround for the near term.
I would like to state here that a signal-desktop backup/archive function would be much appreciated. Even if it remains console based.
@501st-alpha1 Could you elaborate on how to edit the js file to disable the error?
I have tried to comment out the above mentioned line in my local app.asar file which is under "C:\Users\'Username'\AppData\Local\Programs\signal-desktop\resources". However any change I make prevents Signal-desktop to open. I get a message "A javascript error occurred in the main process".
I'm on Windows 10.Reacted by Drag0nFly@betrieb Rather than edit the actual source code, I did I live edit from the debugger. In the console, I clicked the link near the error I mentioned, and it opened up the line in the
backup.jsfile I linked to. From there you can comment out thethrowline and save the file, and then the export will work.Thanks! It worked via the console edit. This is great
Reacted by Scott WeldonI agree on the possibility of moving Signal Desktop data to another machine.
It was possible via the Chrome extension and the irreversible migration option. But just one time.
Could not be possible to do it a second time now.@501st-alpha1 Thanks for pointing how to get past that line. But when I do that, I get an empty
attachementsfolder. That used to work.Is that the same for you?
Oh, and also many files have
messages = []when they didn’t before. It seems this is broken :-(this is a bummer. trying to upgrade (ios) phones and only now do i learn my messages may be truly ephemeral... anybody have the latest commands to hack around it being disabled? maybe some way to dump and decrypt the DB?
Yes, this doesn't seem to work – is there any workaround to export messages for backup? Even if there's no way to re-import it it would be nice to have the ability to archive.
Reacted by Scott Weldon and Kevin CWhere previously you needed to export your data to take it elsewhere (because it was in IndexedDB) you can now just take the database itself. It's SQLCipher, so you need to unlock it with a key, but it's a normal SQLite database after that. As of 1.17.x conversations are there, and since 1.15.x messages have been there. Since spring 2018 the
attachments.noindexfolder has had all the attachments.Reacted by Scott WeldonI'm using signal desktop 1.16.2 on Ubuntu 16.04 (4.15.0-36-generic #39~16.04.1-Ubuntu SMP). I'm using sqlcipher version 3.8.6 2014-08-15 from apt.
I've assumed the key mentioned is in
/home/`whoami`/.config/Signal/config.json
I've assumed the database mentioned is in/home/`whoami`/.config/Signal/sql/db.sqliteI've tried methods described
here
and here➜ signal sqlcipher db.sqlite SQLCipher version 3.8.6 2014-08-15 11:46:33 Enter ".help" for instructions Enter SQL statements terminated with a ";" sqlite> PRAGMA key = 'my_key_pasted_from_config.json'; sqlite> ATTACH DATABASE 'plaintext.db' AS plaintext KEY ''; Error: file is encrypted or is not a database``` sqlite> .backup plaintext.db Error: file is encrypted or is not a databaseI've also tried
DB Browser for SQLite(both the versions in the package manager, and the ones inppa:linuxgndu/sqlitebrowser, which from what I read, is supposed to be able to read sqlcipher encrypted databases now), and I get the messageCould not open database file. Reason: file is encrypted or is not a databaseI think I must be missing something simple.
Reacted by Scott WeldonYou're very close, the only thing you need to change is
PRAGMA key = 'my_key_pasted_from_config.json';For raw keys the format is (not my real key):
PRAGMA key = "x'2DD29CA851E7B56E4697B0E1F08507293D761A05CE4D1B628663F411A8086D99'";(otherwise it will assume it's a passphrase and do key derivation, as described here).
I was able to successfully dump out the DB as a plaintext SQLite database with this.
Reacted by Scott Weldon, Jules Omlor and Kostis Anagnostopoulos- Reacted by Scott Weldon
for those interested in exporting to CSV
➜ signal sqlite3 plaintext.db SQLite version 3.11.0 2016-02-15 17:29:24 Enter ".help" for usage hints. sqlite> .mode csv sqlite> .headers on sqlite> .out messages.csv sqlite> select * from messages; sqlite> .quitThanks again @moyix!
Reacted by Joachim Breitner and Kevin C2 remaining items
Since I was asked by a friend, here again a summary of all steps. This is for a Debian or Ubuntu system.
- (Only once,) install
sqlcipher:apt get install sqlcipher - Maybe start in an empty directory where you want to store your archive:
mkdir signal-archive cd signal-archive - Run
The output should look something like
cat ~/.config/Signal/config.json{ "window": null, "key": "b5bb9d8014a0f9b1d61e21e796d78dccdf1352f23cd32812f4850b878ae4944c" } - Run
In that program, enter these commands (using the key you just saw):
sqlcipher ~/.config/Signal/sql/db.sqlite`
You now should have a filePRAGMA key = "x'b5bb9d8014a0f9b1d61e21e796d78dccdf1352f23cd32812f4850b878ae4944c'"; .mode csv .headers on .out messages.csv select * from messages order by conversationId, sent_at; .quit
messages.csvin the current directory with all your messages. - Copy
~/.config/Signal/attachments.noindex/, which contains all media files:cp -r ~/.config/Signal/attachments.noindex/ attachments
You now have a complete archive in your current directory (although a bit inconvenient to use, unfortunately.)
Reacted by Scott Weldon, Brendan Dolan-Gavitt, Patrik Huber, Johannes Schweig, Peter Magenheimer, Kat Hagan, Thalia Archibald, codywohlers and Arto BendikenReacted by Andrea Dell'Orco, KGMIT, Kat Hagan, Nick and Drag0nFlyReacted by Matt Parlmer and Drag0nFly- (Only once,) install
It's a shame that this was closed and no real solution exists. (@nomeata, thanks for the write-up, very useful, but Signal really needs a first-class citizen way of doing this).
Reacted by Scott Weldon, Niklas, Kevin C, correliebre, Kit Sturgeon, mivk and Drag0nFlyI followed @nomeata's exact instructions on macOS with Signal Desktop 1.22 for macOS and after doing
PRAGMA key = "x'123456789ABCDEF123456789ABCDEF123456789ABCDEF123456789ABCDEF1234'"; .mode csv .headers on .out messages.csv select * from messages;on the sqlcipher CLI I get
Error: file is not a databaseWhen instead trying
.tablesright after setting thekey, I get the same error message.
I did extract the key from config.json.
What am I doing wrong? Did something change since last year?
Feels like I must be very close.Reacted by Scott Weldon and Kedar Iyer@rph8 , the signalapp github is used for bug-reports rather than for tech-troubleshooting of unofficial workarounds :-) If you can please open a new thread over at https://community.signalUsers.org , or if you prefer just add a comment to https://community.signalusers.org/t/cannot-read-db-sqlite-windows/6580 which already exists, that IS the place where unofficial workarounds and tech-troubleshooting are likely to get figured out.
To answer your question, the problem is that you did not quite follow nomeata's instructions, because they specifically said that you needed Linux, and you are on MacOS... which has very similar signal4desktop codebase, but a different directory-structure. Here are the helpdocs, https://github.com/signalapp/Signal-Desktop/blob/master/CONTRIBUTING.md#the-staging-environment and you can see there MacOS prefers
~/Library/Application Support/Signal/sql/db.sqliterather than~/.config/Signal/sql/db.sqlitethat is used on Linux.p.s. No this was not tested on an actual MacOS. But if my guess as to the answer was wrong, please ask in the signalUsers forum, not here on github, which was my main point, somebody there will have tried this on OSX (or be willing to help test).
@nomeata that walkthru was enormously helpful, thanks!
Reacted by KGMITJust to complete information here.
Error: file is not a databasePRAGMA key = 'YourPasswordKere'; PRAGMA cipher_compatibility = 3; ...did the trick for me. See sqlcipher/sqlcipher#299 (comment)
Reacted by Kat HaganPRAGMA key = 'YourPasswordKere';
PRAGMA cipher_compatibility = 3;This still doesn't work for me:
Error: file is encrypted or is not a databaseReacted by Laurent Forthomme, Scott Weldon and Kedar IyerFor me it worked with SQLCipher version 3.30.1 without the cipher_compatibility setting.
Reacted by bartwr and Ruben Barkow-KuderReacted by bartwrFor me it worked with SQLCipher version 3.30.1 without the cipher_compatibility setting.
This worked for me. I had to build sqlcipher from source based on https://www.zetetic.net/sqlcipher/introduction/
and I could simply follow the steps as per: #2516 (comment)
Thanks @nomeata and @zormitReacted by bartwrReacted by bartwrSignal.Crypto.encryptSymmetric = async function (key, plaintext) { return new Uint8Array(plaintext); }
Signal.Backup.exportToDirectory("/tmp", {key: true});I got the error in the console (CTRL+Shift+i):
Promise {<rejected>: Error: export is only supported in test mode at Object.exportToDirectory (/opt/Signal/resources…}__proto__: Promise[[PromiseStatus]]: "rejected"[[PromiseValue]]: Error: export is only supported in test mode at Object.exportToDirectory (/opt/Signal/resources/app.asar/js/modules/backup.js:1215:11) at <anonymous>:1:15 /opt/Signal/resources/app.asar/js/modules/backup.js:1215 Uncaught (in promise) Error: export is only supported in test mode at Object.exportToDirectory (/opt/Signal/resources/app.asar/js/modules/backup.js:1215) at <anonymous>:1:15* In that program, enter these commands (using the key you just saw): ```sql PRAGMA key = "x'b5bb9d8014a0f9b1d61e21e796d78dccdf1352f23cd32812f4850b878ae4944c'"; .mode csv .headers on .out messages.csv select * from messages order by conversationId, sent_at; .quit ```Can you add those commands somehow to the script at https://unix.stackexchange.com/a/505009/20661 ?
I tried the new export method again, and I was still getting the error that @ywwg and @rph8 have reported:
Error: file is encrypted or is not a databaseI tried the
cipher_compatibilitycommand, but that didn't work either, and after digging around in the source code it seems like it shouldn't be needed anyway.This was after installing the latest version of the Debian package from updates.signal.org (shows as version 1.39.4). I had Signal installed before, but when it launched it prompted me to delete the old data, so the database should be freshly created.
The error occurred after issuing any command after opening the database, even a simple
SELECTquery or.databasescommand. I know it wasn't an error copy/pasting the key, because this didn't work either:sqlcipher ~/.config/Signal/sql/db.sqlite \ "PRAGMA key = \"x'$(jq -r '.key' ~/.config/Signal/config.json)'\"; select * from messages limit 1;"(I also tried the old export method, and it looks like the
exportToDirectoryfunction still exists, but even if I disable the test mode check, it gives an error on the key, so the old solution of{key: true}doesn't work.)Finally I noticed that Debian only has SQLCipher 3, while Signal now uses SQLCipher 4. Rather than compile it from source, I found this Docker image, which contains version 4. I ran it like this:
docker run --rm --volume="$HOME/.config/Signal/sql/db.sqlite:/db.sqlite:rw" \ --entrypoint sqlcipher yspreen/sqlcipher /db.sqlite \ "PRAGMA key = \"x'$(jq -r '.key' ~/.config/Signal/config.json)'\"; select * from messages limit 1;"and it worked for me. So the complete export command becomes:
docker run --rm --volume="$HOME/.config/Signal/sql/db.sqlite:/db.sqlite" \ --volume="$(pwd)/plaintext.db:/plaintext.db" --entrypoint sqlcipher yspreen/sqlcipher db.sqlite \ "PRAGMA key = \"x'$(jq -r '.key' ~/.config/Signal/config.json)'\"; attach database 'plaintext.db' as plaintext key ''; SELECT sqlcipher_export('plaintext'); DETACH DATABASE plaintext;"(Note that the file
plaintext.dbmust exist beforehand or it will be mounted as a directory. Do that withtouch plaintext.db.)Reacted by Kedar Iyer and Merouane Atig@501st-alpha1 's above message works for the most recent version of Signal. Just copy paste that docker command and you'll be set.
Reacted by Scott Weldon and Merouane AtigTo access the database directly with the docker command, run
docker run -it --rm --volume="$HOME/.config/Signal/sql/db.sqlite:/db.sqlite" --volume="$(pwd)/plaintext.db:/plaintext.db" --entrypoint sqlcipher yspreen/sqlcipher db.sqliteThen you can unlock the DB with your key from config.json:
PRAGMA key = "x'2DD29CA851E7B56E4697B0E1F08507293D761A05CE4D1B628663F411A8086D99'";Reacted by Scott WeldonAs far as I see, config.json now contains not
keybutencryptedKeyand the sequence above is not applicable:# exit signal-desktop $ cp ~/.config/Signal/sql/db.sqlite /tmp/ $ sqlcipher /tmp/db.sqlite \ "PRAGMA key = \"x'$(jq -r '.encryptedKey' ~/.config/Signal/config.json)'\"; select * from messages limit 1;" ok Error: in prepare, file is not a database (26) # same w/o x' symbols: $ sqlcipher /tmp/db.sqlite \ "PRAGMA key = \"$(jq -r '.encryptedKey' ~/.config/Signal/config.json)\"; select * from messages limit 1;" ok Error: in prepare, file is not a database (26)Is it still possible to extract messages with
sqlcipher?
I just tried to export my Signal data from the desktop app, so that I can archive it in a Git repo that I sync to my server.
To my surprise after I did
in the console, I got
Error: Encrypted export/import is disabled.I tracked it down to this line, and blame indicates commit 8eeaad8, which merely says:
I haven't been able to find any discussion of this in the GitHub issues; why was this removed? I noticed that if I comment out the
throwline, the export works fine.Given the interest in #522, #1013, and others, I think this feature was actually in use, though I can't say how widespread it was. I personally don't care as much about having an import available, as long as I can read my data outside of Signal.
Can this please be re-enabled?