Importing pgp keys and the existing one went away

[snan]

EDIT: hints for reproducing below

• Device: iPad
• iOS version: 17
• Delta Chat version: v 1.40.3
• Expected behavior: When I imported pgp keys it said "existing keys will not be removed"
• Actual behavior: Existing keys were removed.

it wasn't too bad since I could use my other MUA to decrypt the incoming message there so I knew what I was responding too.
As a workaround I re-imported the old keys with "legacy" in the name. Not sure if that worked since that particular message still is unreadable in Delta Chat even though I can read it in my other MUA.

[snan]

The guy is also on Delta Chat fwiw!

[r10s]

thank you for your report!

Actual behavior: Existing keys were removed.

how did you checked that you actually only had one key after import? and that this key is the new key and not the old one?

but yeah, tbh, this is probably a code path that is not tested often, again, thanks for report

The guy is also on Delta Chat fwiw!

👋

[r10s]

hm, i just tried and could not reproduce the issue with 1.42.4 (currently in review, not yet in the store)

what i did:

1. create two fresh Delta Chat testing accounts, Alice and Bob
2. make sure, in the system's "Files" app are no .asc files in the "Delta Chat" folder
3. switch to Alice: Settings / Advanced / Manage Keys / Export Secret Keys
4. switch to Bob: Settings / Advanced / View Log / private_key_count reads "1" - so bob has one private key
5. still Bob: Settings / Advanced / Manage Keys / Import Secret Keys - this imports the keys from Alice
6. still Bob: Settings / Advanced / View Log / private_key_count reads "2" - so Bob has two private keys now, just as expected

[snan]

My current private key count is five somehow.

But a few minutes after I imported the new one, I got an email from a guy who is also on Delta Chat who last had seen a message from me before the new import.

That email was the Delta Chat–generated:

... – [Meddelandet kan inte avkrypteras.

• Det kanske kan hjälpa, att svara på detta meddelande och be avsändaren skicka det igen.

• Ifall du har ominstallerat Delta Chat eller något annat e-postprogram på denna eller någon annan enhet, kanske du behöver skicka ett nytt Autocryp inställningsmeddelande därifrån.]

(I also would wanna send a patch on that string but I forget where the translation project lives)

but in my other MUA (probably better known as Emacs) his message was readable since that one still had access to all my PGP keys. So I didn't wanna confuse him and just replied to his comment as if everything was normal and from then on his messages has been decrypted by Delta Chat.

To prevent this from happening again, I then re-added my old key with a "legacy" name and could verify that that did not override the new one, that I was still using the new one. (But I have not yet been able to verify that the legacy one works since I haven't gotten any more messages sent to it. The unreadable message is still unreadable.)

Now, y'all know that my philosophy is that "a user can't read a message they're supposed to be able to read" is usually a much milder problem than "someone can read something they were NOT supposed to be able to read", i.e. accidentally sending plaintext. So don't take this as me changing my mind about that, I still feel that way. But this still seems like a bug since the import dialogue said it would keep the old keys around. 🤷🏻‍♀️

[r10s]

But a few minutes after I imported the new one, I got an email from a guy who is also on Delta Chat who last had seen a message from me before the new import.

That email was the Delta Chat–generated: [error message]

thanks for clarification, we'll try to reproduce this

(I also would wanna send a patch on that string but I forget where the translation project lives)

see https://delta.chat/en/contribute#translations-and-bug-reports , the english string was recently updated at deltachat/deltachat-android#2832 , you should be able to see that on transifex as well (you can log in there with github)

[r10s]

thanks for clarification, we'll try to reproduce this

k, i could reproduce this as follows:

1. create 3 fresh Delta Chat testing accounts, Alice, Bob and Claire
2. switch to Claire: Settings / Advanced / Manage Keys / Export Secret Keys - you can now delete "Claire"
3. do a QR code scan between Alice and Bob
4. switch to Alice: send text "before key import" to Bob
5. switch to Bob: Settings / Advanced / Manage Keys / Import Secret Keys - import the "Claire" key, original key should be preserved but is not
6. switch to Alice: send text "after key import" to Bob - this message is not readable by Bob

moving this issue to core. further hints:

• private_key_count changes at step 5 as expected from "1" to "2"
• "excryption info" shows different private keys before/after import, also as expected
• maybe secondary keys are just not tried for description, cmp comment from @snan below that "legacy import" does not heal the issue

EDIT: if it helps to avoid adding more complexity to code, we can also consider giving up support of multiple private keys and change documentation. it is a niche anyways, causing lots of ux questions. and we do not have and do not want ui for that. usually, it should be fine with one private key per account. that way, we could also get rid of this "legacy" namings

[snan]

I have new info! Namely that importing the old one as "legacy" did not help 🤦🏻‍♀️
It did zilch and it's as if Delta Chat was only "aware of" the new key even though the log says 5 keys

[snan]

Things are weird.

Now posts that I encrypt in the other MUA (which is Emacs) I can't read in Delta Chat anymore. Even though they are using the new key ID which is 8CF90C2066357D20112DD0389FC344CA12F1484A.

The other MUA can still decrypt all the things and it tells me which key ID was used.

So for example I sent a message using Delta Chat to "Alice" who is also on Delta Chat yesterday evening. I can read it in Delta Chat and in the other MUA, which says it was encrypted with 8CF90C2066357D20112DD0389FC344CA12F1484A which is the new key.

I sent a message to "Bob" who is on PGP but not Autocrypt so I had to use the other MUA. Normally those messages are readable by me in Delta Chat but I'm getting the "can't decrypt" standard message. On messages I have written. Other MUA says it was using 8CF90C2066357D20112DD0389FC344CA12F1484A. Some of the older messages in the thread were encrypted with my previous key, probably better known as 524040874E61D985E71C4FE624DBD6905B61CE27. Delta Chat shows them in plain text with a padlock (but it decrypted them before I switched keys—this whole bug report started when my Delta Chat couldn't decrypt from "Ted" who had sent via Delta Chat continuing a discussion we had had the same day (only that I imported a new key file in between. It couldn't show that one last message to the old key but it can show all the other older messages. This shows me that Delta Chat are storing the messages in plaintext [in a db or something] on the iPad. Which I guess is fine, notmuch has a setting to do the same thing).

[snan]

I'm getting a million

[Meddelandet kan inte avkrypteras.

• Det kanske kan hjälpa, att svara på detta meddelande och be avsändaren skicka det igen.

• Ifall du har ominstallerat Delta Chat eller något annat e-postprogram på denna eller någon annan enhet, kanske du behöver skicka ett nytt Autocryp inställningsmeddelande därifrån.]

all over the place. Even on my own emails. My key situation is totally tangled up and ruined. I can't really use Delta Chat in this state.

[link2xt]

There was an Android change: deltachat/deltachat-android#2681
It is based on the core change that allows selecting individual files: #4737

The core only extended the API to allow selecting individual files rather than directories. When you select a file, it does not matter if it is "legacy" or not, it is set as the default. "legacy" in the filename only matters if you select a directory.

[snan]

Here are two new things I tried.

I tried re-importing my new key, thinking that the "legacy" import might've confused things. Now log says 6 private keys known.

I tried exporting my keys from Delta Chat just to see what was going on. It gave me two keypairs. The old one that I generated with gpg years ago and imported to Delta Chat and have been using for years, and the new one that I generated with gpg the other day. The old one was named public-key-default and the new one was named public-key-6. And the corresponding private keys were there also. I've checked 'em with diff that they're the same.

It seems like it's the new one that's in use both in Delta Chat and in Emacs but Delta Chat can no longer read what Emacs writes, the way it used to be able to do.

Also it can no longer what people are sending to my old public key (even though it's stored in Delta Chat named public-key-default and it's corresponding private key is private-key-default).

[snan]

I haven't been selecting files, only directories (which is the only way possible on iPad OS afaict).

(It doesn't give you a choice of directory even, there's just one directory allowed that it can scan.)

[link2xt]

Ok, so recent core change is unrelated and this behavior is not really new.

[link2xt]

maybe secondary keys are just not tried for description

Maybe this is even the case ever since a switch to rPGP?

Another suspicious change: #3859

Anyway, this needs a test and just actually debugging and see where it fails.

[link2xt]

Ok, so currently core only loads a single secret key when decrypting:

deltachat-core-rust/src/mimeparser.rs

Lines 267 to 269 in 616faff

	let private_keyring = vec![load_self_secret_key(context)
	.await
	.context("Failed to get own key")?];

There is no chance other keys are tried as the vector only has a single secret key.

[link2xt]

This commit is likely where it was broken: 8efc880

load_self_private_for_decrypting which loaded all keys is removed there, in 2020.

[snan]

Yes! More things have been cleared up over here! Turns out Emacs was encrypting to the old key after all because of a stale mml-secure-key-preferences in Emacs. Now I can read what I myself write again.

So a combination between problem-exists-between-escape-and-meta on my end and the bug in Delta Chat it's all been explained. But please still fix the bug since other people are still sending to my old key 🙏🏻