Importing private key fails after all "bad self certificates" removed #1003

Closed
ikelos opened this Issue Nov 14, 2014 · 19 comments

Projects

None yet

6 participants

@ikelos
ikelos commented Nov 14, 2014

Hi there,

I recently tried to import my desktop gpg keys into openkeychain. The error log is quite useful, but unfortunately does not give me much to go on (and nor does the source code). I exported my key from gpg2 using 'gpg2 --export-secret-key -a "name" > file.asc'. After transfering the asc file over, I attempted to import it. The failure message gave me the following:

[Purple] Importing secret key 0xfffffffffffff
[Purple] Canonicalizing secret key 0xffffffffffffff
Processing master key
[Yellow] Removing bad self certificate for user ID 'First user'
Removing invalid user ID 'First user'
[Yellow] Removing bad self certificate for user ID 'Second user'
Removing invalid user ID 'Second user'
[Red] Keyring has no valid user IDs!
Merging in self-certificates data from public keyring
[Green] Nothing to merge
[Purple] Canonicalizing secret key 0xfffffffffffff
A which point it repeats down to the red again...

I've tried it without the public keys installed before hand, with the public keys installed, with the public keys installed and certified by another cert on the phone, with the public keys imported at the same time, all of which gave me the same pattern of the user IDs being removed and then none being left. The key works fine in GPG/seahorse, so I'm really not sure how best to diagnose this. Any help would be much appreciated!!!

[Public key is available on most PGP servers, 0xbbbad6a26c20157a]

@Valodim
Contributor
Valodim commented Nov 14, 2014

Hi, thanks for your bug report.

I can't really think of a reason for this problem. The public key import on its own works correctly, right?

Since these are only user ids and their certificates we are dealing with, we can try to diagnose these without the secret key parts:

Run gpg --export-secret-key > file.gpg to get the secret keyring into a file, then gpgsplit --secret-to-public --no-split file.gpg > otherfile.gpg to get all the data from your secret key minus the actual private keys. you can check this with gpg --list-packets otherfile.gpg, there should be no "secret subkey packet" or anything of the sort in there. You can paste this (gpg --enarmor < otherfile.gpg > otherfile.asc) so we can have a look, or see if the import works in OpenKeychain yourself - hopefully, it reproduces the same errors as your actual secret keyring so we can work from there.

@Valodim
Contributor
Valodim commented Nov 14, 2014

for the record, import of the regular public key looks okay

@ikelos
ikelos commented Nov 16, 2014

Hiya,

So having split the file, and attempting to import otherfile.gpg, it succeeds. Attempting to import file.gpg still fails on that one key (the other private keys in the same file import just fine).

The only things I can think that are special about that key are that the expiry date has been updated (it was originally only valid for 2 years at a time), and that it features two identities/email addresses.

Would the output from --list-packets be valuable in any way, or is there any other output I can provide that might help diagnose the problem?

@Valodim
Contributor
Valodim commented Nov 16, 2014

hm, that's too bad. but yeah, nothing in the otherfile.gpg then that could be helpful.

not sure if --list-packets is helpful, but it's worth a shot. can't really think of any other way to diagnose this..

@ikelos
ikelos commented Nov 16, 2014

Yeah, I was hoping it would be easier to figure out too... 5:\

Here's the list-packets data for the key in question. If you can think of anything else I could try, please let me know...

:secret key packet:
version 4, algo 17, created 1144016020, expires 0
skey[0]: [1024 bits]
skey[1]: [160 bits]
skey[2]: [1024 bits]
skey[3]: [1024 bits]
iter+salt S2K, algo: 2, SHA1 protection, hash: 2, salt: REDACTED
protect count: 65536 (96)
protect IV: REDACTED
encrypted stuff follows
keyid: BBBAD6A26C20157A
:user ID packet: "Mike Auty (Gentoo Key) mike.auty@gmail.com"
:signature packet: algo 17, keyid BBBAD6A26C20157A
version 4, created 1391799158, md5len 0, sigclass 0x13
digest algo 2, begin of digest 38 30
hashed subpkt 27 len 1 (key flags: 03)
hashed subpkt 11 len 5 (pref-sym-algos: 9 8 7 3 2)
hashed subpkt 21 len 3 (pref-hash-algos: 2 8 3)
hashed subpkt 22 len 3 (pref-zip-algos: 2 3 1)
hashed subpkt 30 len 1 (features: 01)
hashed subpkt 23 len 1 (key server preferences: 80)
hashed subpkt 25 len 1 (primary user ID)
hashed subpkt 2 len 4 (sig created 2014-02-07)
hashed subpkt 9 len 4 (key expires after 11y352d13h46m)
subpkt 16 len 8 (issuer key ID BBBAD6A26C20157A)
data: [160 bits]
data: [154 bits]
:user ID packet: "Mike Auty (Gentoo) ikelos@gentoo.org"
:signature packet: algo 17, keyid BBBAD6A26C20157A
version 4, created 1391799158, md5len 0, sigclass 0x13
digest algo 2, begin of digest c2 47
hashed subpkt 27 len 1 (key flags: 03)
hashed subpkt 11 len 5 (pref-sym-algos: 9 8 7 3 2)
hashed subpkt 21 len 3 (pref-hash-algos: 2 8 3)
hashed subpkt 22 len 3 (pref-zip-algos: 2 3 1)
hashed subpkt 30 len 1 (features: 01)
hashed subpkt 23 len 1 (key server preferences: 80)
hashed subpkt 2 len 4 (sig created 2014-02-07)
hashed subpkt 9 len 4 (key expires after 11y352d13h46m)
subpkt 16 len 8 (issuer key ID BBBAD6A26C20157A)
data: [159 bits]
data: [158 bits]
:secret sub key packet:
version 4, algo 16, created 1144016028, expires 0
skey[0]: [2048 bits]
skey[1]: [3 bits]
skey[2]: [2048 bits]
iter+salt S2K, algo: 2, SHA1 protection, hash: 2, salt: REDACTED
protect count: 65536 (96)
protect IV: REDACTED
encrypted stuff follows
keyid: AC0EE9D4ADA96379
:signature packet: algo 17, keyid BBBAD6A26C20157A
version 4, created 1391799241, md5len 0, sigclass 0x18
digest algo 2, begin of digest 57 2a
hashed subpkt 27 len 1 (key flags: 0C)
hashed subpkt 2 len 4 (sig created 2014-02-07)
hashed subpkt 9 len 4 (key expires after 11y352d13h46m)
subpkt 16 len 8 (issuer key ID BBBAD6A26C20157A)
data: [159 bits]
data: [160 bits]

@Valodim
Contributor
Valodim commented Nov 20, 2014

the gpg dump doesn't help, but here's another idea:
gpg --armor --export-secret-subkeys SUBKEYID0\! SUBKEYID1\! > all_stripped.asc
this exports all named keys as "stripped". you can check with gpg --list-packets that they are all marked as "gnu-dummy S2K" which means they aren't really there. The filesize is also insufficient to contain the secret keys, and the keys should show up as "stripped" in OK.

if the problem exists for that keyring, we can use it to diagnose since it contains no secret information.

@ikelos
ikelos commented Nov 23, 2014

Sorry for the delay, I was away and then had to verify that my secret key blocks weren't going anywhere. The following file exhibits the error I've been seeing, OK never says stripped, but that's most likely because I can't import the keys. When wiping the cache & data for OK and then attempting to install the key from file on the first import it doesn't show the error, but no keys are imported (whereas they are with a working key).

-----BEGIN PGP PRIVATE KEY BLOCK-----
Version: GnuPG v2

lQGqBEQwTJQRBAC1FvsTYlZ669GutRGDOEPRS5ZZbemcGUIDM1iFzUh94DPFLeRl
dcaXqtVHFPcp42Fy4I9nzKvLNoV6a77Ruf03MEGWxSHZ/eHueb8KQ2gPsZkr2SWm
F2deco+rRVRPzy/0O9wJ6WT55zo6FiB5NPuAUxPJ+Fm4FDbykEm3lSJi3wCguK+K
98BEpD9E0P+ULE15AguT8k8EAKHSoebyKMkhlZ/3Y0GBMG+TvTpfnOBcd7BPnumS
d1Wo43+JiMnN5gJ1SdI7kgLMmpne+YijYPwSXjpjYG7L3ODyd8rFcw5OmsZWuhn/
YPZU6gnHlIquDn1DOQS1cSKOmlkFIGF1J7QT353nxQOTnIvvUciY4qLTn6i2jdit
C8NOBACCyPA754bBLw9Cd6oSiec3X5xmq26NS2HRyBGSy8StFeqQ9b7Q9HcCa2/i
/cPAW37mh2/7K6JzPVHoEVmem28623lG3oAI8/lnGQ3bMIXTrFfbmvLr5VgUE5v5
lXzClPnuW/Is18RoWpcWyN/mzeuRYrSYmAAyt42uO7wWVcDSuP4CZQJHTlUBtCxN
aWtlIEF1dHkgKEdlbnRvbyBLZXkpIDxtaWtlLmF1dHlAZ21haWwuY29tPohpBBMR
AgApAhsDBgsJCAcDAgQVAggDBBYCAwECHgECF4ACGQEFAlL1K3YFCRZ+CCwACgkQ
u7rWomwgFXo4MACgtwC/rnRibKKFk+bDSU7sopH761UAmgLEhwXacPb3GoCHfVxW
3isp0MlltCZNaWtlIEF1dHkgKEdlbnRvbykgPGlrZWxvc0BnZW50b28ub3JnPohm
BBMRAgAmAhsDBgsJCAcDAgQVAggDBBYCAwECHgECF4AFAlL1K3YFCRZ+CCwACgkQ
u7rWomwgFXrCRwCfU4E/xYLr4ACO/0ef6g1EsikgTCYAniU2/WOu+Khivf+bTG3D
W4H6LjCE
=9eI6
-----END PGP PRIVATE KEY BLOCK-----

@ikelos
ikelos commented Dec 20, 2014

Any progress on this?

@zanchey
zanchey commented Dec 27, 2014

I have a similar problem; the key I am trying to import has a master signing key and two subkeys (one signing, one encryption), but the secret key for the master signing key is not included.

@Valodim
Contributor
Valodim commented Jan 16, 2015

Hi. Sorry for the lack of response, we sort of took a break in december and are only now working on the backlog.

So here's the rundown: changing the expiry date of a key in gpg prior to version 2.1.0 breaks the secret key in a way which emerges only on export. It's not a problem with OpenKeychain, we correctly reject the key because its self-certificates are either invalid, or have wrong flags (@zanchey's case).

This issue has been reported before (#996, #1026), and can be assumed to affect a large number of users. The bug in gpg has been fixed in gpg 2.1.0, but that version is as of now only deployed in debian experimental, not even sid. Another bug report has been opened by jas just today to backport the fix, so I hope this gets fixed soonish.

I'm afraid I can't be more helpful than this, there is not enough information in the key to reliably detect the issue so we can't even create a notice about it when import fails.

@ikelos
ikelos commented Jan 16, 2015

Hiya, that worked great. I upgraded to gnupg-2.1.1, exported my keys and they imported absolutely fine. I can now downgrade to gnupg-2.0 again and carry on as before. Thanks for solving the problem! I'm happy to have this closed if you'd like, but I'll leave it up to you in case it's easier for other people to find whilst it's open?

@dschuermann
Member
@zanchey
zanchey commented Jan 20, 2015

Sadly it's been wontfixed upstream; I wonder if there is a way to correctly export the keys at all without upgrading to 2.1.0. Presumably the flags are invalid on all published copies of my key as well as the master copy...

@Valodim
Contributor
Valodim commented Jan 20, 2015

while it is an understatement to say that the keys are "getting out of sync", he is right in that the information is still there and correct in the public key. But: gpg --export-secret-keys only exports the secret key, not the public one as well like some implementations do.

What we could do is keep secret keys we encounter around even if they aren't certified, so that we can use them if we find a certificate for them (from a public key), but that is not a simple change so I can't promise anything.

@zanchey
zanchey commented Jan 22, 2015

Understood - many thanks.

@kolAflash

Accepting that GPG-2.0.x does something wrong. Nevertheless the APG Android app seems to be able to handle those private keys exported by GPG-2.0.x.
Would it be hard or bring problems to implement this behaviour for OpenKeychain too?

@dschuermann
Member

APG is simply not verifying the certificates corrently, that's why it "works". It also means that there are attack vectors present in the current APG version. Please consider upgrading GPG to fix this bug.

@Millak
Millak commented Sep 8, 2015

It turns out that if you use gnupg version 1.4.19 (probably all of 1.4.*) also has the bug with gpg --export-secret-(sub)keys, but it does work with gpg2.

@Valodim
Contributor
Valodim commented Sep 9, 2015

yes, that's right. we asked werner to backport the fix, but he refused. not much more we can do about this.

secret key import is pending a rewrite, we might add a special mention of this bug in there to at least provide users with some guidance on the matter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment