x/crypto/openpgp: strips \r decrypting binary data

[tobstarr]

When encrypting specific (in my case gzip compressed) plaintexts it seems that golang.org/x/crypto/openpgp generates ciphertexts which can then no longer be decrypted with the gpg cli tool. Those broken ciphertexts can be decrypted with the openpgp package itself just fine though.

I created a gist with some example plaintexts (good and bad) and some test code to reproduce the issue at https://gist.github.com/tobstarr/a4afa1ec7abf06fbe9e5

As described in the gist I tested it both with gpg versions 1.4.16 on ubuntu 14.04 and gpg version 1.4.18 on ubuntu 15.10 with my own key (4096 RSA) and a newly generated one (2048 RSA).

[tobstarr]

/cc @nightlyone

[tobstarr]

Regarding go versions: I tested it on ubuntu 15.10 with go versions 1.4.3 and 1.5.1 and on ubuntu 14.04 with version 1.2.1

[rsc]

Looking at the gist, it shouldn't be necessary to drag tar into this. Either gpg spits out the original file or it does not. When it does not, how does the gpg output differ from the original?

[tobstarr]

I did some brute forcing and it seems all issues are related to different treating of the "\r" character.

When decrypting with gpg all "\r" characters seem to be stripped, when decrypting with the openpgp package they are still present (so that data must still be somewhere in the ciphertext).

[rsc]

/cc @bradfitz openpgp seems to be stripping \r in Decrypt somewhere.

[tobstarr]

@bradfitz actually it does something to "\r" when encrypting which then leads to the gpg cli tool to strip those, when decrypting with openpgp they are still present.

[tobstarr]

See the example in the (now updated) gist at https://gist.github.com/tobstarr/a4afa1ec7abf06fbe9e5

[bradfitz]

/cc @agl as FYI

But I can look at this too.

[rsc]

You said the "now updated gist" but I still see the old one with output not matching the program.

l.Printf("input=%q output=%q otuput_openpgp=%q", pt, dec, decOpenpgp)

does not match:

input="hello world" output="hello world"
input="hello\n\rworld\n\r" output="hello\nworld\n"
input="\n" output="\n"
input="\t" output="\t"
input="\r" output=""

There is no otuput_openpgp= shown.

[tobstarr]

@rsc sorry, by updated I was referring to original version (with the random tar input). I now also updated the output in the README though.

[rsc]

OK, thanks. With the corrected output I now see that:

1. Using Go's openpgp library you encrypt a file with \r characters.
2. Using Go's openpgp library to decrypt produces the original bytes verbatim.
3. Using the gpg command line tool to decrypt strips the \r characters.

That sounds like a problem in gpg, not Go's openpgp library.

Can you check whether a round trip using gpg alone (leave Go openpgp completely out of it) preserves the \r characters?

If gpg cannot round trip a binary file containing \r characters, then I think it's just gpg's issue, although it may be intended that gpg is only used for text, and binary uses may be expected to do base64 or uuencoding or something like that first. But if gpg can round trip a binary file containing \r's, then maybe Go's openpgp library encryption in step 1 is failing to set some kind of "binary-safe" bit in the encrypted output.

Thanks.

[tobstarr]

Okay, my bad. I looked through the code and saw that there is "IsBinary" flag in the FileHints options which is false by default. My problem was that gpg treats everything as binary by default and in openpgp it is the other way round.

When I set that flag all my tests pass. Sorry for the inconvenience,