Addressing the various security improvements

[synctext]

Several issues where listed by "Yawning Angel" on Tor mailing list.

A casual review brought up some issues and we here provide a structured reaction to them. All issues are addressed already or will be fixed soon.
In general, the reviewer is correct, our code needed to be written more clearly and dead code was insufficiently cleaned.
Issues:

• issue: How not to do Diffie-Hellman: key = pow(dh_received, dh_secret, DIFFIE_HELLMAN_MODULUS) which relied on gmpy.rand which was improperly seeded
• Reaction: We're actually not sure if this was the case, but we change the seeding of the gmpy.rand method to make sure were always using a seeded rand now. Moreover, we aim to replace this custom DH with the one implemented in M2Crypto asap.
• Reaction2: The gmpy.rand is cannot be saved, removed all references to it.
• issue: one-time AES keys are generated with python's random.randint()
• Reaction: Fortunately not used for long-term keys, cause by the now famous ImportError which prevented PyCrypto from being imported.
• issue: How not to do RSA: def rsa_encrypt(key, element):
• Reaction: RSA was never used in the tunnels, implemented while researching homomorphic encryption. This was for a paper published in WIFS 2013 http://dx.doi.org/10.1109/WIFS.2013.6707798. In this paper we implemented/evaluated three different approaches to Private Set Intersection-problem and tested their applicability in a P2P system. One of these approaches used RSA, which if used in unpadded mode (http://en.wikipedia.org/wiki/Homomorphic_encryption#Unpadded_RSA) has homomorphic properties. However, neither M2Crypto/PyCrypto allowed us to generate such an compatible key. Therefore, we wrote a small piece of python which allowed us to do so, hence the "compatible_key" method https://github.com/Tribler/tribler/blob/devel/Tribler/community/privatesemantic/crypto/rsa.py#L23.
  This shouldn't be used in the wild, and wasn't/isn't used in the tunnels.
  A pull request is submitted which fixes the dodgy optional_crypto file, by removing the optional part.
• issue: ECB-AES128 usage in the code.
  Yes, we are aware of this issue. ECB-AES128 needs to be replaced asap with our new code that adds packet loss resilience. We aim to use CTR mode instead, and compensate for UDP packet loss with optimistic decryption that uses sequence number estimation, please read our detailed report: http://repository.tudelft.nl/view/ir/uuid%3Ace3bd867-6540-426d-87d0-348bdf78279d/

[Yawning]

For the record, I have nothing against the concept of what you all are trying to do, and think it's an admirable goal, and do wish you all the best of luck. That said, there is a lot of work that needs to go into certain kinds of systems before real users should see them.

With regards to the improve_crypto branch, here, have some followup:

• Yay, a lot of the old code is removed so this is easier to follow.

• Ok, using pycrypto's StrongRandom() is a reasonable thing to do.

• Diffie-Hellman key generation is still catastrophically broken, and it is possible that it got worse. On a Linux system, with glibc, the first DH private key generated (assuming the code is run in isolation, eg in a unit test) will always be: 0xb43d2488d8be2449e737dd6308168ab2c22668e57cb1a10a140ebabcc01ab988b331289d248731af85720ddbee86609da2877b56962c84c8f481e85e942a5b6e871add8296accc54880a785ef1f5d466e0632dd76b65fccff6b69d1af7a99d9e0490f300829c0c2960bdd3fe3d47d17f135e32f79e32c6bbe65f965d63185694. There's nothing particularly magical about "first" (The gist linked has 10), and with the changes in the branch, key generation is weak to the point where link layer security is non-existent.

  See: https://gist.github.com/Yawning/4cca041c456ccf113700 (Note: Example depends on your libc's rand() matching mine, version included in the gist.)

• The ECElgamal code uses "rand('next', 10000)" to calculate k (Wow, I can't believe I missed this in my original writeup). Given that "rand('next', 10000)" is not cryptographically secure, there is no security here as recovering the plaintext is trivial.

  Really, gmpy's "rand()" does not belong anywhere near cryptography, except in dire warnings such as this sentence.

• Re: the link crypto in the thesis. I don't have time at the moment to give this thorough review, but it scares me, because Not-Invented-Here approaches to cryptography SHOULD scare people, especially when doing things that do have time tested solutions (IPsec, DTLS, i2p, CurveCP...), and at a quick glance it seems over-complicated (and probably broken in interesting ways). The fact that opportunistic decryption enters the picture is particularly troublesome because none of other protocols require such steps.

Obligatory disclaimer: Even fixing the further issues is probably still not enough to protect the system from adversaries, including those in your threat model. Without fixing said issues, both master and the improved branch (which does have some nice cleanups), will not protect vs a bored undergrad.

[NielsZeilemaker]

@Yawning many thanks again for these comments. I've removed the custom DH and replaced it by the one included in M2Crypto/OpenSSL. Again, why we choose to implement DH ourselves instead of using this version is beyond me.

Regarding the gmpy's rand method, after reading the docs I was really under the impression that letting gmpy set the seed would suffice. But its clear now that it does not.
I'm going to refactor/remove all references to rand.

[NielsZeilemaker]

@Yawning all references to rand are now gone, additionally the ECElgamal code now uses a k which is relatively prime to the modulus of the curve.

[Yawning]

@NielsZeilemaker Glad to hear it. What's most important to me is that all these things get fixed, and I'm glad to see that you're working hard (though you should also remember to enjoy the holidays).

If you want someone to talk to about how to do the symmetric cryptography in a manner suited for the UDP transport, feel free to drop me an e-mail (at either the one I used to post to tor-dev or at my torproject.org one), since it's a problem space I'm relatively familiar with.

[NielsZeilemaker]

@Yawning I guess the first thing to do now, is replace the custom ecelgamal with one implemented by a library. Do you have a pointer towards a reliable library which implements it?

[whirm]

I'm moving this to v6.4.2 milestone as we are releasing v6.4.1 today.

[whirm]

Note that 6.4.1 has all the fixes except for the ECElgamal replacement.

[NielsZeilemaker]

@Yawning I was thinking to replace the custom ecelgamal stuff with an approach describe here
http://stackoverflow.com/a/1175343
E.g. using ECDH combined with a one-time EC key to generate a AES key to be used to encrypt a packet. What do you think of this approach? We already have the public key of the receipient, so I guess this should work. And ECDH is included in OpenSSL/exposed by M2Crypto