New abstract.Cipher interface

[bford]

This branch represents a significant batch of changes that I hope to merge in the next day or two, but would like to seek comment on first if anyone has feedback.

The main change is to create a new interface 'abstract.Cipher' representing a generic "all-purpose symmetric cipher" abstraction that can subsume block/stream ciphers, hash functions, authenticated encryption functionality, and in effect basically all the useful symmetric cipher functionality we know of. The current main implementation of abstract.Cipher, also in this set of changes, is based on the SHA3 or "Keccak" sponge function, which slightly contrary to the "SHA3" name is far more general than just a hash function. For background on the amazingly awesome coolness of cryptographic sponge functions, see for example: http://sponge.noekeon.org

The main impact of this change that will directly impact other code is probably just that you'll now have to call 'suite.Cipher()' instead of 'suite.Stream()' to get one of these general Cipher instances from the cipher suite. The new Cipher interface is backward-compatible with the old cipher.Stream interface, however, so an instance of Cipher will work fine wherever a cipher.Stream was used before. (I haven't decided whether or not to retain this backward-compatibility provision in the long term, but it isn't very cumbersome to do so I'm inclined to.)

But the more significant impact is the expressiveness and ease with which I believe this new interface allows us to do a lot of common things we need to do with symmetric ciphers. Here's one quick example of how it simplified authenticated-encryption, from anon/enc.go:

OLD anon/enc:
@@ -148,21 +145,18 @@ func Encrypt(suite abstract.Suite, rand cipher.Stream, message []byte,
anonymitySet Set, hide bool) []byte {

    xb,hdr := encryptKey(suite, rand, anonymitySet, hide)

    // We now know the ciphertext layout

•   hdrlen := len(hdr)
  

•   msglen := len(message)
  

•   maclen := suite.KeyLen()
  

•   ciphertext := make([]byte, hdrlen+msglen+maclen)
  copy(ciphertext,hdr)
  
  // Now encrypt and MAC the message based on the master secret
  

•   stream := abstract.HashStream(suite, xb, nil)
  

•   mackey := random.Bytes(maclen, stream)
  

•   mac := hmac.New(suite.Hash, mackey)
  

•   stream.XORKeyStream(ciphertext[hdrlen:hdrlen+msglen], message)
  

•   mac.Write(ciphertext[:hdrlen+msglen])
  

•   ciphertext = mac.Sum(ciphertext[:hdrlen+msglen])[:hdrlen+msglen+maclen]
  return ciphertext
  

  }

NEW anon/enc.go:
anonymitySet Set, hide bool) []byte {

    xb,hdr := encryptKey(suite, rand, anonymitySet, hide)

•   cipher := suite.Cipher(xb, abstract.Encrypt)
  
  // We now know the ciphertext layout
  

•   hdrhi := 0 + len(hdr)
  

•   msghi := hdrhi + len(message)
  

•   machi := msghi + cipher.KeySize()
  

•   ciphertext := make([]byte, machi)
  copy(ciphertext,hdr)
  
  // Now encrypt and MAC the message based on the master secret
  

•   cipher.Crypt(ciphertext[hdrhi:msghi], message)
  

•   cipher.Crypt(ciphertext[msghi:machi], nil) // MAC
  return ciphertext
  

  }

I believe this general-purpose Cipher interface may be at least marginally useful and interesting from a research perspective, as a possible high-level "narrow-waist" or "hourglass-interface" abstraction for symmetric-key crypto in general. I've started a sketch of a write-up on this idea in the papers/net/cipher directory of the papers repo for anyone interested, but it's still just a skeleton and probably not very readable yet. Anyone wanting to work on fleshing this out and exploring it further, feel free to volunteer.

In any case, I have NOT yet updated all the higher-level crypto functionality throughout the library to take advantage of the features of the new Cipher interface; there are still lingering uses of hash functions and HMACs in which code could probably simplified (and perhaps made more efficient) by making it use the Cipher interface. We can do these cleanups and simplifications incrementally as we get to them.

Thanks
Bryan

[bford]

Closing this in favor of an inter-branch pull in the same repo as suggested by Danny (#12 (comment)).