Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

beyond crypto-browserify: modular crypto #128

Closed
dominictarr opened this Issue May 13, 2015 · 36 comments

Comments

Projects
None yet
@dominictarr
Copy link
Contributor

dominictarr commented May 13, 2015

When I started the crypto-browserify project it was a simple idea: A pure javascript drop-in replacement for node's crypto module that would run in the browser. I never really expected that we could get as feature complete as we are now - I can only take a small part of the credit for this.

At the time, getting node's crypto module to work in the browser seemed like the right idea, but in the process, I have learnt a great deal more about cryptography and now I can see we can do much better.

Edit for clarity: crypto-browserify is not the end of the story, I think we need a new project (in a new org) that aims to do bulletproof crypto, with clearly defined properties and no surprising edgecases.

node's crypto is based on buggy standards - SHA, RSA, AES, TLS are all buggy. SHA1 is now considered to weak to be used in new protocols. SHA256 has length extension bug. RSA is unsafe without padding. AES has many insecure modes! If we are interested in building new cryptosystems (i am) then I do not think we should do so on primitives that you must understand the internal details of to use safely - What we need is a new collection of quality primitives. Hashes without length extension, signing and encrypting where the requirement for a clean nonce or padding is made explicit via the interface. Security properties must be simply stated and clearly documented.

edit for clarity: nacl has solved this problem for the primitives. It's a selection of quality primitives which have the clearest behavior, and are easy to understand with no surprises. This makes protocols built on them is easier to audit. We should use nacl, but build protocols on top of it applying the same philosophy.

Also, building your cryptosystem directly from the primitives is unwise. Even with quality primitives, there are many ways to assemble them, not all safe. Applications often need the same features, secure-channels, authentication, Encrypted storage, asymmetric encryption. These cryptomodules are assembled from the primitives, and likewise, their properties should be simply stated and clearly documented. The design process for a crypto module or protocol should be taken very seriously. Agile does work for crypto. To design a crypto protocol I think you should begin with a survey of prior art, and summarize and document the security properties. This will give you a good idea for the properties that are achiveable, you'll probably think of some new ones too! Document these, and design a protocol that realizes them. Describe the behavior your protocol will have with different kinds of attacks on it. Now you may begin to write code.

Yes, this is basically the waterfall method. Big Design Up Front. However, I am not advocating designing whole systems like this - just modules. What this is really, is a design process optimized for auditability.

I think this should probably be in the form of markdown documents with english descriptions and pseudocode.
A discussion needs to result in a easy to read document - not a back and forth issue discussion (though that is important while producing the document). Also, clear protocol design documents will be useful to developers across languages, not just javascript.

To give a concrete example of the kind of design process I am advocating have a look at what I've done here while designing a secure channel protocol: first, revew of prior art: https://github.com/ssbc/scuttlebot/wiki/secure-private-channels:-the-good,-the-bad,-and-the-ugly
then summary of desirable properties: https://github.com/ssbc/scuttlebot/wiki/desirable-properties-for-a-secure-channel then a description of a protocol that meets all those properties https://github.com/ssbc/scuttlebot/wiki/a-secure-secure-channel

Who is with me?

@calvinmetcalf @dcousens @jprichardson @lsegal @kyledrake @wolfeidau @nathan7 @ahdinosaur @substack @hij1nx @ralphtheninja @jbenet @mafintosh

@dominictarr

This comment has been minimized.

Copy link
Contributor Author

dominictarr commented May 13, 2015

also @pfraze and @kumavis

@kyledrake

This comment has been minimized.

Copy link
Contributor

kyledrake commented May 13, 2015

I'm not really an expert on this, but a review of the philosophy of NaCl may be illustrative here: https://github.com/cryptosphere/rbnacl

@kyledrake

This comment has been minimized.

Copy link
Contributor

kyledrake commented May 13, 2015

I mean really, it would be nice to be able to use NaCl in the browser, but that's the least practical thing I have ever said in my career writing software.

@lsegal

This comment has been minimized.

Copy link
Contributor

lsegal commented May 13, 2015

I agree with the general idea here. Specifically, I wholeheartedly agree that caution is much more important than moving fast when it comes to security and crypto. Getting it right is paramount.

That said, perhaps I might be confused, but I feel like this discussion might be overloaded. IMO this discussion should clearly separate "designing a protocol" from "implementing a protocol". These are two very different things. Implementing AES, or SHA-x, or even TLS, should not require a design document. These protocols have already been designed. Implementing per the existing spec should be the priority, which isn't quite as hefty a process. There are plenty of good reference implementations for many of these. Sure, some of these implementations have bugs, but no software is perfect. Using these references should provide a solid basis for this library.

The mention of "designing protocols" gets me a little concerned-- and scared. Is the goal of this module really to design new protocols? I respect all the minds contributing to this project (especially @dominictarr for stewarding this library, thank you!), but "I can do this better" is usually someone's famous last words when it comes to cryptography. More importantly, from a selfish-I'm-a-user perspective, I'm explicitly uninterested in any new crypto protocol, regardless of how much more amazing it may be. My project requirements require (lack of better term) me to use SHA-256, MD5, TLS, and others, regardless of how broken they might be. If crypto-browserify were to stop supporting those protocols, I wouldn't be able to use this library anymore. If new protocols really is the goal, they should be implemented in a new library. I'd gladly support the work done in this new library! (though "selfish-me" would have little use for it, or time to contribute)

If I'm misunderstanding feel free to correct me (I hope I am...), but I want to see this project continue to implement existing crypto protocols (AES, SHA-1, SHA-256, MD5, TLS) per spec, not going out and inventing new specs. Doing this should not require a heavy handed design doc, as those already exist as RFCs and other documentation. I am, of course, all for doing a more stringent job of ensuring that these existing specs are followed properly.

@dominictarr

This comment has been minimized.

Copy link
Contributor Author

dominictarr commented May 13, 2015

@kyledrake nacl in the browser: https://www.npmjs.com/package/ecma-nacl yes nacl is an inspiration here.

@lsegal sorry yes I am talking about starting a new project; I did not mean to suggest changing crypto-browserify's at all, it's exactly what it should be. I'll edit my issue to make that more clear.

For the record, I'm also not interested in inventing new primitives (hashes, ciphers, key exchanges, etc), but I think there is a need to curate the best of those.

@pfrazee

This comment has been minimized.

Copy link

pfrazee commented May 13, 2015

I'm in favor of having fewer sharp edges on my crypto libraries. It is similar to nacl, how do you plan to differ? Different primitives, different features?

@calvinmetcalf

This comment has been minimized.

Copy link
Member

calvinmetcalf commented May 13, 2015

this would probably fall under some sort of crypto-browserifyable banner but here is as good a place as any to discuss it, some other interesting prior art include open whisper systems and their version of otr.

@dominictarr

This comment has been minimized.

Copy link
Contributor Author

dominictarr commented May 13, 2015

@pfraze we'll most probably use nacl. we don't really need more primitives but we do need higher level things such as a secure-channel, and authentication protocols.

@jbenet

This comment has been minimized.

Copy link

jbenet commented May 13, 2015

@dominictarr I generally agree very strongly with you sentiment here. We should help.

Relevant:

  • I desire to split TLS into two protocols: jbenet/random-ideas#32
  • TLS 1.3 has major improvements towards a simple AEAD transport.
  • QUIC crypto has major improvements towards 0-RTT transport (with multiple stream support)
  • Talk by agl: https://news.yahoo.com/video/yahoo-trust-unconference-tls-adam-223046696.html (not tagging him because he should not be bothered unless we have something very, very concrete).
  • As agl described, making very simple, usable AEAD primitives would be very, very useful.

@kyledrake

it would be nice to be able to use NaCl in the browser, but that's the least practical thing I have ever said in my career writing software.

hahah

((Aside, one might be able make a NaCl NaCl binary so you can salt hashes with salt running on salt. ))


Important note: if i'll contribute time to this, please note i've moved on from language-or-platform-dependent programming. I'd be game for defining library interfaces that are language independent -- meaning, to be implemented in multiple languages -- as idiomatic to those languages (e.g. callbacks in js, etc). (no "oh this works in the browser, so that's enough"). I mention this because to "build things that last" we must think beyond what's in vogue today.

@jbenet

This comment has been minimized.

Copy link

jbenet commented May 13, 2015

@lsegal important to address your comment.

I hear you-- very much understand your concerns. but i think @dominictarr is not asking for new protocols, rather simple, clear, usable implementations of existing, well understood protocols. Most cryptographic primitives are given to users with clear labels of how to use them, and the guarantees they provide. These already very clear labels are just not enough. As both djb and agl have pointed out multiple times, we must have extremely simple abstractions, down to simple functions of "the right thing to do given what we know today". We must not have the user choose a particular construction with a particular cipher in a particular mode -- this invariably leads to failure, as few users follow eprint and the relevant mailing lists, to know about the latest attacks. We must give users abstractions as easy as:

package crypto

func Encrypt(sk Key, plaintext []byte) (ciphertext []byte) { ...}
func Decrypt(sk Key, cipher []byte) (plaintext []byte) { ... }

func Sign(sk Key, data []byte) (sig []byte) { ... }
func Verify(pk Key, data []byte, sig []byte) (ok bool) { ... }

or continue to fool ourselves.

@dominictarr

This comment has been minimized.

Copy link
Contributor Author

dominictarr commented May 13, 2015

@jbenet I think we need to clarify some of the meanings we are using here.

  • "primitives" are hashes, signatures, key exchange, asymmetric encryption, ciphers, macs.
  • "protocols" or "modules" are things created from primitives, so crypto_box is a protocol, tls is a protocol, pbkdf2 is a protocol, because it's build from hashes.

A well designed protocol (like crypto_box) can be treated like a primitive, but it's not really a primitive (it uses key exchange - curve25517, a cipher - Salsa20, and a mac - poly1305) but it combines it's primitives into a simple api (though it could have better documentation).

we don't need new primitives, but we do need new protocols. @jbenet I also need a secure-channel - part 2 of your two layer tls protocol - secure-scuttlebutt has it's own approach to part 1. But I would love to collaborate on the secure-channel part.

@dominictarr dominictarr reopened this May 13, 2015

@dominictarr

This comment has been minimized.

Copy link
Contributor Author

dominictarr commented May 13, 2015

oops, didn't mean to close that

@kumavis

This comment has been minimized.

Copy link

kumavis commented May 14, 2015

@kumavis

This comment has been minimized.

Copy link

kumavis commented May 14, 2015

so how do we start? order some hexagonal stickers?

@wanderer

This comment has been minimized.

Copy link

wanderer commented May 14, 2015

One thing that would be really nice is a good standard implementation of ECIES. It would be awesome to have some standered secure transportation protocols. I'm working on implementing this with @subtly. I would love to see how it compares to @dominictarr secure-channel and ipfs's secure channel.

@kumavis

This comment has been minimized.

Copy link

kumavis commented May 14, 2015

I think this is really important.
crypto is too inaccessible.

I agree with the prior-art review and design-first approach.
I agree with the approach of drawing the lines of modules based on features/use-cases.
I agree with @jbenet's strategy of laying out a language/platform-agnostic spec for crypto modules.

There are a lot of topics to cover here, and we'll need to grow a community to generate easy-to-understand + well-reviewed specs.

so how do we start? order some hexagonal stickers?

By which I mean, what is the first step -- spin up a github org, create repos for different use cases (secure storage, tls), and start compiling research and ideas?

@dominictarr

This comment has been minimized.

Copy link
Contributor Author

dominictarr commented May 14, 2015

@wanderer looks interesting - I will have a look at this. Do you do IRC? I might need to ask you questions.

@kumavis yeah! before we can have stickers or github org we need a name. I've been calling this "modular crypto" to myself... that overloads the meaning of "modular" in the context of cryptography. You could call it a "library" in the sense of books on a shelf... but I don't really want to put everything in one repo (rather have one protocol one repo). More like an indie record label that is a curated selection of works that have a certain vibe. so someone needs to pick a reasonable name.

@kumavis

This comment has been minimized.

Copy link

kumavis commented May 14, 2015

so someone needs to pick a reasonable name.

oh man i've been in so many of these threads this year

@wanderer

This comment has been minimized.

Copy link

wanderer commented May 14, 2015

@dominictarr yeah I actually lurk on #scuttlebutt ;)

@kumavis

This comment has been minimized.

Copy link

kumavis commented May 14, 2015

  • Electronic Locksmiths Guild 🔌
  • CloakRoom 😈
  • The one-time-pals 😵
  • Deadbolt Society ⚡️
  • Key Cutters Consortium 🔑
@kumavis

This comment has been minimized.

Copy link

kumavis commented May 14, 2015

Guild and Consortium do sound a bit nose-in-the-air, we should ideally evoke an open community

@dominictarr

This comment has been minimized.

Copy link
Contributor Author

dominictarr commented May 14, 2015

@kumavis "Audit Driven Crypto" "auditdrivencrypto" ADC.

"audit driven" = do the security audit first, like test driven development.

@kumavis

This comment has been minimized.

Copy link

kumavis commented May 14, 2015

@dominictarr I like it. by naming the org after an approach to crypto development, it invites others to take that approach with us.

@kumavis

This comment has been minimized.

Copy link

kumavis commented May 14, 2015

@dominictarr if you're decently happy with that name I would encourage you to start an org and we can get to the good stuff. can still bike shed and rename org later.

@dominictarr

This comment has been minimized.

Copy link
Contributor Author

dominictarr commented May 14, 2015

@jbenet we need a word for the nonconcrete version of a protocol. I guess the best is "abstract protocol". An "abstract protocol" is based on "abstract primitives" so a hash is just a "hash", specifying the properties but not the implementation. then the implementation (with concrete primitives selected) is the concrete protocol.

@dominictarr

This comment has been minimized.

Copy link
Contributor Author

dominictarr commented May 14, 2015

done. https://github.com/auditdrivencrypto
@kumavis need a hexagon logo.

@dominictarr

This comment has been minimized.

Copy link
Contributor Author

dominictarr commented May 14, 2015

should probably involve a lock with a green tick

@dcousens

This comment has been minimized.

Copy link
Contributor

dcousens commented May 14, 2015

@dominictarr +1 to all the above. Thanks for clarifying the intentions, I wasn't sure what you were proposing at first.

@jprichardson

This comment has been minimized.

Copy link
Member

jprichardson commented May 14, 2015

If we are interested in building new cryptosystems (i am) then I do not think we should do so on primitives that you must understand the internal details of to use safely - What we need is a new collection of quality primitives. Hashes without length extension, signing and encrypting where the requirement for a clean nonce or padding is made explicit via the interface. Security properties must be simply stated and clearly documented.

@dominictarr This is paramount. I completely agree.

@lsegal has some important points...

The mention of "designing protocols" gets me a little concerned-- and scared. Is the goal of this module really to design new protocols? I respect all the minds contributing to this project (especially @dominictarr for stewarding this library, thank you!), but "I can do this better" is usually someone's famous last words when it comes to cryptography.

I share this sentiment. Maybe due to the whole "avoid rolling your own crypto" mantra so often touted, but I can definitely say that it makes me uncomfortable. For the sake of discussion, I'm glad that @dominictarr clarified the usage of cryptographic "primitives" vs "protocols". It sounds like this discussion is all about designing new protocols? When it comes to proving the security of cryptographic protocols (not primitives), what is the proper approach? Perhaps someone can point me to some resources (outside of a cursory Google search) that would help to shed some light?

Since this new organization is primarily about curated cryptographic primitives and new protocols, will this organization include any code or is it all about design? Overall, It definitely seems like writing up draft documents/specs is a reasonable start.

@pfrazee

This comment has been minimized.

Copy link

pfrazee commented May 14, 2015

@jprichardson let me throw my take on this

Dominic is working on the security features of https://github.com/ssb/secure-scuttlebutt, which involves authentication, private channels, and blob encryption. He's doing his research and this project is how that's congealed

in secure-channel you can see in the desired properties and prior art what the goals are and why we dont use TLS/SSH/etc as a matter of course.

The goal's not to design protocols, it's to make informed decisions and properly document them, but new protocols may result. What would worry me is choosing TLS because it's a standard. What may happen is we choose TLS because it's the right choice for the crypto-system

@subtly

This comment has been minimized.

Copy link

subtly commented May 14, 2015

I'm in.

It might be good to start with something simple and iteratively bring the secure-channel draft into compliance with the format/process. I would add one thing -- authors have veto. I'm not sure how that would work but it would be great to have some system for building this process around the author rather than the author conforming to the process. Then figure out how to integrate the two. I suppose this would mean an effort would have two specifications -- the author's and a specification which conforms to this process. That is, that the process has been followed, but not necessarily by the author.

Writing clean and easy to use software is difficult, let alone integrating cryptography. I think a project which exposes cryptography as a useable system deserves as much appreciation as the underlying cryptography. Applied cryptography is an application and I feel like this discussion is about applied cryptography. It looks like there's already a name but something along the lines of 'Applied Cryptography' would be nice (assuming no copyright conflict exists).

I'd just like to say that TLS/X.509 are legit. If TLS was tweaked just a bit, and it's implementations were a bit easier to manage, we might not be having this conversation. TLS is an excellent example of just how difficult it is to build a system which doesn't define primitives.

@dominictarr I do IRC. I would be happy to help @wanderer to build libraries which helps facilitate systems like scuttle-butt, ipfs, ethereum, etc. At a base level, all of these projects need the same sorts of codes and terminology -- key management, encoding/decoding, portability, authentication, etc.

Re: AEAD
http://competitions.cr.yp.to/caesar.html

@dominictarr

This comment has been minimized.

Copy link
Contributor Author

dominictarr commented May 15, 2015

@subtly names are a classic bikeshed, and Applied Cryptography is already the title of a classic textbook

@jprichardson absolutely. If we are gonna challenge the taboo of creating new crypto (protocols) then we have to do it right. In a few years of learning about cryptography (from casual to pouring over classic papers) I havn't really encountered very much theory about how to design a protocol at all. But, there is a lot written about design process for regular software engineering.

I think we can transpose some of those concepts into crypto. For example, you would design your module or application using test driven development - well, to design a crypto protocol the test is the audit. So, we can do audit driven development. What does that look like? I think we write up an abstract description of the abstract protocol with pseudo code (we don't need to talk about byte offsets, or specific primitives, just "hash", "box" or "signature") - we then describe the operation of that protocol, in it's correct operation, but also what happens in the case of a replay attack, a man in the middle attack, and key compromise. Describe what an eavesdropper, replayer, middleman, and later key compromizer can learn from the protocol. It's essential to know what behavior occurs in the various attacks or failure modes, and what information can be extracted.

During the process we may learn about other failure modes, or other properties, in that case, iterate.

To start, study prior art. This lets come up with list of what properties are possible/desireable. You may have some ideas for properties that no current property supports (an example is that no secure-channel doesn't let a replay attacker confirm the server key - even in curvecp!)

Most crypto protocols are not documented to this level of details - A description of the correct operation is common enough but I think we need the How, What, and Why to really be spelled out. To optimize the design process for auditing, like in test driven development you optimist for testability.

this is not rolling "your own crypto" it's rolling "our own crypto".

@dcousens

This comment has been minimized.

Copy link
Contributor

dcousens commented Aug 13, 2015

@dominictarr has this discussion been realized in any way? Perhaps a new repository to trial out ideas about what it is we want to do?

@ahdinosaur

This comment has been minimized.

Copy link

ahdinosaur commented Aug 13, 2015

@dcousens you can see the progress that followed from this discussion in auditdrivencrypto, dominictarr/secret-handshake-paper, dominictarr/secret-handshake, ssbc/secret-stack, dominictarr/pull-box-stream, and probably heaps more that i'm missing.

@dcousens

This comment has been minimized.

Copy link
Contributor

dcousens commented Aug 13, 2015

Awesome! Thanks @ahdinosaur

I might close this issue then. @dominictarr please re-open if you feel it is relevant.

@dcousens dcousens closed this Aug 13, 2015

@dominictarr

This comment has been minimized.

Copy link
Contributor Author

dominictarr commented Aug 13, 2015

yes close this, we have created https://github.com/auditdrivencrypto

On Thu, Aug 13, 2015 at 8:23 AM, Daniel Cousens notifications@github.com
wrote:

Awesome! Thanks @ahdinosaur https://github.com/ahdinosaur

I might close this issue then. @dominictarr
https://github.com/dominictarr please re-open if you feel it is
relevant.


Reply to this email directly or view it on GitHub
#128 (comment)
.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.