Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SSL support #43

Closed
anteph opened this issue Apr 3, 2015 · 155 comments
Closed

SSL support #43

anteph opened this issue Apr 3, 2015 · 155 comments

Comments

@anteph
Copy link

@anteph anteph commented Apr 3, 2015

Hi!
I would like to know if you plan to include SSL support in the libraries.
I've tryed to send some https requests with no sucess.

It would be cool if it the ESP could act as a secure server too.

I've checked the some examples from the sdk and they have a code to create a secure server with digital certificate.

Thanks!

@abl
Copy link

@abl abl commented Apr 22, 2015

SSL has been ~broken since 0.9.3-patch1 (and you needed a new libssl, separate from patch1, from Espressif.)

It's supposedly fixed again in 1.0.1b2 (you still need a special libssl for the moment) - http://bbs.espressif.com/viewtopic.php?f=5&t=382 - I haven't had a chance to try it yet, but I've heard good reports.

There are still some features missing for practical SSL - iirc there's no way to pin a certificate or do trust validation, it'll silently accept self-signed certs - but it's still a huge positive step.

@Links2004
Copy link
Collaborator

@Links2004 Links2004 commented Apr 23, 2015

i have add the latest sdk + ssl patches
see here:
Links2004@af6c400
pull request is waiting for merged (#93)

@igrr
Copy link
Member

@igrr igrr commented Apr 23, 2015

Merged that, thanks.
Just one (obvious) thing to note: adding SSL libraries (axTLS) is one thing, but to make them useful we also need to add SSL support to WiFiClient/WiFiServer.

@igrr igrr mentioned this issue May 20, 2015
@vicatcu
Copy link

@vicatcu vicatcu commented Jun 17, 2015

@igrr does this issue cover adding HTTPS support to a WebClient application for accessing secure-connection-only APIs? What's the roadmap/timeline for including this feature? Is it actually possible?

@vicatcu
Copy link

@vicatcu vicatcu commented Jun 27, 2015

@Links2004 @igrr Do I read this thread correctly that there is support for an HTTPS WebClient built into the core now, but WiFiClient class "just" needs that core functionality integrated into it?

@abl
Copy link

@abl abl commented Jun 27, 2015

@vicatcu:

does this issue cover adding HTTPS support to a WebClient application for accessing secure-connection-only APIs?

Yes - HTTPS is just HTTP over SSL sockets, so once SSL sockets are integrated with WiFiClient/WiFiServer, you should be able to talk to HTTPS servers.

What's the roadmap/timeline for including this feature?

I'm not very familiar with this project's codebase.

Is it actually possible?

Yes. Other projects (like https://github.com/tuanpmt/esp_bridge) are able to make HTTPS requests from an ESP8266. The primary limiting factor is certificate size and protocol support (TLS v1.2 is not yet supported and is mandatory for some servers) - and that the Espressif SDK doesn't always have working SSL built in to it (it works now!)

Do I read this thread correctly that there is support for an HTTPS WebClient built into the core now, but WiFiClient class "just" needs that core functionality integrated into it?

It's now possible to write a WiFiClient that opens a secure socket. The existing WebClient shouldn't need major changes.

@vicatcu
Copy link

@vicatcu vicatcu commented Jun 27, 2015

@abl cool, I need to come up to speed on on getting a dev environment set up so I can contribute...

@glennalgie
Copy link

@glennalgie glennalgie commented Jul 20, 2015

looking forward to SSL support as well for ESP8266. I am looking at using Golgi.io arduino stack as an option for the SSL shim as they provide other time saving and mission critical networking support functions.

@vicatcu
Copy link

@vicatcu vicatcu commented Jul 25, 2015

@igrr @Links2004 can you give any pointers on how to use axTLS to implement HTTPS capability into WiFiClient/WiFiServer? Any updates on progress here would be appreciated. I personally would like to see the WiFiClient capability implemented as the priority. Would it be beneficial splitting this issue into two; one for WiFiClient and one for WiFiServer?

@igrr
Copy link
Member

@igrr igrr commented Jul 25, 2015

@vicatcu
WiFiClient needs only TLS, not HTTPS. You can use WiFiClient for things other than HTTP — SSH and SMTP to name a few. The way to do this is to create a new class, let's say WiFiSecureClient, and override read/write methods of WiFiClient to pump data through TLS engine. Some additional methods for certificate management should be added as well. I'm not sure if there is anything that needs to be done on the server side, since WiFiServer's only job is to create a WiFiClient instance for every new connection. I suppose someone more familiar with TLS can comment on this.

@vicatcu
Copy link

@vicatcu vicatcu commented Jul 25, 2015

@igrr ah yes of course, that (that WiFiClient is not just about HTTP) makes a lot of sense.

@fmgomes
Copy link

@fmgomes fmgomes commented Jul 30, 2015

@vicatcu , have you started to implement the TLS on the WiFiClient? I need to connect to a webserver using https, and I'm interested in this functionality, and could try to help (don't know if I am able to do it, no previous experience with HTTPS or TLS).

Best regards

Fernando

@vicatcu
Copy link

@vicatcu vicatcu commented Jul 30, 2015

@fmgomes honestly I haven't really gotten anywhere yet - it's a pretty steep learning curve for me so I'm kind of in the same boat as you :-/. It's a bit frustrating to know it's possible for the last few months, but that I haven't had time / know-how / etc to help make it happen. @igrr spelled out what needs to happen pretty well in his last response, but I'm not sure what "certificate management" requires...

@abl
Copy link

@abl abl commented Jul 30, 2015

When a client connects to a server and performs a TLS handshake (which is what turns a socket in to a secure socket, and HTTPS is just HTTP on a secure socket) the server sends over a certificate. The certificate is what identifies the server; without the certificate validation step, HTTPS prevents against eavesdropping but you could potentially be communicating with an attacker (aka a MitM attack.)

Validating a certificate:

  1. Check to see if the name of the server you're trying to connect to matches the server name on the certificate.
  2. Check to see if the certificate has expired.
  3. Check to see if the certificate was issued by a trusted CA.
  4. Check to see if the certificate has been revoked.

1 should be pretty easy to implement; the names should be exactly equal. Wildcard certificates exist but are considered harmful; not supporting them is probably fine for v1.

2 can probably be ignored for now; the vast majority of ESP8266 work will be bound to specific certificates because...

3 is the most complex part. On a normal computer, the OS (and sometimes the browser) maintains a list of trusted root certificates. For Debian, the size of this certificate package is 502kB (see https://packages.debian.org/sid/ca-certificates) - it's possible to shrink this down a bit but it's obviously a nonstarter for the average ESP8266 project. An alternative to this is simply maintaining a list of certificate fingerprints and checking that instead - creating an explicit whitelist of certificates. That's what I'd recommend - and when you're using certificate fingerprints, you can be lax about expiration and revocation and manage it yourself.

4 is a complicated topic - check out CRL and OCSP if you're curious - and way beyond what we'd want to do in an embedded system.

AxTLS probably has simple-ish calls to handle all of this; they'd need to be exposed via SecureWifiClient.

SecureWifiServer would need a new method that allows a user to set the public and private key to use.

@mtnbrit
Copy link

@mtnbrit mtnbrit commented Jul 30, 2015

does thread this also cover applying TLS to MQTT eventually?

@anteph
Copy link
Author

@anteph anteph commented Aug 2, 2015

For what I've read, adding https support is a little bit hard. But what about simple ciphering, for direct socket communication? Something with a symmetric key, that would be a good start.

I've read in the Espressif page that ESP8266 has a built in AES engine, but they don't specify if it is hardware or software based. Has anyone tried it?

@torntrousers
Copy link
Contributor

@torntrousers torntrousers commented Aug 2, 2015

Just a comment on that list of work from @abl - to get going i'd be quite happy to not have any of that certificate validation stuff. Wont be sending my bank account details or anything top secret just need to post to a remote service that only supports https. So just binning the certificate after its read would be fine for my uses.

@drmpf
Copy link

@drmpf drmpf commented Aug 2, 2015

If you just want to prevent un-authorized use rather then hid the messages, check out this page on using SipHash as a secure hash.
http://www.forward.com.au/pfod/secureChallengeResponse/index.html

@cottsak
Copy link

@cottsak cottsak commented Aug 28, 2015

So what's the latest?

  • Can I make a HTTPS request?
  • can I use TLS 1.2?
  • can I pin a certificate (avoid the CA lookups)?
@jacealot
Copy link

@jacealot jacealot commented Aug 28, 2015

Is it now possible to make a HTTPS request?
Or are there any libraries out there that I could use?

@igrr
Copy link
Member

@igrr igrr commented Aug 31, 2015

Started working on TLS support: https://github.com/igrr/axtls-8266
This is the same library that Espressif uses for their libssl, so TLS 1.0/1.1 only.

igrr added a commit that referenced this issue Sep 1, 2015
@abl
Copy link

@abl abl commented Sep 2, 2015

Going that route, it seems like wolfSSL and mbed TLS (well, especially mbed TLS) might be easier to bring in, although we know axTLS works since Espressif has it working :)

They also support TLS 1.2.

@igrr
Copy link
Member

@igrr igrr commented Sep 2, 2015

Both wolfSSL and PolarSSL (aka mbed TLS) are GPL (not LGPL) unless you
apply for a commercial license. I would like to keep the library
LGPL-compatible.

On Wed, Sep 2, 2015, 21:24 Alexander notifications@github.com wrote:

Going that route, it seems like wolfSSL and mbed TLS (well, especially
mbed TLS) might be easier to bring in, although we know axTLS works since
Espressif has it working :)

They also support TLS 1.2.


Reply to this email directly or view it on GitHub
#43 (comment).

@madsci1016
Copy link

@madsci1016 madsci1016 commented Mar 7, 2016

@igrr one of your future milestones was "release memory allocated for certificate storage after check is complete" has that been implemented yet?

I'm now running into a memory issue:

State:    receiving Certificate (11)
crypto/bigint.c:1072 realloc 1032, left 15480
crypto/bigint.c:1072 realloc 1032, left 11944
crypto/bigint.c:1072 realloc 1028, left 7584
crypto/bigint.c:1072 realloc 2056, left 5424
crypto/bigint.c:1072 realloc 1032, left 4344
crypto/bigint.c:1072 realloc 1024, left 3832
crypto/bigint.c:1072 realloc 1032, left 1048
crypto/bigint.c:1072 realloc 512 failed, left 752
Fatal exception 29(StoreProhibitedCause):

I try to malloc them memory I need after the handshake, but at that point ESP.getFreeHeap() only shows 12KB available. Before the client.conneect call it is 27KB+. I

@igrr
Copy link
Member

@igrr igrr commented Mar 7, 2016

It is implemented in 2.1.0. After WiFiClient::connect returns, certificate will be kept until the first call to read or write functions. This is done to allow one call verify function and check the certificate. Once you call read or write, certificate will be released.

@apigeoje
Copy link

@apigeoje apigeoje commented Mar 7, 2016

@igrr - You're frikin awesome. I've just spent 15 minutes reading this. I haven't even tried to use it yet but I'm already grateful for all your work...

@madsci1016
Copy link

@madsci1016 madsci1016 commented Mar 7, 2016

Yes, @igrr is super awesome and needs to show us a paypal donate button so we can show our appreciation.

I've added some of my own debug as well as turned on the builtin. Here's what I got:

`Free Mem before connection: 38536

ssl/tls1.c:549 malloc 6864, left 30960

please start sntp first !

State: sending Client Hello (1)

State: receiving Server Hello (2)

State: receiving Certificate (11)

crypto/bigint.c:1072 realloc 1032, left 26584

crypto/bigint.c:1072 realloc 1032, left 23048

crypto/bigint.c:1072 realloc 1028, left 18688

crypto/bigint.c:1072 realloc 2056, left 16528

crypto/bigint.c:1072 realloc 1032, left 15448

crypto/bigint.c:1072 realloc 1024, left 14936

crypto/bigint.c:1072 realloc 1032, left 12024

State: receiving Server Hello Done (14)

crypto/bigint.c:1072 realloc 1024, left 8432

State: sending Client Key Exchange (16)

State: sending Finished (16)

State: receiving Finished (16)

Free Mem after handshake: 12184

ssl/tls1.c:1422 malloc 17408, left 19280

Free Mem after print: 19648

Response: :rd 5, 330, 0

Client closing connection

Alert: close notify`

So with 38KB free originally, 30KB are consumed during the handshake and 4KB are given back right away, leaving the user with 12KB free. After the first write call, an additional 7KB are given back, bringing free mem to 19KB with the connection open.

@igrr Is this the expected behavior? Are there any other improvements you could suggest on the server end to optimize memory usage?

My application is a IoT sensor that will include a 1 or 2 second WAV file recorded by the ESP8266, That's why I'm being a stickler for memory.

@krebbi
Copy link

@krebbi krebbi commented Mar 10, 2016

is there a known issue with Let's Encrypt certificates?

because I get

please start sntp first !
State: sending Client Hello (1)
Alert: handshake failure
Error: SSL error 40
Alert: unexpected message
Alert: close notify

@igrr
Copy link
Member

@igrr igrr commented Mar 10, 2016

@krebbi does the server allow TLS 1.1? axTLS doesn't support TLS 1.2, which may be the reason for this error.

@krebbi
Copy link

@krebbi krebbi commented Mar 10, 2016

TLS 1.0, 1.1 and 1.2
https://api.12view.me

@krebbi
Copy link

@krebbi krebbi commented Mar 10, 2016

or is RSA 4096 bits too big?

@madsci1016
Copy link

@madsci1016 madsci1016 commented Mar 10, 2016

I believe this ESP version only supports AES 128 or 256

@krebbi
Copy link

@krebbi krebbi commented Mar 11, 2016

these Cipher Suites are supported by my cert:

TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030) ECDH secp256r1 (eq. 3072 bits RSA) FS 256
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f) ECDH secp256r1 (eq. 3072 bits RSA) FS 128
TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x9f) DH 2048 bits FS 256
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x9e) DH 2048 bits FS 128
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028) ECDH secp256r1 (eq. 3072 bits RSA) FS 256
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014) ECDH secp256r1 (eq. 3072 bits RSA) FS 256
TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 (0x6b) DH 2048 bits FS 256
TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x39) DH 2048 bits FS 256

@slaff
Copy link

@slaff slaff commented Mar 18, 2016

@igrr Do you know if NGINX supports max_fragment_length negotiation ?

@electronicsguy
Copy link

@electronicsguy electronicsguy commented Mar 29, 2016

@igrr Could someone please tell me if https can work on the ESP-01?

@svdgraaf
Copy link

@svdgraaf svdgraaf commented Mar 29, 2016

@electronicsguy some https connections will work, some won't. It has to do with the TLS version, the used ciphers, and/or the configuration of the webserver.

@slaff
Copy link

@slaff slaff commented Mar 31, 2016

..." Nginx hardcodes 16KB size in ngx_event_openssl, which you can change and recompile from source."

Do you know if NGINX supports max_fragment_length negotiation ?

It turns out that you can set SSL fragment size in NGINX (>= 1.5.9 ). The directive is called ssl_buffer_size. And there is no need to recompile NGINX.

@electronicsguy
Copy link

@electronicsguy electronicsguy commented Apr 1, 2016

@svdgraaf @igrr Thanks for the follow-up. I am still trying to understand the details. All I want to do is send data from esp8266 directly into a google spreadsheet (without using a http to https converter service like pushingbox). So is incorporating support for TLS1.2 unachieveable with the current esp hardware? Or is it a matter of someone writing up that code, which has just not been done as yet?

@AdamMiltonBarker
Copy link

@AdamMiltonBarker AdamMiltonBarker commented Apr 11, 2016

Anyone got TLS1.2 working yet and a way to add trusted certs ?

@fsommer1968
Copy link

@fsommer1968 fsommer1968 commented Apr 16, 2016

@AdamMiltonBarker!
Have a look at WolfSSL, they support TLS V1.2 but I´m not sure whether the library works with ESP. I want to use TLS 1.2 with Bluemix IoT but I have currently not the time to test. Maybe you can try?

@marvinroger
Copy link
Contributor

@marvinroger marvinroger commented Apr 16, 2016

wolfSSL has already been discussed, and the problem is it is GPL and not LGPL. I am not sure I understand the difference, to be honest.

@fsommer1968
Copy link

@fsommer1968 fsommer1968 commented Apr 16, 2016

For my project it would´nt be important whether GPL or LGPL.

@AdamMiltonBarker
Copy link

@AdamMiltonBarker AdamMiltonBarker commented Apr 18, 2016

I agree I think security is more important that the license.

@DaKaZ
Copy link

@DaKaZ DaKaZ commented May 5, 2016

@igrr - I too greatly appreciate all you do for the esp8266 community. Having been developing on this hardware for the last year I could not have done it without you!

I am having trouble with the WiFiClientSecure implementation though. I basically have the exact same code for HTTP as HTTPS but I never get the response back from the server with SSL (unless I remove connection: close from the http headers and wait for a timeout). I have verified that the server is indeed sending a response back (wireshark). The code is here: https://gist.github.com/DaKaZ/5044f3168412ce0b3f5bf1d46130765e

From my debug output, I see the Alert: close notify before I even call sslClient.connected() or sslClient.available() so I fear I am missing something. Any ideas?? Here is a little more from my client debug:

HTTP: connect
please start sntp first !
State:  sending Client Hello (1)
State:  receiving Server Hello (2)
State:  receiving Certificate (11)
State:  receiving Server Hello Done (14)
State:  sending Client Key Exchange (16)
State:  sending Finished (16)
State:  receiving Finished (16)
Verifiying SSL certificate
SSL certificate matches
HTTP: connected
REQUEST: 

SSL Print: POST
SSL Print:  
SSL Print: /api/v1/plugs
SSL Print:  HTTP/1.1

SSL Print: Accept-Encoding: application/json
SSL Print: 

SSL Print: Timestamp: 1462488709
SSL Print: 

SSL Print: Signature: <removed for online post>
SSL Print: 

SSL Print: Host: 
SSL Print: <removed for online post>
SSL Print: 

SSL Print: Connection: close

SSL Print: Content-Length: 31

SSL Print: Content-Type: 
SSL Print: application/json
SSL Print: 

SSL Print: 

SSL Print: { "mac" : "<removed for online post>" }
SSL Print: 

SSL Print: 
Alert: close notify

END REQUEST
HTTP: call readResponse
HTTP: NON-NULL RESPONSE POINTER: 
HTTP: RESPONSE: 
HTTP: Connect: 0 Available: 0
HTTPS client closed 
HTTP: return readResponse3
HTTP: return readResponse
HTTP: stop client
HTTP: client stopped
Got response: 
pm open,type:2 0
@DaKaZ
Copy link

@DaKaZ DaKaZ commented May 6, 2016

Very interesting... thanks @chaeplin - So I changed the multiple write(...); lines into one string just called write(newString.c_str()); once and now I get the response. Not sure why but its working thats to your reference!!

@DaKaZ
Copy link

@DaKaZ DaKaZ commented May 6, 2016

If anyone comes across this thread - here is a working version of the RestClient with SSL support: https://github.com/DaKaZ/esp8266-restclient

Enjoy

@Potato-Matic
Copy link

@Potato-Matic Potato-Matic commented Jul 18, 2016

If there's an issue storing certificates in RAM, is it possible you could stream them into the file system and walk through them there? (Note: I've also seen a hack to upgrade the flash to 4M or bigger, so you might be able to store a fair bit)

@AdamMiltonBarker
Copy link

@AdamMiltonBarker AdamMiltonBarker commented Jul 18, 2016

What the plans for secure server? I feel this is a major let down, if we can provide secure endpoints to customers on the ESP I think it would be a massive improvement.

@igrr
Copy link
Member

@igrr igrr commented Jul 18, 2016

@esp8266 esp8266 locked and limited conversation to collaborators Jul 18, 2016
fuzzball03 referenced this issue in fuzzball03/esp-link Nov 29, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.