New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SSL support #43

Closed
anteph opened this Issue Apr 3, 2015 · 155 comments

Comments

Projects
None yet
@anteph

anteph commented Apr 3, 2015

Hi!
I would like to know if you plan to include SSL support in the libraries.
I've tryed to send some https requests with no sucess.

It would be cool if it the ESP could act as a secure server too.

I've checked the some examples from the sdk and they have a code to create a secure server with digital certificate.

Thanks!

@abl

This comment has been minimized.

Show comment
Hide comment
@abl

abl Apr 22, 2015

SSL has been ~broken since 0.9.3-patch1 (and you needed a new libssl, separate from patch1, from Espressif.)

It's supposedly fixed again in 1.0.1b2 (you still need a special libssl for the moment) - http://bbs.espressif.com/viewtopic.php?f=5&t=382 - I haven't had a chance to try it yet, but I've heard good reports.

There are still some features missing for practical SSL - iirc there's no way to pin a certificate or do trust validation, it'll silently accept self-signed certs - but it's still a huge positive step.

abl commented Apr 22, 2015

SSL has been ~broken since 0.9.3-patch1 (and you needed a new libssl, separate from patch1, from Espressif.)

It's supposedly fixed again in 1.0.1b2 (you still need a special libssl for the moment) - http://bbs.espressif.com/viewtopic.php?f=5&t=382 - I haven't had a chance to try it yet, but I've heard good reports.

There are still some features missing for practical SSL - iirc there's no way to pin a certificate or do trust validation, it'll silently accept self-signed certs - but it's still a huge positive step.

@Links2004

This comment has been minimized.

Show comment
Hide comment
@Links2004

Links2004 Apr 23, 2015

Collaborator

i have add the latest sdk + ssl patches
see here:
Links2004@af6c400
pull request is waiting for merged (#93)

Collaborator

Links2004 commented Apr 23, 2015

i have add the latest sdk + ssl patches
see here:
Links2004@af6c400
pull request is waiting for merged (#93)

@igrr

This comment has been minimized.

Show comment
Hide comment
@igrr

igrr Apr 23, 2015

Member

Merged that, thanks.
Just one (obvious) thing to note: adding SSL libraries (axTLS) is one thing, but to make them useful we also need to add SSL support to WiFiClient/WiFiServer.

Member

igrr commented Apr 23, 2015

Merged that, thanks.
Just one (obvious) thing to note: adding SSL libraries (axTLS) is one thing, but to make them useful we also need to add SSL support to WiFiClient/WiFiServer.

@igrr igrr referenced this issue May 20, 2015

Closed

TLS support? #286

@vicatcu

This comment has been minimized.

Show comment
Hide comment
@vicatcu

vicatcu Jun 17, 2015

@igrr does this issue cover adding HTTPS support to a WebClient application for accessing secure-connection-only APIs? What's the roadmap/timeline for including this feature? Is it actually possible?

vicatcu commented Jun 17, 2015

@igrr does this issue cover adding HTTPS support to a WebClient application for accessing secure-connection-only APIs? What's the roadmap/timeline for including this feature? Is it actually possible?

@Imroy Imroy referenced this issue Jun 25, 2015

Open

SSL Support #18

@vicatcu

This comment has been minimized.

Show comment
Hide comment
@vicatcu

vicatcu Jun 27, 2015

@Links2004 @igrr Do I read this thread correctly that there is support for an HTTPS WebClient built into the core now, but WiFiClient class "just" needs that core functionality integrated into it?

vicatcu commented Jun 27, 2015

@Links2004 @igrr Do I read this thread correctly that there is support for an HTTPS WebClient built into the core now, but WiFiClient class "just" needs that core functionality integrated into it?

@abl

This comment has been minimized.

Show comment
Hide comment
@abl

abl Jun 27, 2015

@vicatcu:

does this issue cover adding HTTPS support to a WebClient application for accessing secure-connection-only APIs?

Yes - HTTPS is just HTTP over SSL sockets, so once SSL sockets are integrated with WiFiClient/WiFiServer, you should be able to talk to HTTPS servers.

What's the roadmap/timeline for including this feature?

I'm not very familiar with this project's codebase.

Is it actually possible?

Yes. Other projects (like https://github.com/tuanpmt/esp_bridge) are able to make HTTPS requests from an ESP8266. The primary limiting factor is certificate size and protocol support (TLS v1.2 is not yet supported and is mandatory for some servers) - and that the Espressif SDK doesn't always have working SSL built in to it (it works now!)

Do I read this thread correctly that there is support for an HTTPS WebClient built into the core now, but WiFiClient class "just" needs that core functionality integrated into it?

It's now possible to write a WiFiClient that opens a secure socket. The existing WebClient shouldn't need major changes.

abl commented Jun 27, 2015

@vicatcu:

does this issue cover adding HTTPS support to a WebClient application for accessing secure-connection-only APIs?

Yes - HTTPS is just HTTP over SSL sockets, so once SSL sockets are integrated with WiFiClient/WiFiServer, you should be able to talk to HTTPS servers.

What's the roadmap/timeline for including this feature?

I'm not very familiar with this project's codebase.

Is it actually possible?

Yes. Other projects (like https://github.com/tuanpmt/esp_bridge) are able to make HTTPS requests from an ESP8266. The primary limiting factor is certificate size and protocol support (TLS v1.2 is not yet supported and is mandatory for some servers) - and that the Espressif SDK doesn't always have working SSL built in to it (it works now!)

Do I read this thread correctly that there is support for an HTTPS WebClient built into the core now, but WiFiClient class "just" needs that core functionality integrated into it?

It's now possible to write a WiFiClient that opens a secure socket. The existing WebClient shouldn't need major changes.

@vicatcu

This comment has been minimized.

Show comment
Hide comment
@vicatcu

vicatcu Jun 27, 2015

@abl cool, I need to come up to speed on on getting a dev environment set up so I can contribute...

vicatcu commented Jun 27, 2015

@abl cool, I need to come up to speed on on getting a dev environment set up so I can contribute...

@glennalgie

This comment has been minimized.

Show comment
Hide comment
@glennalgie

glennalgie Jul 20, 2015

looking forward to SSL support as well for ESP8266. I am looking at using Golgi.io arduino stack as an option for the SSL shim as they provide other time saving and mission critical networking support functions.

glennalgie commented Jul 20, 2015

looking forward to SSL support as well for ESP8266. I am looking at using Golgi.io arduino stack as an option for the SSL shim as they provide other time saving and mission critical networking support functions.

@vicatcu

This comment has been minimized.

Show comment
Hide comment
@vicatcu

vicatcu Jul 25, 2015

@igrr @Links2004 can you give any pointers on how to use axTLS to implement HTTPS capability into WiFiClient/WiFiServer? Any updates on progress here would be appreciated. I personally would like to see the WiFiClient capability implemented as the priority. Would it be beneficial splitting this issue into two; one for WiFiClient and one for WiFiServer?

vicatcu commented Jul 25, 2015

@igrr @Links2004 can you give any pointers on how to use axTLS to implement HTTPS capability into WiFiClient/WiFiServer? Any updates on progress here would be appreciated. I personally would like to see the WiFiClient capability implemented as the priority. Would it be beneficial splitting this issue into two; one for WiFiClient and one for WiFiServer?

@igrr

This comment has been minimized.

Show comment
Hide comment
@igrr

igrr Jul 25, 2015

Member

@vicatcu
WiFiClient needs only TLS, not HTTPS. You can use WiFiClient for things other than HTTP — SSH and SMTP to name a few. The way to do this is to create a new class, let's say WiFiSecureClient, and override read/write methods of WiFiClient to pump data through TLS engine. Some additional methods for certificate management should be added as well. I'm not sure if there is anything that needs to be done on the server side, since WiFiServer's only job is to create a WiFiClient instance for every new connection. I suppose someone more familiar with TLS can comment on this.

Member

igrr commented Jul 25, 2015

@vicatcu
WiFiClient needs only TLS, not HTTPS. You can use WiFiClient for things other than HTTP — SSH and SMTP to name a few. The way to do this is to create a new class, let's say WiFiSecureClient, and override read/write methods of WiFiClient to pump data through TLS engine. Some additional methods for certificate management should be added as well. I'm not sure if there is anything that needs to be done on the server side, since WiFiServer's only job is to create a WiFiClient instance for every new connection. I suppose someone more familiar with TLS can comment on this.

@vicatcu

This comment has been minimized.

Show comment
Hide comment
@vicatcu

vicatcu Jul 25, 2015

@igrr ah yes of course, that (that WiFiClient is not just about HTTP) makes a lot of sense.

vicatcu commented Jul 25, 2015

@igrr ah yes of course, that (that WiFiClient is not just about HTTP) makes a lot of sense.

@fmgomes

This comment has been minimized.

Show comment
Hide comment
@fmgomes

fmgomes Jul 30, 2015

@vicatcu , have you started to implement the TLS on the WiFiClient? I need to connect to a webserver using https, and I'm interested in this functionality, and could try to help (don't know if I am able to do it, no previous experience with HTTPS or TLS).

Best regards

Fernando

fmgomes commented Jul 30, 2015

@vicatcu , have you started to implement the TLS on the WiFiClient? I need to connect to a webserver using https, and I'm interested in this functionality, and could try to help (don't know if I am able to do it, no previous experience with HTTPS or TLS).

Best regards

Fernando

@vicatcu

This comment has been minimized.

Show comment
Hide comment
@vicatcu

vicatcu Jul 30, 2015

@fmgomes honestly I haven't really gotten anywhere yet - it's a pretty steep learning curve for me so I'm kind of in the same boat as you :-/. It's a bit frustrating to know it's possible for the last few months, but that I haven't had time / know-how / etc to help make it happen. @igrr spelled out what needs to happen pretty well in his last response, but I'm not sure what "certificate management" requires...

vicatcu commented Jul 30, 2015

@fmgomes honestly I haven't really gotten anywhere yet - it's a pretty steep learning curve for me so I'm kind of in the same boat as you :-/. It's a bit frustrating to know it's possible for the last few months, but that I haven't had time / know-how / etc to help make it happen. @igrr spelled out what needs to happen pretty well in his last response, but I'm not sure what "certificate management" requires...

@abl

This comment has been minimized.

Show comment
Hide comment
@abl

abl Jul 30, 2015

When a client connects to a server and performs a TLS handshake (which is what turns a socket in to a secure socket, and HTTPS is just HTTP on a secure socket) the server sends over a certificate. The certificate is what identifies the server; without the certificate validation step, HTTPS prevents against eavesdropping but you could potentially be communicating with an attacker (aka a MitM attack.)

Validating a certificate:

  1. Check to see if the name of the server you're trying to connect to matches the server name on the certificate.
  2. Check to see if the certificate has expired.
  3. Check to see if the certificate was issued by a trusted CA.
  4. Check to see if the certificate has been revoked.

1 should be pretty easy to implement; the names should be exactly equal. Wildcard certificates exist but are considered harmful; not supporting them is probably fine for v1.

2 can probably be ignored for now; the vast majority of ESP8266 work will be bound to specific certificates because...

3 is the most complex part. On a normal computer, the OS (and sometimes the browser) maintains a list of trusted root certificates. For Debian, the size of this certificate package is 502kB (see https://packages.debian.org/sid/ca-certificates) - it's possible to shrink this down a bit but it's obviously a nonstarter for the average ESP8266 project. An alternative to this is simply maintaining a list of certificate fingerprints and checking that instead - creating an explicit whitelist of certificates. That's what I'd recommend - and when you're using certificate fingerprints, you can be lax about expiration and revocation and manage it yourself.

4 is a complicated topic - check out CRL and OCSP if you're curious - and way beyond what we'd want to do in an embedded system.

AxTLS probably has simple-ish calls to handle all of this; they'd need to be exposed via SecureWifiClient.

SecureWifiServer would need a new method that allows a user to set the public and private key to use.

abl commented Jul 30, 2015

When a client connects to a server and performs a TLS handshake (which is what turns a socket in to a secure socket, and HTTPS is just HTTP on a secure socket) the server sends over a certificate. The certificate is what identifies the server; without the certificate validation step, HTTPS prevents against eavesdropping but you could potentially be communicating with an attacker (aka a MitM attack.)

Validating a certificate:

  1. Check to see if the name of the server you're trying to connect to matches the server name on the certificate.
  2. Check to see if the certificate has expired.
  3. Check to see if the certificate was issued by a trusted CA.
  4. Check to see if the certificate has been revoked.

1 should be pretty easy to implement; the names should be exactly equal. Wildcard certificates exist but are considered harmful; not supporting them is probably fine for v1.

2 can probably be ignored for now; the vast majority of ESP8266 work will be bound to specific certificates because...

3 is the most complex part. On a normal computer, the OS (and sometimes the browser) maintains a list of trusted root certificates. For Debian, the size of this certificate package is 502kB (see https://packages.debian.org/sid/ca-certificates) - it's possible to shrink this down a bit but it's obviously a nonstarter for the average ESP8266 project. An alternative to this is simply maintaining a list of certificate fingerprints and checking that instead - creating an explicit whitelist of certificates. That's what I'd recommend - and when you're using certificate fingerprints, you can be lax about expiration and revocation and manage it yourself.

4 is a complicated topic - check out CRL and OCSP if you're curious - and way beyond what we'd want to do in an embedded system.

AxTLS probably has simple-ish calls to handle all of this; they'd need to be exposed via SecureWifiClient.

SecureWifiServer would need a new method that allows a user to set the public and private key to use.

@mtnbrit

This comment has been minimized.

Show comment
Hide comment
@mtnbrit

mtnbrit Jul 30, 2015

does thread this also cover applying TLS to MQTT eventually?

mtnbrit commented Jul 30, 2015

does thread this also cover applying TLS to MQTT eventually?

@anteph

This comment has been minimized.

Show comment
Hide comment
@anteph

anteph Aug 2, 2015

For what I've read, adding https support is a little bit hard. But what about simple ciphering, for direct socket communication? Something with a symmetric key, that would be a good start.

I've read in the Espressif page that ESP8266 has a built in AES engine, but they don't specify if it is hardware or software based. Has anyone tried it?

anteph commented Aug 2, 2015

For what I've read, adding https support is a little bit hard. But what about simple ciphering, for direct socket communication? Something with a symmetric key, that would be a good start.

I've read in the Espressif page that ESP8266 has a built in AES engine, but they don't specify if it is hardware or software based. Has anyone tried it?

@torntrousers

This comment has been minimized.

Show comment
Hide comment
@torntrousers

torntrousers Aug 2, 2015

Contributor

Just a comment on that list of work from @abl - to get going i'd be quite happy to not have any of that certificate validation stuff. Wont be sending my bank account details or anything top secret just need to post to a remote service that only supports https. So just binning the certificate after its read would be fine for my uses.

Contributor

torntrousers commented Aug 2, 2015

Just a comment on that list of work from @abl - to get going i'd be quite happy to not have any of that certificate validation stuff. Wont be sending my bank account details or anything top secret just need to post to a remote service that only supports https. So just binning the certificate after its read would be fine for my uses.

@drmpf

This comment has been minimized.

Show comment
Hide comment
@drmpf

drmpf Aug 2, 2015

If you just want to prevent un-authorized use rather then hid the messages, check out this page on using SipHash as a secure hash.
http://www.forward.com.au/pfod/secureChallengeResponse/index.html

drmpf commented Aug 2, 2015

If you just want to prevent un-authorized use rather then hid the messages, check out this page on using SipHash as a secure hash.
http://www.forward.com.au/pfod/secureChallengeResponse/index.html

@cottsak

This comment has been minimized.

Show comment
Hide comment
@cottsak

cottsak Aug 28, 2015

So what's the latest?

  • Can I make a HTTPS request?
  • can I use TLS 1.2?
  • can I pin a certificate (avoid the CA lookups)?

cottsak commented Aug 28, 2015

So what's the latest?

  • Can I make a HTTPS request?
  • can I use TLS 1.2?
  • can I pin a certificate (avoid the CA lookups)?
@jacealot

This comment has been minimized.

Show comment
Hide comment
@jacealot

jacealot Aug 28, 2015

Is it now possible to make a HTTPS request?
Or are there any libraries out there that I could use?

jacealot commented Aug 28, 2015

Is it now possible to make a HTTPS request?
Or are there any libraries out there that I could use?

@igrr

This comment has been minimized.

Show comment
Hide comment
@igrr

igrr Aug 31, 2015

Member

Started working on TLS support: https://github.com/igrr/axtls-8266
This is the same library that Espressif uses for their libssl, so TLS 1.0/1.1 only.

Member

igrr commented Aug 31, 2015

Started working on TLS support: https://github.com/igrr/axtls-8266
This is the same library that Espressif uses for their libssl, so TLS 1.0/1.1 only.

@abl

This comment has been minimized.

Show comment
Hide comment
@abl

abl Sep 2, 2015

Going that route, it seems like wolfSSL and mbed TLS (well, especially mbed TLS) might be easier to bring in, although we know axTLS works since Espressif has it working :)

They also support TLS 1.2.

abl commented Sep 2, 2015

Going that route, it seems like wolfSSL and mbed TLS (well, especially mbed TLS) might be easier to bring in, although we know axTLS works since Espressif has it working :)

They also support TLS 1.2.

@igrr

This comment has been minimized.

Show comment
Hide comment
@igrr

igrr Sep 2, 2015

Member

Both wolfSSL and PolarSSL (aka mbed TLS) are GPL (not LGPL) unless you
apply for a commercial license. I would like to keep the library
LGPL-compatible.

On Wed, Sep 2, 2015, 21:24 Alexander notifications@github.com wrote:

Going that route, it seems like wolfSSL and mbed TLS (well, especially
mbed TLS) might be easier to bring in, although we know axTLS works since
Espressif has it working :)

They also support TLS 1.2.


Reply to this email directly or view it on GitHub
#43 (comment).

Member

igrr commented Sep 2, 2015

Both wolfSSL and PolarSSL (aka mbed TLS) are GPL (not LGPL) unless you
apply for a commercial license. I would like to keep the library
LGPL-compatible.

On Wed, Sep 2, 2015, 21:24 Alexander notifications@github.com wrote:

Going that route, it seems like wolfSSL and mbed TLS (well, especially
mbed TLS) might be easier to bring in, although we know axTLS works since
Espressif has it working :)

They also support TLS 1.2.


Reply to this email directly or view it on GitHub
#43 (comment).

@abl

This comment has been minimized.

Show comment
Hide comment
@abl

abl Sep 2, 2015

Ah, I missed that on WolfSSL.

For PolarSSL/mbed, they have an explicit exception list that includes the LGPL: https://tls.mbed.org/foss-license-exception

The language appears to prevent the distribution of a non-GPL fork of mbed TLS but has other odd quirks. Certainly a larger headache than axTLS's BSD-ish license.

abl commented Sep 2, 2015

Ah, I missed that on WolfSSL.

For PolarSSL/mbed, they have an explicit exception list that includes the LGPL: https://tls.mbed.org/foss-license-exception

The language appears to prevent the distribution of a non-GPL fork of mbed TLS but has other odd quirks. Certainly a larger headache than axTLS's BSD-ish license.

@cottsak

This comment has been minimized.

Show comment
Hide comment
@cottsak

cottsak Sep 10, 2015

I can't believe this conversation has come to licencing..

cottsak commented Sep 10, 2015

I can't believe this conversation has come to licencing..

@igrr

This comment has been minimized.

Show comment
Hide comment
@igrr

igrr Sep 14, 2015

Member

WiFiClientSecure is mostly ready now. Things left to do:

  • seed PRNG properly
  • check host name against certificate's common name and alternative DNS names (wildcards are quite common, 4 out of 5 services I used for my tests do use wildcards).
  • release memory allocated for certificate storage after check is complete
Member

igrr commented Sep 14, 2015

WiFiClientSecure is mostly ready now. Things left to do:

  • seed PRNG properly
  • check host name against certificate's common name and alternative DNS names (wildcards are quite common, 4 out of 5 services I used for my tests do use wildcards).
  • release memory allocated for certificate storage after check is complete
@cottsak

This comment has been minimized.

Show comment
Hide comment
@cottsak

cottsak Sep 14, 2015

@igrr Coz I'm new, what is the simplest and quickest way to get your changes onto my esp8266?

cottsak commented Sep 14, 2015

@igrr Coz I'm new, what is the simplest and quickest way to get your changes onto my esp8266?

@vicatcu

This comment has been minimized.

Show comment
Hide comment
@vicatcu

vicatcu Sep 14, 2015

@igrr that's awesome, I'm looking forward to trying it out. Thanks for your work on this!

vicatcu commented Sep 14, 2015

@igrr that's awesome, I'm looking forward to trying it out. Thanks for your work on this!

@igrr

This comment has been minimized.

Show comment
Hide comment
@igrr

igrr Sep 14, 2015

Member

@cottsak You can either build IDE from source or wait a bit for the next staging version.

Member

igrr commented Sep 14, 2015

@cottsak You can either build IDE from source or wait a bit for the next staging version.

@cottsak

This comment has been minimized.

Show comment
Hide comment
@cottsak

cottsak Sep 15, 2015

@igrr So that's it? Once built, run the IDE from the build output location and the WiFiClientSecure changes will be available to my sketch and should upload to my esp8266 no problem?

cottsak commented Sep 15, 2015

@igrr So that's it? Once built, run the IDE from the build output location and the WiFiClientSecure changes will be available to my sketch and should upload to my esp8266 no problem?

@madsci1016

This comment has been minimized.

Show comment
Hide comment
@madsci1016

madsci1016 Mar 5, 2016

Just tested WiFiClientSecure() and it worked great, thanks. My application is a IoT sensing application, and I'd like to enable 'client authentication' so the server verifies it's my ESP8266 trying to publish new sensor data.

Some googling shows axTLS supports client authentication, but before i lock myself in a room for a week to go through the source code, I'd figure I'd ask. Does the current port of WiFiClientSecure() have a way to support client authentication? IE, I'd have to embed a trusted cert in the firmware that would be presented to the server during the handshake.

madsci1016 commented Mar 5, 2016

Just tested WiFiClientSecure() and it worked great, thanks. My application is a IoT sensing application, and I'd like to enable 'client authentication' so the server verifies it's my ESP8266 trying to publish new sensor data.

Some googling shows axTLS supports client authentication, but before i lock myself in a room for a week to go through the source code, I'd figure I'd ask. Does the current port of WiFiClientSecure() have a way to support client authentication? IE, I'd have to embed a trusted cert in the firmware that would be presented to the server during the handshake.

@Links2004

This comment has been minimized.

Show comment
Hide comment
@madsci1016

This comment has been minimized.

Show comment
Hide comment
@madsci1016

madsci1016 Mar 5, 2016

Thanks Links, that's a start.

madsci1016 commented Mar 5, 2016

Thanks Links, that's a start.

@madsci1016

This comment has been minimized.

Show comment
Hide comment
@madsci1016

madsci1016 Mar 7, 2016

@igrr one of your future milestones was "release memory allocated for certificate storage after check is complete" has that been implemented yet?

I'm now running into a memory issue:

State:    receiving Certificate (11)
crypto/bigint.c:1072 realloc 1032, left 15480
crypto/bigint.c:1072 realloc 1032, left 11944
crypto/bigint.c:1072 realloc 1028, left 7584
crypto/bigint.c:1072 realloc 2056, left 5424
crypto/bigint.c:1072 realloc 1032, left 4344
crypto/bigint.c:1072 realloc 1024, left 3832
crypto/bigint.c:1072 realloc 1032, left 1048
crypto/bigint.c:1072 realloc 512 failed, left 752
Fatal exception 29(StoreProhibitedCause):

I try to malloc them memory I need after the handshake, but at that point ESP.getFreeHeap() only shows 12KB available. Before the client.conneect call it is 27KB+. I

madsci1016 commented Mar 7, 2016

@igrr one of your future milestones was "release memory allocated for certificate storage after check is complete" has that been implemented yet?

I'm now running into a memory issue:

State:    receiving Certificate (11)
crypto/bigint.c:1072 realloc 1032, left 15480
crypto/bigint.c:1072 realloc 1032, left 11944
crypto/bigint.c:1072 realloc 1028, left 7584
crypto/bigint.c:1072 realloc 2056, left 5424
crypto/bigint.c:1072 realloc 1032, left 4344
crypto/bigint.c:1072 realloc 1024, left 3832
crypto/bigint.c:1072 realloc 1032, left 1048
crypto/bigint.c:1072 realloc 512 failed, left 752
Fatal exception 29(StoreProhibitedCause):

I try to malloc them memory I need after the handshake, but at that point ESP.getFreeHeap() only shows 12KB available. Before the client.conneect call it is 27KB+. I

@igrr

This comment has been minimized.

Show comment
Hide comment
@igrr

igrr Mar 7, 2016

Member

It is implemented in 2.1.0. After WiFiClient::connect returns, certificate will be kept until the first call to read or write functions. This is done to allow one call verify function and check the certificate. Once you call read or write, certificate will be released.

Member

igrr commented Mar 7, 2016

It is implemented in 2.1.0. After WiFiClient::connect returns, certificate will be kept until the first call to read or write functions. This is done to allow one call verify function and check the certificate. Once you call read or write, certificate will be released.

@apigeoje

This comment has been minimized.

Show comment
Hide comment
@apigeoje

apigeoje Mar 7, 2016

@igrr - You're frikin awesome. I've just spent 15 minutes reading this. I haven't even tried to use it yet but I'm already grateful for all your work...

apigeoje commented Mar 7, 2016

@igrr - You're frikin awesome. I've just spent 15 minutes reading this. I haven't even tried to use it yet but I'm already grateful for all your work...

@madsci1016

This comment has been minimized.

Show comment
Hide comment
@madsci1016

madsci1016 Mar 7, 2016

Yes, @igrr is super awesome and needs to show us a paypal donate button so we can show our appreciation.

I've added some of my own debug as well as turned on the builtin. Here's what I got:

`Free Mem before connection: 38536

ssl/tls1.c:549 malloc 6864, left 30960

please start sntp first !

State: sending Client Hello (1)

State: receiving Server Hello (2)

State: receiving Certificate (11)

crypto/bigint.c:1072 realloc 1032, left 26584

crypto/bigint.c:1072 realloc 1032, left 23048

crypto/bigint.c:1072 realloc 1028, left 18688

crypto/bigint.c:1072 realloc 2056, left 16528

crypto/bigint.c:1072 realloc 1032, left 15448

crypto/bigint.c:1072 realloc 1024, left 14936

crypto/bigint.c:1072 realloc 1032, left 12024

State: receiving Server Hello Done (14)

crypto/bigint.c:1072 realloc 1024, left 8432

State: sending Client Key Exchange (16)

State: sending Finished (16)

State: receiving Finished (16)

Free Mem after handshake: 12184

ssl/tls1.c:1422 malloc 17408, left 19280

Free Mem after print: 19648

Response: :rd 5, 330, 0

Client closing connection

Alert: close notify`

So with 38KB free originally, 30KB are consumed during the handshake and 4KB are given back right away, leaving the user with 12KB free. After the first write call, an additional 7KB are given back, bringing free mem to 19KB with the connection open.

@igrr Is this the expected behavior? Are there any other improvements you could suggest on the server end to optimize memory usage?

My application is a IoT sensor that will include a 1 or 2 second WAV file recorded by the ESP8266, That's why I'm being a stickler for memory.

madsci1016 commented Mar 7, 2016

Yes, @igrr is super awesome and needs to show us a paypal donate button so we can show our appreciation.

I've added some of my own debug as well as turned on the builtin. Here's what I got:

`Free Mem before connection: 38536

ssl/tls1.c:549 malloc 6864, left 30960

please start sntp first !

State: sending Client Hello (1)

State: receiving Server Hello (2)

State: receiving Certificate (11)

crypto/bigint.c:1072 realloc 1032, left 26584

crypto/bigint.c:1072 realloc 1032, left 23048

crypto/bigint.c:1072 realloc 1028, left 18688

crypto/bigint.c:1072 realloc 2056, left 16528

crypto/bigint.c:1072 realloc 1032, left 15448

crypto/bigint.c:1072 realloc 1024, left 14936

crypto/bigint.c:1072 realloc 1032, left 12024

State: receiving Server Hello Done (14)

crypto/bigint.c:1072 realloc 1024, left 8432

State: sending Client Key Exchange (16)

State: sending Finished (16)

State: receiving Finished (16)

Free Mem after handshake: 12184

ssl/tls1.c:1422 malloc 17408, left 19280

Free Mem after print: 19648

Response: :rd 5, 330, 0

Client closing connection

Alert: close notify`

So with 38KB free originally, 30KB are consumed during the handshake and 4KB are given back right away, leaving the user with 12KB free. After the first write call, an additional 7KB are given back, bringing free mem to 19KB with the connection open.

@igrr Is this the expected behavior? Are there any other improvements you could suggest on the server end to optimize memory usage?

My application is a IoT sensor that will include a 1 or 2 second WAV file recorded by the ESP8266, That's why I'm being a stickler for memory.

@krebbi

This comment has been minimized.

Show comment
Hide comment
@krebbi

krebbi Mar 10, 2016

is there a known issue with Let's Encrypt certificates?

because I get

please start sntp first !
State: sending Client Hello (1)
Alert: handshake failure
Error: SSL error 40
Alert: unexpected message
Alert: close notify

krebbi commented Mar 10, 2016

is there a known issue with Let's Encrypt certificates?

because I get

please start sntp first !
State: sending Client Hello (1)
Alert: handshake failure
Error: SSL error 40
Alert: unexpected message
Alert: close notify

@igrr

This comment has been minimized.

Show comment
Hide comment
@igrr

igrr Mar 10, 2016

Member

@krebbi does the server allow TLS 1.1? axTLS doesn't support TLS 1.2, which may be the reason for this error.

Member

igrr commented Mar 10, 2016

@krebbi does the server allow TLS 1.1? axTLS doesn't support TLS 1.2, which may be the reason for this error.

@krebbi

This comment has been minimized.

Show comment
Hide comment
@krebbi

krebbi commented Mar 10, 2016

TLS 1.0, 1.1 and 1.2
https://api.12view.me

@krebbi

This comment has been minimized.

Show comment
Hide comment
@krebbi

krebbi Mar 10, 2016

or is RSA 4096 bits too big?

krebbi commented Mar 10, 2016

or is RSA 4096 bits too big?

@madsci1016

This comment has been minimized.

Show comment
Hide comment
@madsci1016

madsci1016 Mar 10, 2016

I believe this ESP version only supports AES 128 or 256

madsci1016 commented Mar 10, 2016

I believe this ESP version only supports AES 128 or 256

@krebbi

This comment has been minimized.

Show comment
Hide comment
@krebbi

krebbi Mar 11, 2016

these Cipher Suites are supported by my cert:

TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030) ECDH secp256r1 (eq. 3072 bits RSA) FS 256
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f) ECDH secp256r1 (eq. 3072 bits RSA) FS 128
TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x9f) DH 2048 bits FS 256
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x9e) DH 2048 bits FS 128
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028) ECDH secp256r1 (eq. 3072 bits RSA) FS 256
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014) ECDH secp256r1 (eq. 3072 bits RSA) FS 256
TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 (0x6b) DH 2048 bits FS 256
TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x39) DH 2048 bits FS 256

krebbi commented Mar 11, 2016

these Cipher Suites are supported by my cert:

TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030) ECDH secp256r1 (eq. 3072 bits RSA) FS 256
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f) ECDH secp256r1 (eq. 3072 bits RSA) FS 128
TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x9f) DH 2048 bits FS 256
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x9e) DH 2048 bits FS 128
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028) ECDH secp256r1 (eq. 3072 bits RSA) FS 256
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014) ECDH secp256r1 (eq. 3072 bits RSA) FS 256
TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 (0x6b) DH 2048 bits FS 256
TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x39) DH 2048 bits FS 256

@slaff

This comment has been minimized.

Show comment
Hide comment
@slaff

slaff Mar 18, 2016

@igrr Do you know if NGINX supports max_fragment_length negotiation ?

slaff commented Mar 18, 2016

@igrr Do you know if NGINX supports max_fragment_length negotiation ?

@electronicsguy

This comment has been minimized.

Show comment
Hide comment
@electronicsguy

electronicsguy Mar 29, 2016

@igrr Could someone please tell me if https can work on the ESP-01?

electronicsguy commented Mar 29, 2016

@igrr Could someone please tell me if https can work on the ESP-01?

@svdgraaf

This comment has been minimized.

Show comment
Hide comment
@svdgraaf

svdgraaf Mar 29, 2016

@electronicsguy some https connections will work, some won't. It has to do with the TLS version, the used ciphers, and/or the configuration of the webserver.

svdgraaf commented Mar 29, 2016

@electronicsguy some https connections will work, some won't. It has to do with the TLS version, the used ciphers, and/or the configuration of the webserver.

@slaff

This comment has been minimized.

Show comment
Hide comment
@slaff

slaff Mar 31, 2016

..." Nginx hardcodes 16KB size in ngx_event_openssl, which you can change and recompile from source."

Do you know if NGINX supports max_fragment_length negotiation ?

It turns out that you can set SSL fragment size in NGINX (>= 1.5.9 ). The directive is called ssl_buffer_size. And there is no need to recompile NGINX.

slaff commented Mar 31, 2016

..." Nginx hardcodes 16KB size in ngx_event_openssl, which you can change and recompile from source."

Do you know if NGINX supports max_fragment_length negotiation ?

It turns out that you can set SSL fragment size in NGINX (>= 1.5.9 ). The directive is called ssl_buffer_size. And there is no need to recompile NGINX.

@electronicsguy

This comment has been minimized.

Show comment
Hide comment
@electronicsguy

electronicsguy Apr 1, 2016

@svdgraaf @igrr Thanks for the follow-up. I am still trying to understand the details. All I want to do is send data from esp8266 directly into a google spreadsheet (without using a http to https converter service like pushingbox). So is incorporating support for TLS1.2 unachieveable with the current esp hardware? Or is it a matter of someone writing up that code, which has just not been done as yet?

electronicsguy commented Apr 1, 2016

@svdgraaf @igrr Thanks for the follow-up. I am still trying to understand the details. All I want to do is send data from esp8266 directly into a google spreadsheet (without using a http to https converter service like pushingbox). So is incorporating support for TLS1.2 unachieveable with the current esp hardware? Or is it a matter of someone writing up that code, which has just not been done as yet?

@AdamMiltonBarker

This comment has been minimized.

Show comment
Hide comment
@AdamMiltonBarker

AdamMiltonBarker Apr 11, 2016

Anyone got TLS1.2 working yet and a way to add trusted certs ?

AdamMiltonBarker commented Apr 11, 2016

Anyone got TLS1.2 working yet and a way to add trusted certs ?

@fsommer1968

This comment has been minimized.

Show comment
Hide comment
@fsommer1968

fsommer1968 Apr 16, 2016

@AdamMiltonBarker!
Have a look at WolfSSL, they support TLS V1.2 but I´m not sure whether the library works with ESP. I want to use TLS 1.2 with Bluemix IoT but I have currently not the time to test. Maybe you can try?

fsommer1968 commented Apr 16, 2016

@AdamMiltonBarker!
Have a look at WolfSSL, they support TLS V1.2 but I´m not sure whether the library works with ESP. I want to use TLS 1.2 with Bluemix IoT but I have currently not the time to test. Maybe you can try?

@marvinroger

This comment has been minimized.

Show comment
Hide comment
@marvinroger

marvinroger Apr 16, 2016

Contributor

wolfSSL has already been discussed, and the problem is it is GPL and not LGPL. I am not sure I understand the difference, to be honest.

Contributor

marvinroger commented Apr 16, 2016

wolfSSL has already been discussed, and the problem is it is GPL and not LGPL. I am not sure I understand the difference, to be honest.

@fsommer1968

This comment has been minimized.

Show comment
Hide comment
@fsommer1968

fsommer1968 Apr 16, 2016

For my project it would´nt be important whether GPL or LGPL.

fsommer1968 commented Apr 16, 2016

For my project it would´nt be important whether GPL or LGPL.

@AdamMiltonBarker

This comment has been minimized.

Show comment
Hide comment
@AdamMiltonBarker

AdamMiltonBarker Apr 18, 2016

I agree I think security is more important that the license.

AdamMiltonBarker commented Apr 18, 2016

I agree I think security is more important that the license.

@DaKaZ

This comment has been minimized.

Show comment
Hide comment
@DaKaZ

DaKaZ May 5, 2016

@igrr - I too greatly appreciate all you do for the esp8266 community. Having been developing on this hardware for the last year I could not have done it without you!

I am having trouble with the WiFiClientSecure implementation though. I basically have the exact same code for HTTP as HTTPS but I never get the response back from the server with SSL (unless I remove connection: close from the http headers and wait for a timeout). I have verified that the server is indeed sending a response back (wireshark). The code is here: https://gist.github.com/DaKaZ/5044f3168412ce0b3f5bf1d46130765e

From my debug output, I see the Alert: close notify before I even call sslClient.connected() or sslClient.available() so I fear I am missing something. Any ideas?? Here is a little more from my client debug:

HTTP: connect
please start sntp first !
State:  sending Client Hello (1)
State:  receiving Server Hello (2)
State:  receiving Certificate (11)
State:  receiving Server Hello Done (14)
State:  sending Client Key Exchange (16)
State:  sending Finished (16)
State:  receiving Finished (16)
Verifiying SSL certificate
SSL certificate matches
HTTP: connected
REQUEST: 

SSL Print: POST
SSL Print:  
SSL Print: /api/v1/plugs
SSL Print:  HTTP/1.1

SSL Print: Accept-Encoding: application/json
SSL Print: 

SSL Print: Timestamp: 1462488709
SSL Print: 

SSL Print: Signature: <removed for online post>
SSL Print: 

SSL Print: Host: 
SSL Print: <removed for online post>
SSL Print: 

SSL Print: Connection: close

SSL Print: Content-Length: 31

SSL Print: Content-Type: 
SSL Print: application/json
SSL Print: 

SSL Print: 

SSL Print: { "mac" : "<removed for online post>" }
SSL Print: 

SSL Print: 
Alert: close notify

END REQUEST
HTTP: call readResponse
HTTP: NON-NULL RESPONSE POINTER: 
HTTP: RESPONSE: 
HTTP: Connect: 0 Available: 0
HTTPS client closed 
HTTP: return readResponse3
HTTP: return readResponse
HTTP: stop client
HTTP: client stopped
Got response: 
pm open,type:2 0

DaKaZ commented May 5, 2016

@igrr - I too greatly appreciate all you do for the esp8266 community. Having been developing on this hardware for the last year I could not have done it without you!

I am having trouble with the WiFiClientSecure implementation though. I basically have the exact same code for HTTP as HTTPS but I never get the response back from the server with SSL (unless I remove connection: close from the http headers and wait for a timeout). I have verified that the server is indeed sending a response back (wireshark). The code is here: https://gist.github.com/DaKaZ/5044f3168412ce0b3f5bf1d46130765e

From my debug output, I see the Alert: close notify before I even call sslClient.connected() or sslClient.available() so I fear I am missing something. Any ideas?? Here is a little more from my client debug:

HTTP: connect
please start sntp first !
State:  sending Client Hello (1)
State:  receiving Server Hello (2)
State:  receiving Certificate (11)
State:  receiving Server Hello Done (14)
State:  sending Client Key Exchange (16)
State:  sending Finished (16)
State:  receiving Finished (16)
Verifiying SSL certificate
SSL certificate matches
HTTP: connected
REQUEST: 

SSL Print: POST
SSL Print:  
SSL Print: /api/v1/plugs
SSL Print:  HTTP/1.1

SSL Print: Accept-Encoding: application/json
SSL Print: 

SSL Print: Timestamp: 1462488709
SSL Print: 

SSL Print: Signature: <removed for online post>
SSL Print: 

SSL Print: Host: 
SSL Print: <removed for online post>
SSL Print: 

SSL Print: Connection: close

SSL Print: Content-Length: 31

SSL Print: Content-Type: 
SSL Print: application/json
SSL Print: 

SSL Print: 

SSL Print: { "mac" : "<removed for online post>" }
SSL Print: 

SSL Print: 
Alert: close notify

END REQUEST
HTTP: call readResponse
HTTP: NON-NULL RESPONSE POINTER: 
HTTP: RESPONSE: 
HTTP: Connect: 0 Available: 0
HTTPS client closed 
HTTP: return readResponse3
HTTP: return readResponse
HTTP: stop client
HTTP: client stopped
Got response: 
pm open,type:2 0
@chaeplin

This comment has been minimized.

Show comment
Hide comment
@DaKaZ

This comment has been minimized.

Show comment
Hide comment
@DaKaZ

DaKaZ May 6, 2016

Very interesting... thanks @chaeplin - So I changed the multiple write(...); lines into one string just called write(newString.c_str()); once and now I get the response. Not sure why but its working thats to your reference!!

DaKaZ commented May 6, 2016

Very interesting... thanks @chaeplin - So I changed the multiple write(...); lines into one string just called write(newString.c_str()); once and now I get the response. Not sure why but its working thats to your reference!!

@DaKaZ

This comment has been minimized.

Show comment
Hide comment
@DaKaZ

DaKaZ May 6, 2016

If anyone comes across this thread - here is a working version of the RestClient with SSL support: https://github.com/DaKaZ/esp8266-restclient

Enjoy

DaKaZ commented May 6, 2016

If anyone comes across this thread - here is a working version of the RestClient with SSL support: https://github.com/DaKaZ/esp8266-restclient

Enjoy

@Matthew-Bradley

This comment has been minimized.

Show comment
Hide comment
@Matthew-Bradley

Matthew-Bradley Jul 18, 2016

If there's an issue storing certificates in RAM, is it possible you could stream them into the file system and walk through them there? (Note: I've also seen a hack to upgrade the flash to 4M or bigger, so you might be able to store a fair bit)

Matthew-Bradley commented Jul 18, 2016

If there's an issue storing certificates in RAM, is it possible you could stream them into the file system and walk through them there? (Note: I've also seen a hack to upgrade the flash to 4M or bigger, so you might be able to store a fair bit)

@AdamMiltonBarker

This comment has been minimized.

Show comment
Hide comment
@AdamMiltonBarker

AdamMiltonBarker Jul 18, 2016

What the plans for secure server? I feel this is a major let down, if we can provide secure endpoints to customers on the ESP I think it would be a massive improvement.

AdamMiltonBarker commented Jul 18, 2016

What the plans for secure server? I feel this is a major let down, if we can provide secure endpoints to customers on the ESP I think it would be a massive improvement.

@igrr

This comment has been minimized.

Show comment
Hide comment
Member

igrr commented Jul 18, 2016

@esp8266 esp8266 locked and limited conversation to collaborators Jul 18, 2016

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.