Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

restrict TLS cipher suites of the server #5245

Merged
merged 3 commits into from Jan 13, 2018

Conversation

Projects
None yet
8 participants
@aead
Copy link
Member

aead commented Nov 28, 2017

Description

This change restircts the supported cipher suites of the minio server.
The server only supports AEAD ciphers (Chacha20Poly1305 and AES-GCM)

The supported cipher suites are:

  • tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
  • tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
  • tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
  • tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
  • tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
  • tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384

Motivation and Context

Fixes #5244
Fixes #5291

How Has This Been Tested?

manually

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added unit tests to cover my changes.
  • I have added/updated functional tests in mint. (If yes, add mint PR # here: )
  • All new and existing tests passed.
@@ -155,8 +155,18 @@ func NewServer(addrs []string, handler http.Handler, certificate *tls.Certificat
if certificate != nil {
tlsConfig = &tls.Config{
PreferServerCipherSuites: true,
MinVersion: tls.VersionTLS12,
NextProtos: []string{"http/1.1", "h2"},
CipherSuites: []uint16{

This comment has been minimized.

@harshavardhana

harshavardhana Nov 28, 2017

Member

We should take this list as perhaps an argument to NewServer() function, in future we might need to change these as well. We can take this as an input from top level and not default/restrict this package to the following ciphers.

This comment has been minimized.

@aead

aead Nov 28, 2017

Author Member

Agree 👍

@codecov

This comment has been minimized.

Copy link

codecov bot commented Nov 28, 2017

Codecov Report

Merging #5245 into master will increase coverage by 0.3%.
The diff coverage is 90%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master    #5245     +/-   ##
=========================================
+ Coverage   61.12%   61.43%   +0.3%     
=========================================
  Files         201      201             
  Lines       29509    29303    -206     
=========================================
- Hits        18038    18002     -36     
+ Misses      10083     9908    -175     
- Partials     1388     1393      +5
Impacted Files Coverage Δ
pkg/http/server.go 84.81% <90%> (+58.78%) ⬆️
cmd/xl-v1-list-objects-heal.go 71.08% <0%> (-3.07%) ⬇️
cmd/admin-handlers.go 53.49% <0%> (-2.22%) ⬇️
cmd/xl-v1-healing-common.go 78.02% <0%> (-1.65%) ⬇️
cmd/object-api-errors.go 62.26% <0%> (-1.26%) ⬇️
cmd/jwt.go 75% <0%> (-1.12%) ⬇️
cmd/bucket-policy-handlers.go 88.18% <0%> (-0.22%) ⬇️
cmd/bucket-policy.go 62.13% <0%> (-0.08%) ⬇️
cmd/bucket-handlers.go 69.09% <0%> (-0.08%) ⬇️
cmd/xl-v1-object.go 79.05% <0%> (-0.05%) ⬇️
... and 17 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3f8379d...ce07746. Read the comment docs.

@aead aead force-pushed the aead:tls-ciphersuite branch from d60da39 to 2d6d9b0 Nov 28, 2017

@harshavardhana

This comment has been minimized.

Copy link
Member

harshavardhana commented Nov 28, 2017

Go implementation of CBC does not contain effective countermeasures
against timing attacks.

Some counter measures implemented

golang/go@f28cf83
golang/go@4822e76
golang/go@48d8edb

Refer here golang/go#13385

Disabling CBC for 128bit and above can have real issues with clients like CloudBerry which was using CBC. Historically we had this in the past

https://github.com/minio/minio/pull/3608/files#diff-e6845802830d0358dfc98e01152842c2R379

Thus we had to bring back CBC to support it, so we should validate those clients before accepting this patch and also have a thorough review such that we are not too strict about this.

@aead

This comment has been minimized.

Copy link
Member Author

aead commented Nov 28, 2017

Oh okay, wasn't aware that Filippo fixed some CBC timing issues. So we can "safely" enable CBC-*-SHA suites. However SHA256 suites are still vulnerable.

Agree that we have to check clients.

@harshavardhana
Copy link
Member

harshavardhana left a comment

LGTM tested

@deekoder deekoder requested review from donatello and harshavardhana and removed request for balamurugana Dec 12, 2017

@aead

This comment has been minimized.

Copy link
Member Author

aead commented Dec 12, 2017

Disabling RSA PKCS v1.5 ciphers seems to make no difference. However it's theoretically possible that there are clients which does not support ECC... As far as I can see disabling RSA-only ciphers doesn't break anything I'm aware of...

@harshavardhana
Copy link
Member

harshavardhana left a comment

We will perhaps tell the clients to upgrade rather use vulnerable access. LGTM

@@ -167,6 +167,16 @@ var (
colorYellow = color.New(color.FgYellow).SprintfFunc()
)

// Secure Go implementations of TLS ciphers
var tlsCipherSuites = []uint16{

This comment has been minimized.

@balamurugana

balamurugana Dec 13, 2017

Member

If this is more secure TLS configuration, how about making this default cipher suite for http server in case of TLS enabled? You always has an option to override this default behavior.

This comment has been minimized.

@aead

aead Dec 13, 2017

Author Member

Don't understand - what do you mean with "making this the default cipher suites". These ciphers are used as the "default" cipher suites for the server and the gateway. Or do I miss something?
See https://github.com/minio/minio/pull/5245/files#diff-736572b438aa05056f334265beb9507bR158

This comment has been minimized.

@balamurugana

balamurugana Dec 13, 2017

Member

My suggestion is below

  1. move tlsCipherSuites to pkg/http/server.go itself i.e. no definition in cmd/*.go
  2. there is no cipherSuites argument to NewServer() i.e. tlsCipherSuites is set always for TLS.
  3. if user wants to override this, he could directly set Server.TLSConfig accordingly to his need.
  4. this way there is no change in cmd/*.go

This comment has been minimized.

@aead

aead Dec 13, 2017

Author Member

That's exactly how the patch looked before but harsha mentioned:

We should take this list as perhaps an argument to NewServer() function, in future we might need to change these as well. We can take this as an input from top level and not default/restrict this package to the following ciphers.

This comment has been minimized.

@balamurugana

balamurugana Dec 13, 2017

Member

@harshavardhana As the approach provides support to the use case you mentioned, do you have any other concern to do the change?

This comment has been minimized.

@harshavardhana

harshavardhana Dec 13, 2017

Member

That won't be a problem, since the TLSConfig can be overriden. I guess having a default is fine for now.

@aead aead dismissed stale reviews from donatello and harshavardhana via f4a77a7 Dec 13, 2017

@aead aead force-pushed the aead:tls-ciphersuite branch from b027bf4 to f4a77a7 Dec 13, 2017

@aead

This comment has been minimized.

Copy link
Member Author

aead commented Dec 13, 2017

Okay, reverted changes according to @balamurugana comment.
@harshavardhana @donatello PTAL

@aead aead force-pushed the aead:tls-ciphersuite branch from f4a77a7 to 7a73f63 Dec 13, 2017

@harshavardhana
Copy link
Member

harshavardhana left a comment

GIt commit message here still says we are allowing following ciphers but code is changed can you amend your commit as well @aead ?

 - tls.TLS_RSA_WITH_AES_128_GCM_SHA256
 - tls.TLS_RSA_WITH_AES_256_GCM_SHA384
@harshavardhana
Copy link
Member

harshavardhana left a comment

A typo s/restircts/restricts/

because the Go implementation of CBC does not contain effective countermeasures against timing attacks.

Do we still need to mention this? @aead

Further disable RSA PKCSv1.5 cipher suites because of timing issues as shown with the ROBOT attack.

This sentence is not needed perhaps. Unless you want to really say why we even removed RSA_AES support.

// NewServer - creates new HTTP server using given arguments.
func NewServer(addrs []string, handler http.Handler, certificate *tls.Certificate) *Server {
var tlsConfig *tls.Config
if certificate != nil {
tlsConfig = &tls.Config{
PreferServerCipherSuites: true,
CipherSuites: defaultCipherSuites,

This comment has been minimized.

@balamurugana

balamurugana Dec 14, 2017

Member

could you add a unit tests to address below cases for default http server and TLSConfig'ed http server?

  1. http client connecting with defaultCipherSuites
  2. http client connecting without defaultCipherSuites
  3. http client connecting with unsupported cipher

This comment has been minimized.

@aead

aead Dec 14, 2017

Author Member

Added tests - PTAL

@aead aead force-pushed the aead:tls-ciphersuite branch from 7a73f63 to 828eb61 Dec 14, 2017

@aead

This comment has been minimized.

Copy link
Member Author

aead commented Dec 14, 2017

@harshavardhana
Updated commit msg, thanks.
I also reduced commit message - However added comments so it is understandable why we disabled a specific cipher.

BTW: I disabled RSA-PKCS1-v1.5 not because of ROBOT itself (Go stack is not vulnerable to ROBOT)
but as the name says (ROBOT = Return Of Bleichenbacher's Oracle Threat) its again a variant of the Bleichenbacher timing attack. See this -> It's not unlikely that it will strike again...

@aead aead force-pushed the aead:tls-ciphersuite branch 2 times, most recently from 7da2406 to 9940668 Dec 14, 2017

restrict TLS cipher suites of the server
This change restircts the supported cipher suites of the minio server.
The server only supports AEAD ciphers (Chacha20Poly1305 and AES-GCM)

The supported cipher suites are:
 - tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
 - tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
 - tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
 - tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
 - tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
 - tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384

Fixes #5244
Fixes #5291
@aead

This comment has been minimized.

Copy link
Member Author

aead commented Jan 10, 2018

@balamurugana

As this is underneath net.listener behavior, I am not sure what ErrServerClosed applicable to a listener. There is nothing overridden here. As I mentioned previously panic(server.Start()) is wrong in your test.

It looks like the httpListener implements behavior which is already in the standard library. However I don't know since when this behavior is in the standard library. (Would have to check) As an example: The listener implements closing functionality which should (and is) part of the server - not of the listener.

Unable to understand your questions. What is the closing channel already provided by the server? Where is the channel apart from listener in minio/pkg/http`

The http.Server.Close() or http.Server.Shutdown() already use a closing mechanism and closes all open connections. My question was: "Why does the pkg/http implements this functionality again and in a different way than the standard library.

It seems like a recent enhancement in Go http server. You could send a separate fix.

If that's the case we may should refactor our current pkg/http package.


However I'm not sure what unit-tests should achieve here. The only thing this PR changes is (excluding minior bug fixes - e.g.TLSConfig copy) the default TLS ciphers of the server - basically a default configuration change. What can be tested:

  1. That a default TLS Go client can connect to the server.
  2. That a TLS Go client which does not support any "secure" cipher cannot connect - This would be equivalent to: The server does not support any "unsecure" cipher.

For both tests I have to spin up a Server and try to connect with a Go TLS client using a specific set of cipher suites.

@aead aead force-pushed the aead:tls-ciphersuite branch from 47104f6 to cf12784 Jan 10, 2018

@balamurugana

This comment has been minimized.

Copy link
Member

balamurugana commented Jan 10, 2018

@aead You go ahead and remove unit tests added in this PR. I will send a separate PR to address unit tests.

@aead

This comment has been minimized.

Copy link
Member Author

aead commented Jan 10, 2018

@balamurugana Have added tests - Can you take a look and if something is missing from your point of view can you address?

@balamurugana

This comment has been minimized.

Copy link
Member

balamurugana commented Jan 10, 2018

@aead My previous comment You could have one test function doing multiple test cases i.e.func TestServerStart() we would need. is not yet addressed.

Let me know I can send a PR with the fix to your fork

@balamurugana

This comment has been minimized.

Copy link
Member

balamurugana commented Jan 11, 2018

@aead Please review aead#3

balamurugana and others added some commits Jan 11, 2018

Merge pull request #3 from balamurugana/aead-tls-ciphersuite-fix-unit…
…-tests

Fix unit tests to follow style used in minio
@aead

This comment has been minimized.

Copy link
Member Author

aead commented Jan 12, 2018

@harshavardhana @donatello
PTAL - Merged requested changes by @balamurugana

@balamurugana

This comment has been minimized.

Copy link
Member

balamurugana commented Jan 12, 2018

Mint Automation

Test Result
mint-xl.sh more...
mint-fs.sh more...
mint-dist-xl.sh more...
mint-gateway-s3.sh more...
mint-gateway-azure.sh more...

5245-ce07746/mint-gateway-azure.sh.log:

Running with
SERVER_ENDPOINT: minikube:31111
ACCESS_KEY:      minioazure1
SECRET_KEY:      ***REDACTED***
ENABLE_HTTPS:    0
SERVER_REGION:   us-east-1
MINT_DATA_DIR:   /mint/data
MINT_MODE:       core

To get logs, run 'docker cp a6e36aab9e03:/mint/log /tmp/mint-logs'
(1/11) Running aws-sdk-go tests ... done in 1 seconds
(2/11) Running aws-sdk-php tests ... done in 1 minutes and 21 seconds
(3/11) Running aws-sdk-ruby tests ... done in 1 minutes and 5 seconds
(4/11) Running awscli tests ... done in 9 minutes and 5 seconds
(5/11) Running mc tests ... done in 9 minutes and 30 seconds
(6/11) Running minio-dotnet tests ... FAILED in 3 minutes and 13 seconds

Unhandled Exception: System.AggregateException: One or more errors occurred. (Assert.Fail failed. ) ---> Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.Fail failed. 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.HandleFail(String assertionName, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.Fail()
   at Minio.Functional.Tests.FunctionalTest.<PutObject_Task>d__52.MoveNext() in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 755
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Minio.Functional.Tests.FunctionalTest.<ListObjects_Test5>d__72.MoveNext() in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 1633
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at Minio.Functional.Tests.FunctionalTest.Main(String[] args) in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 216

Executed 5 out of 11 tests successfully.

5245-ce07746/mint-fs.sh.log:

Running with
SERVER_ENDPOINT: minikube:30539
ACCESS_KEY:      minio
SECRET_KEY:      ***REDACTED***
ENABLE_HTTPS:    0
SERVER_REGION:   us-east-1
MINT_DATA_DIR:   /mint/data
MINT_MODE:       core

To get logs, run 'docker cp 12c36d82c300:/mint/log /tmp/mint-logs'
(1/11) Running aws-sdk-go tests ... done in 0 seconds
(2/11) Running aws-sdk-php tests ... done in 50 seconds
(3/11) Running aws-sdk-ruby tests ... done in 6 seconds
(4/11) Running awscli tests ... done in 6 minutes and 47 seconds
(5/11) Running mc tests ... done in 2 minutes and 43 seconds
(6/11) Running minio-dotnet tests ... FAILED in 1 minutes and 59 seconds

Unhandled Exception: System.AggregateException: One or more errors occurred. (Assert.Fail failed. ) ---> Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.Fail failed. 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.HandleFail(String assertionName, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.Fail()
   at Minio.Functional.Tests.FunctionalTest.<PutObject_Task>d__52.MoveNext() in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 755
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Minio.Functional.Tests.FunctionalTest.<ListObjects_Test5>d__72.MoveNext() in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 1633
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at Minio.Functional.Tests.FunctionalTest.Main(String[] args) in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 216

Executed 5 out of 11 tests successfully.

5245-ce07746/mint-xl.sh.log:

Running with
SERVER_ENDPOINT: minikube:31839
ACCESS_KEY:      minio
SECRET_KEY:      ***REDACTED***
ENABLE_HTTPS:    0
SERVER_REGION:   us-east-1
MINT_DATA_DIR:   /mint/data
MINT_MODE:       core

To get logs, run 'docker cp 83a5ef885f38:/mint/log /tmp/mint-logs'
(1/11) Running aws-sdk-go tests ... done in 0 seconds
(2/11) Running aws-sdk-php tests ... done in 1 minutes and 5 seconds
(3/11) Running aws-sdk-ruby tests ... done in 15 seconds
(4/11) Running awscli tests ... done in 6 minutes and 57 seconds
(5/11) Running mc tests ... done in 3 minutes and 45 seconds
(6/11) Running minio-dotnet tests ... FAILED in 3 minutes and 18 seconds

Unhandled Exception: System.AggregateException: One or more errors occurred. (Assert.Fail failed. ) ---> Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.Fail failed. 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.HandleFail(String assertionName, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.Fail()
   at Minio.Functional.Tests.FunctionalTest.<PutObject_Task>d__52.MoveNext() in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 755
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Minio.Functional.Tests.FunctionalTest.<ListObjects_Test5>d__72.MoveNext() in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 1633
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at Minio.Functional.Tests.FunctionalTest.Main(String[] args) in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 216

Executed 5 out of 11 tests successfully.

5245-ce07746/mint-gateway-s3.sh.log:

Running with
SERVER_ENDPOINT: minikube:31642
ACCESS_KEY:      AKIAJH7GQKYNU3C2ADXA
SECRET_KEY:      ***REDACTED***
ENABLE_HTTPS:    0
SERVER_REGION:   us-east-1
MINT_DATA_DIR:   /mint/data
MINT_MODE:       core

To get logs, run 'docker cp 72314a057327:/mint/log /tmp/mint-logs'
(1/11) Running aws-sdk-go tests ... done in 2 seconds
(2/11) Running aws-sdk-php tests ... done in 1 minutes and 28 seconds
(3/11) Running aws-sdk-ruby tests ... done in 1 minutes and 31 seconds
(4/11) Running awscli tests ... done in 8 minutes and 52 seconds
(5/11) Running mc tests ... done in 9 minutes and 23 seconds
(6/11) Running minio-dotnet tests ... FAILED in 2 minutes and 23 seconds

Unhandled Exception: System.AggregateException: One or more errors occurred. (Assert.Fail failed. ) ---> Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.Fail failed. 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.HandleFail(String assertionName, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.Fail()
   at Minio.Functional.Tests.FunctionalTest.<PutObject_Task>d__52.MoveNext() in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 755
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Minio.Functional.Tests.FunctionalTest.<ListObjects_Test5>d__72.MoveNext() in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 1633
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at Minio.Functional.Tests.FunctionalTest.Main(String[] args) in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 216

Executed 5 out of 11 tests successfully.

5245-ce07746/mint-dist-xl.sh.log:

Running with
SERVER_ENDPOINT: minikube:30706
ACCESS_KEY:      minio
SECRET_KEY:      ***REDACTED***
ENABLE_HTTPS:    0
SERVER_REGION:   us-east-1
MINT_DATA_DIR:   /mint/data
MINT_MODE:       core

To get logs, run 'docker cp 6a4c2284170a:/mint/log /tmp/mint-logs'
(1/11) Running aws-sdk-go tests ... done in 3 seconds
(2/11) Running aws-sdk-php tests ... done in 1 minutes and 8 seconds
(3/11) Running aws-sdk-ruby tests ... done in 1 minutes and 6 seconds
(4/11) Running awscli tests ... done in 7 minutes and 32 seconds
(5/11) Running mc tests ... done in 3 minutes and 17 seconds
(6/11) Running minio-dotnet tests ... FAILED in 4 minutes and 9 seconds

Unhandled Exception: System.AggregateException: One or more errors occurred. (Assert.Fail failed. ) ---> Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.Fail failed. 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.HandleFail(String assertionName, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.Fail()
   at Minio.Functional.Tests.FunctionalTest.<PutObject_Task>d__52.MoveNext() in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 755
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Minio.Functional.Tests.FunctionalTest.<ListObjects_Test5>d__72.MoveNext() in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 1633
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at Minio.Functional.Tests.FunctionalTest.Main(String[] args) in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 216

Executed 5 out of 11 tests successfully.
@harshavardhana
Copy link
Member

harshavardhana left a comment

LGTM

@balamurugana
Copy link
Member

balamurugana left a comment

LGTM

@nitisht nitisht merged commit dd202a1 into minio:master Jan 13, 2018

4 checks passed

codecov/patch 90% of diff hit (target 61.12%)
Details
codecov/project 61.43% (+0.3%) compared to 3f8379d
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@gowithplanb

This comment has been minimized.

Copy link

gowithplanb commented Feb 13, 2018

This patch has broken CloudBerry support for Minio since CloudBerry clients are now unable to initialise a TLS connection.

[2018-02-13T10:25:09.295333193Z] [ERROR] TLS handshake failed with new connection CLIENTIP:PORT at server SERVERIP:PORT (tls: no cipher suite supported by both client and server)
Trace: 1: /q/.q/sources/gopath/src/github.com/minio/minio/pkg/http/listener.go:177:http.(*httpListener).start.func2()

@aead

This comment has been minimized.

Copy link
Member Author

aead commented Feb 13, 2018

@gowithplanb
What product/version of cloudberry do you use? Can you provide the cipher suites which are tried during the handshake? / A recorded TLS handshake?

@gowithplanb

This comment has been minimized.

Copy link

gowithplanb commented Feb 13, 2018

I'm using the latest stable version 5.8 for MBS customers (similar to the one publically available). I've captured traffic between client and server, and this is what the client offers:

Secure Sockets Layer
TLSv1.2 Record Layer: Handshake Protocol: Client Hello
Content Type: Handshake (22)
Version: TLS 1.2 (0x0303)
Length: 179
Handshake Protocol: Client Hello
Handshake Type: Client Hello (1)
Length: 175
Version: TLS 1.2 (0x0303)
Random: 5a82eb24e65476fd97df8b7c48fbba021bc694441ccc9f1f...
Session ID Length: 0
Cipher Suites Length: 56
Cipher Suites (28 suites)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
Cipher Suite: TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x009f)
Cipher Suite: TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x009e)
Cipher Suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x0039)
Cipher Suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA (0x0033)
Cipher Suite: TLS_RSA_WITH_AES_256_GCM_SHA384 (0x009d)
Cipher Suite: TLS_RSA_WITH_AES_128_GCM_SHA256 (0x009c)
Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA256 (0x003d)
Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA256 (0x003c)
Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 (0xc024)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 (0xc023)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)
Cipher Suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 (0x006a)
Cipher Suite: TLS_DHE_DSS_WITH_AES_128_CBC_SHA256 (0x0040)
Cipher Suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA (0x0038)
Cipher Suite: TLS_DHE_DSS_WITH_AES_128_CBC_SHA (0x0032)
Cipher Suite: TLS_RSA_WITH_3DES_EDE_CBC_SHA (0x000a)
Cipher Suite: TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA (0x0013)
Cipher Suite: TLS_RSA_WITH_RC4_128_SHA (0x0005)
Cipher Suite: TLS_RSA_WITH_RC4_128_MD5 (0x0004)
Compression Methods Length: 1
Compression Methods (1 method)
Compression Method: null (0)
Extensions Length: 78
Extension: server_name (len=25)
Extension: supported_groups (len=6)
Extension: ec_point_formats (len=2)
Extension: signature_algorithms (len=20)
Extension: extended_master_secret (len=0)
Extension: renegotiation_info (len=1)

Minio then responds with:

Secure Sockets Layer
TLSv1.2 Record Layer: Alert (Level: Fatal, Description: Handshake Failure)
Content Type: Alert (21)
Version: TLS 1.2 (0x0303)
Length: 2
Alert Message
Level: Fatal (2)
Description: Handshake Failure (40)

@aead

This comment has been minimized.

Copy link
Member Author

aead commented Feb 13, 2018

Thanks @gowithplanb I guess you are using a RSA certificate and the client does not support ECDHE_RSA with AES_GCM or CHACHA20_POLY1305. For a ECDSA certificate the client and server should be able to negotiate a cipher suite. (AES_GCM).

The problem is that the Go TLS stack only contains timing attack countermeasures for TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA and TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA and those are not perfect. So we disabled them for security reasons. I'll come back to you with more details on how that issue can be solved.

@gowithplanb

This comment has been minimized.

Copy link

gowithplanb commented Feb 13, 2018

@aead I have contact with CloudBerry about this too now. They're informed about it and are looking into it as well. I will not upgrade Minio in production until this is resolved.

@aead

This comment has been minimized.

Copy link
Member Author

aead commented Feb 13, 2018

@gowithplanb

I have contact with CloudBerry about this too now.

Great, this is actually a Cloudberry configuration issue. The correct way to fix that is:

  • Cloudberry should enable AES_GCM for ECDHE_RSA cipher suites. They should be able to do that since they already support AES_GCM for ECDHE_ECDSA.
  • Further this issue is MBS software specific. CBB desktop client doesn't have this problem.

You can point them also to this PR thread so they can reach out if the need more information.

@gowithplanb

This comment has been minimized.

Copy link

gowithplanb commented Feb 13, 2018

Thanks @aead! PR pointed out to them.

@aead aead deleted the aead:tls-ciphersuite branch Mar 5, 2018

@topeju

This comment has been minimized.

Copy link

topeju commented Mar 7, 2018

Is it just me or does this PR also prevent Windows 8.1 / Windows Server 2012 R2 from connecting to Minio? IE on Windows Server 2012 R2 just reports

Turn on TLS 1.0, TLS 1.1, and TLS 1.2 in Advanced settings and try connecting to [https URL to the minio server] again. If this error persists, it is possible that this site uses an unsupported protocol or cipher suite such as RC4 (link for the details), which is not considered secure

when all of these TLS versions are enabled in IE.

@aead

This comment has been minimized.

Copy link
Member Author

aead commented Mar 7, 2018

@topeju
Just from looking at the a mircosoft blog post they say:

TLS 1.1 & TLS 1.2 are enabled by default on post Windows 8.1 releases. Prior to that they were disabled by default. So the administrators have to enable the settings manually via the registry.

So my guess from the reported msg would be that TLS 1.2 is not enabled on your Windows 8.1 / Windows Server 2012 R2 machine. Further I don't think that this PR causes the problem because according to the ms doc the ms TLS 1.2 stack supports the named cipher suites.

So can you try to enable TLS 1.2 and check if the issue still exists? (on all participating machines)

@topeju

This comment has been minimized.

Copy link

topeju commented Mar 7, 2018

No, TLS 1.2 is (and was) already enabled.

When I check it with cipherscan, Minio seems to report supporting ECDHE-RSA-CHACHA20-POLY1305, ECDHE-RSA-AES128-GCM-SHA256, and ECDHE-RSA-AES256-GCM-SHA384. In the Microsoft documentation (which you linked to), no ChaCha ciphers are listed, so the first one is out. All the ECDHE_RSA cipher suites listed seem to use CBC instead of GCM, as far as I can tell, so they too should be incompatible?

@aead

This comment has been minimized.

Copy link
Member Author

aead commented Mar 7, 2018

The linked ms doc lists:

  • TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256
  • TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384

However it seems Windows 8.1 does not support AES-GCM with ECDHE_RSA. Since your cipherscan reports the RSA cipher suites I assume you're using an RSA private-key / certificate.
So I think the combination of RSA private-key / certificate and no AES_GCM support for ECDHE_RSA cipher by Windows 8.1 causes this issue.
I will report this internally - A work-around would be to use ECDSA certificates instead of RSA.

BTW: You were right - this is exactly the same issue reported using Cloudberry above.

@topeju

This comment has been minimized.

Copy link

topeju commented Mar 7, 2018

Yes, I'm using an RSA private key. I tried to generate an ECDSA key using openssl ecparam -genkey -name prime256v1 -out private.key (from the documentation page) but Minio wouldn't accept the key afterwards, reporting "Invalid SSL certificate file (TLS: private key contains additional data)".

@aead

This comment has been minimized.

Copy link
Member Author

aead commented Mar 7, 2018

Oh you are right - I'm sorry about that. I will fix the documentation. In the meantime you can use the following openssl command: openssl ecparam -genkey -name prime256v1 | openssl ec -out private.key

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.