Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generic ("*") wildcard certificates aren't accepted #663

Open
c0b opened this issue Nov 28, 2018 · 30 comments
Open

Generic ("*") wildcard certificates aren't accepted #663

c0b opened this issue Nov 28, 2018 · 30 comments
Assignees

Comments

@c0b
Copy link

c0b commented Nov 28, 2018

this is a duplication of #157 ( it wasn't resolved but author gave up, I don't understand why need to lock it )

it is duplicated here, grpc Server was written in Golang deployed with self signed certificates, our existing Go based grpc Client has been working with this Go based grpc Server perfectly (for a long time already);

Now I am trying to write another gRPC client in NodeJS, just enjoy JS fast prototyping, testing feasibility of grpc-node

const client = new api.Api(
  'server-at-intranet-address:8443',
  // grpc.credentials.createInsecure(),
  grpc.credentials.createSsl(
    fs.readFileSync('./testdata/tls/ca.pem'),
    // ca, key, cert,
    fs.readFileSync('./testdata/tls/client-key.pem'),
    fs.readFileSync('./testdata/tls/client.pem'),
    { checkServerIdentity() {} },
  ));

client.MethodCall1({ param1: 1 }, (err, resp) => {
  console.log(new Date, 'ERROR:', err, resp);
});

but it didn't run till my console.log line, has failed with this log:

E1128 08:18:41.645669379    9869 ssl_transport_security.cc:1229] Handshake failed with fatal error SSL_ERROR_SSL: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed.

I have tried NODE_TLS_REJECT_UNAUTHORIZED=0 or grpc.credentials.createInsecure() or some other env variables, seems not use; all failed

from #160 (comment) I got

Cert validation is indeed always enforced.

from @jboeuf is this accurate? is there any workaround like NODE_TLS_REJECT_UNAUTHORIZED=0 to self-certs https site? To not enforce cert validation? Or would you guys say grpc-node library is not the same stable/mature as grpc Go is more native

@nicolasnoble
Copy link
Member

Change

    { checkServerIdentity() {} },

to

    { checkServerIdentity() { return true } },

You're injecting your own callback for checking the server's ssl certificate, without any body in the function, which is quite nonsensical. In JavaScript, such nop function is going to be evaluated as false, meaning you are systematically rejecting any and all server certificate, thus causing the error you're seeing: you failed to verify your certificate.

@c0b
Copy link
Author

c0b commented Nov 29, 2018

change it to return true doesn't change anything, it is still failing with:

E1128 17:50:17.640026428   28505 ssl_transport_security.cc:1229] Handshake failed with fatal error SSL_ERROR_SSL: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed.

I am using an empty function as documented, this function is supposed to return Error only, or otherwise return undefined if no error, which is same as an empty function
https://grpc.io/grpc/node/grpc.credentials.html#.createSsl

verify_options.checkServerIdentity function   Optional callback receiving the expected hostname and peer certificate for additional verification. The callback should return an Error if verification fails and otherwise return undefined.

@nicolasnoble could you confirm the documentation is correct? I have complained the doc site in another issue, and then people are telling me the doc site is somewhat deprecated, and it's not updated since August

Documentation generated by JSDoc 3.5.5 on 2018-08-10T10:27:03-07:00 using the DocStrap template.

@nicolasnoble
Copy link
Member

Right... I stand corrected. That's right, this API was always a bit odd to me, and I wish to amend it at some point so it makes a bit more sense.

I don't know why it would refuse to verify then. That'd work properly.

@nicolasnoble
Copy link
Member

If you provide us with a fully reproducible repository that demonstrates the issue, we could better help you there.

@c0b
Copy link
Author

c0b commented Nov 29, 2018

I am feeling this could be related to cfssl-certs which is not handled very well by grpc-node?
( our Go based gRPC Server&Client is nicely working for a long time, with self-signed certs generated by the CloudFlare's cfssl , )

https://github.com/c0b/grpc-helloworld/tree/the-problem/cfssl-certs

because I just tried similar openssl generted certs which indeed work with grpc-node client,

$ go run -- ./greeter_server/ -ca-path ./cfssl-certs/ca.pem -server-cert ./cfssl-certs/server.pem -server-key ./cfssl-certs/server-key.pem
listening &{{0xc00000e080 <nil> <nil> <nil> <nil> <nil> <nil> <nil> 0 4194304 2147483647 <nil> {0 0 0 0 0} {0 false} 0 0 32768 32768 120000000000 <nil>} {0 0} map[] map[] false false 0xc00003c400 map[helloworld.Greeter:0xc00016a540 grpc.reflection.v1alpha.ServerReflection:0xc00016a5d0] <nil> 0xc0000302a0 0xc000030300 {{0 0} 0} {{0 0} 0} {{0 0} 0} {{} [0 0 0]} 0 0xc00002b0c0}

...

The Go client has no problem to use this cfssl certs:

$ go run -- ./greeter_client/ -ca-path ./cfssl-certs/ca.pem -client-cert ./cfssl-certs/client.pem -client-key ./cfssl-certs/client-key.pem
2018/11/29 15:59:31 Greeting: Hello world
2018/11/29 15:59:31 Greeting: Hello again world

but grpc-node client is not working:

$ node --trace-warnings ./greeter_client.js
{ helloworld:
   { Greeter:
      { [Function: ServiceClient]
        super_: [Function: Client],
        service:
         { SayHello:
            { path: '/helloworld.Greeter/SayHello',
              requestStream: false,
              responseStream: false,
              requestSerialize: [Function: serialize],
              requestDeserialize: [Function: deserialize],
              responseSerialize: [Function: serialize],
              responseDeserialize: [Function: deserialize],
              originalName: 'sayHello' },
           SayHelloAgain:
            { path: '/helloworld.Greeter/SayHelloAgain',
              requestStream: false,
              responseStream: false,
              requestSerialize: [Function: serialize],
              requestDeserialize: [Function: deserialize],
              responseSerialize: [Function: serialize],
              responseDeserialize: [Function: deserialize],
              originalName: 'sayHelloAgain' } } } } }
ServiceClient {
  '$interceptors': [],
  '$interceptor_providers': [],
  '$callInvocationTransformer': undefined,
  '$channel': Channel {} }
E1129 08:32:50.889773363   12165 ssl_transport_security.cc:1526] Invalid wildchar entry.
E1129 08:32:51.911793960   12165 ssl_transport_security.cc:1526] Invalid wildchar entry.

@c0b
Copy link
Author

c0b commented Nov 29, 2018

the js client code is in https://github.com/c0b/grpc-helloworld/blob/the-problem/greeter_client.js

const api = grpc.loadPackageDefinition(packageDefinition);

const client = new api.helloworld.Greeter(
  'localhost:50051',
  // grpc.credentials.createInsecure(),
  grpc.credentials.createSsl(
    // fs.readFileSync('./openssl-certs/ca.crt'),
    fs.readFileSync('./cfssl-certs/ca.pem'),
    // ca, key, cert,
    fs.readFileSync('./cfssl-certs/client-key.pem'),
    // fs.readFileSync('./openssl-certs/client.key'),
    fs.readFileSync('./cfssl-certs/client.pem'),
    // fs.readFileSync('./openssl-certs/client.crt'),
    // { checkServerIdentity() {} },
  ),
  // { 'grpc.ssl_target_name_override' : 'example', 'grpc.default_authority': 'localhost', },
);

client.SayHello({ name: 'abc' }, (err, resp) => {
  console.log(new Date, 'ERROR:', err, resp);
});

client.SayHelloAgain({ name: 'abc2' }, (err, resp) => {
  console.log(new Date, 'ERROR:', err, resp);
});

as comments indicated, I have tried all kinds of constructor options with new api.helloworld.Greeter(...), none of them work, would you have some more options (or environ variables?) to try?

@nicolasnoble
Copy link
Member

Ooooh, this is an interesting smoking gun:

E1129 08:32:50.889773363   12165 ssl_transport_security.cc:1526] Invalid wildchar entry.

@jboeuf, @jiangtaoli2016, this may be a corner case with the C/C++ core's SSL certificate parser.

I'm going to try to fully reproduce your issue still, but that may be a problem with all the implementations based off the native core, not just nodejs. Meanwhile it could be useful to see if it also reproduces when using the C++ API.

@nicolasnoble
Copy link
Member

Yes, I'm fairly certain there's a bug somewhere in the core:

$ GRPC_VERBOSITY=DEBUG GRPC_TRACE=all node --trace-warnings ./greeter_client.js |& grep Peer.name.example | head -1
D1129 18:37:51.072344517   17543 security_handshaker.cc:129] Security handshake failed: {"created":"@1543513071.072336439","description":"Peer name example is not in peer certificate","file":"../deps/grpc/src/core/lib/security/security_connector/ssl/ssl_security_connector.cc","file_line":225}
$ openssl s_client -servername example -showcerts -connect localhost:50051 -CAfile cfssl-certs/ca.pem < /dev/null | head -5
depth=1 C = US, ST = California, L = San Francisco, O = "Internet Widgets, Inc.", OU = testca, CN = ca
verify return:1
depth=0 C = US, ST = California, L = San Francisco, O = "Internet Widgets, Inc.", OU = testca, CN = example
verify return:1
140586468929984:error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate:../ssl/record/rec_layer_s3.c:1399:SSL alert number 42
CONNECTED(00000003)
---
Certificate chain
 0 s:/C=US/ST=California/L=San Francisco/O=Internet Widgets, Inc./OU=testca/CN=example
   i:/C=US/ST=California/L=San Francisco/O=Internet Widgets, Inc./OU=testca/CN=ca
$ openssl x509 -in cfssl-certs/server.pem -noout -text | grep Subject:
        Subject: C = US, ST = California, L = San Francisco, O = "Internet Widgets, Inc.", OU = testca, CN = example

The C/C++ core thinks that the peer certificate doesn't match the subject name "example", whereas both a static inspection of the certificate, and an actual connection with openssl's s_client are showing that the certificate is valid and proper.

This isn't a bug on grpc-node, and I now strongly believe this is a bug on all the grpc implementations that are based off the C/C++ implementation. Beautiful.

@nicolasnoble nicolasnoble changed the title Client call with self signed certificates don't verify properly Client call with certificates from Cloudflare fail to parse properly Nov 29, 2018
@nicolasnoble
Copy link
Member

Also, one minor edit: what you're having isn't a self-signed certificate. You DO have a root CA, which is used to sign your server certificate. A self-signed certificate is a certificate which acts as its own CA. That's not the case here - the CA is separate, therefore no "self-signed" override is needed, as long as you're properly providing the CA on both ends, which you do.

@nicolasnoble
Copy link
Member

Including full dump of openssl's s_client output. There IS a warning from OpenSSL that the certificate is bad, but that's a red herring. Looking at OpenSSL's errors, it simply means we aren't providing the client certificate.

$ openssl s_client -servername example -showcerts -connect localhost:50051 -CAfile cfssl-certs/ca.pem < /dev/null
CONNECTED(00000003)
depth=1 C = US, ST = California, L = San Francisco, O = "Internet Widgets, Inc.", OU = testca, CN = ca
verify return:1
depth=0 C = US, ST = California, L = San Francisco, O = "Internet Widgets, Inc.", OU = testca, CN = example
verify return:1
140685381788096:error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate:../ssl/record/rec_layer_s3.c:1399:SSL alert number 42
---
Certificate chain
 0 s:/C=US/ST=California/L=San Francisco/O=Internet Widgets, Inc./OU=testca/CN=example
   i:/C=US/ST=California/L=San Francisco/O=Internet Widgets, Inc./OU=testca/CN=ca
-----BEGIN CERTIFICATE-----
MIIECjCCAvKgAwIBAgIUMHtXlKDbAHenyvn9Pcm0QBBuut8wDQYJKoZIhvcNAQEL
BQAweTELMAkGA1UEBhMCVVMxEzARBgNVBAgTCkNhbGlmb3JuaWExFjAUBgNVBAcT
DVNhbiBGcmFuY2lzY28xHzAdBgNVBAoTFkludGVybmV0IFdpZGdldHMsIEluYy4x
DzANBgNVBAsTBnRlc3RjYTELMAkGA1UEAxMCY2EwHhcNMTgxMTI5MTUxMTAwWhcN
MjMxMTI4MTUxMTAwWjB+MQswCQYDVQQGEwJVUzETMBEGA1UECBMKQ2FsaWZvcm5p
YTEWMBQGA1UEBxMNU2FuIEZyYW5jaXNjbzEfMB0GA1UEChMWSW50ZXJuZXQgV2lk
Z2V0cywgSW5jLjEPMA0GA1UECxMGdGVzdGNhMRAwDgYDVQQDEwdleGFtcGxlMIIB
IjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAv30gJg/QmQJ6x432lxokwb60
ZC3PmmUn+gWxw/J/RgDn44gTNbSmVQlWR4PNA9NtqS9uYqeWSlzFE0ODXt/lj6YA
Is/PYqaNunUdB8PCckNX53tcyiUZoxeE9gCZYBzYVKWNDi+xykr+k7X58YeLu4kp
nmmFQbHHFrK9vqH1/BejsE2XBBeTRyJyM4MJmsIfAG02T7uV4w5/5U3OLYN0BqwG
hhBHkMfV6mNbxXrx8fjWXqJyH3dCP/1Xvg0L9B3m8MZ5973C5toHpqXvmX4R1CkW
HYq/WLlQ2GNT1Ie8+atHWXCPUXadqfPZjmxU06hRgeaMjrr4B9zfixtHyPehMQID
AQABo4GEMIGBMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUEDDAKBggrBgEFBQcDATAM
BgNVHRMBAf8EAjAAMB0GA1UdDgQWBBRuRIYMysFkFQ59A8Glqs/Jbe6MXzAfBgNV
HSMEGDAWgBQzW5LyhqKsj719qJtDXeQhuRqryzAMBgNVHREEBTADggEqMA0GCSqG
SIb3DQEBCwUAA4IBAQA47qAZpWtyGAR7uW4vDQVGl3BtuEYeeVlsOORKjko7irzw
vngc8JXweX4RFis2fMcehGFvcllpHgk+EKrWnsF7mzQrvWaBFiEmGaZcu3A2wvx5
SZj3JUv+15xY1Z+/O5EC+7ZruHDOMAj5QSWEpkn/gbV4Kb6PVQZP6w7bu3Fx4GH+
bw49laVM9Tre7yUlcfzbTEfQbCuddQC54fs/S833mK9iu5/cA+DHUFmgMb20+KA1
i3QGPDe5Kl2UsAeMviVA8W0CREgei86aAGdYWhco2bgFezPLc3jWyBbGj9BXFj7v
ahXDv1nzQSOZeWPwimsEEMbzXFrKDAmLoMrkUXep
-----END CERTIFICATE-----
---
Server certificate
subject=/C=US/ST=California/L=San Francisco/O=Internet Widgets, Inc./OU=testca/CN=example
issuer=/C=US/ST=California/L=San Francisco/O=Internet Widgets, Inc./OU=testca/CN=ca
---
Acceptable client certificate CA names
/C=US/ST=California/L=San Francisco/O=Internet Widgets, Inc./OU=testca/CN=ca
Client Certificate Types: RSA sign, ECDSA sign
Requested Signature Algorithms: RSA+SHA256:ECDSA+SHA256:RSA+SHA384:ECDSA+SHA384:RSA+SHA512:ECDSA+SHA512:RSA+SHA1:ECDSA+SHA1
Shared Requested Signature Algorithms: RSA+SHA256:ECDSA+SHA256:RSA+SHA384:ECDSA+SHA384:RSA+SHA512:ECDSA+SHA512:RSA+SHA1:ECDSA+SHA1
Peer signing digest: SHA512
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 1589 bytes and written 297 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID:
    Session-ID-ctx:
    Master-Key: FF737380CE89FAD19C6BB28BF4E2569B863417A5D8447DD9841CD112046DE57BBD527D8D181ACCD2B3F6F9C0588F81A4
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1543514121
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
---

@nicolasnoble
Copy link
Member

@c0b alternatively, it might be possible to try with @grpc/grpc-js, the pure javascript client. If this is indeed a bug in the native code as I'm suspecting it is, then this implementation wouldn't have that problem, hopefully. Depending on what you want to do ultimately, the pure javascript client may or may not fit your requirements, but it may be worth evaluating it while the core bug is investigated and fixed.

@nicolasnoble nicolasnoble changed the title Client call with certificates from Cloudflare fail to parse properly Server certificates from Cloudflare fail to parse properly Nov 30, 2018
@jiangtaoli2016
Copy link
Contributor

I just reproduced and traced the code. The SAN in the cfssl generated certificate is '*' and being rejected by gRPC SSL.

SAN '*' (wildcard without domain) shall be rejected is because it is too general
https://en.wikipedia.org/wiki/Wildcard_certificate

Let me see whether I can find RFC that allows or disallows wildcard without domain. We can revise gRPC code if needed.

@jiangtaoli2016
Copy link
Contributor

By the way, the reason the certificate generated by openssl works is because CN in cert is localhost. But for certificate generated by cfssl, the CN is example, and SAN is *. Thus hostname check failed. If CN in cfssl certificate changes to localhost, it should work as well.

@jiangtaoli2016
Copy link
Contributor

certificate generated by openssl

Certificate:
    Data:
        Version: 1 (0x0)
        Serial Number: 1 (0x1)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = CL, ST = RM, L = Santiago, O = Test, OU = Test, CN = ca
        Validity
            Not Before: Nov 29 12:56:00 2018 GMT
            Not After : Nov 29 12:56:00 2019 GMT
        Subject: C = CL, ST = RM, L = Santiago, O = Test, OU = Server, CN = localhost
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (4096 bit)
                Modulus:
                    00:b6:a7:54:37:74:a4:e0:a0:27:13:05:8f:81:ff:

certificate generated by cfssl from https://github.com/c0b/grpc-helloworld/tree/the-problem/cfssl-certs

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            30:7b:57:94:a0:db:00:77:a7:ca:f9:fd:3d:c9:b4:40:10:6e:ba:df
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = US, ST = California, L = San Francisco, O = "Internet Widgets, Inc.", OU = testca, CN = ca
        Validity
            Not Before: Nov 29 15:11:00 2018 GMT
            Not After : Nov 28 15:11:00 2023 GMT
        Subject: C = US, ST = California, L = San Francisco, O = "Internet Widgets, Inc.", OU = testca, CN = example
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:bf:7d:20:26:0f:d0:99:02:7a:c7:8d:f6:97:1a:
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage:
                TLS Web Server Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Subject Key Identifier:
                6E:44:86:0C:CA:C1:64:15:0E:7D:03:C1:A5:AA:CF:C9:6D:EE:8C:5F
            X509v3 Authority Key Identifier:
                keyid:33:5B:92:F2:86:A2:AC:8F:BD:7D:A8:9B:43:5D:E4:21:B9:1A:AB:CB

            X509v3 Subject Alternative Name:
                DNS:*

@jiangtaoli2016
Copy link
Contributor

It seems that grpc-go accepts wildcard without domain SAN. I need to verify by the source code. If so, we probably want to have consistent behavior across grpc languages.

@jiangtaoli2016
Copy link
Contributor

After consulting with our internal TLS expert, the wildcard without any domain '*' in SAN is not acceptable.

"there's no scenario where it's okay for a CA to generate a "*" certificate. one entity does not control the entire internet"

@nicolasnoble nicolasnoble changed the title Server certificates from Cloudflare fail to parse properly Generic ("*") wildcard certificates aren't accepted Nov 30, 2018
@nicolasnoble
Copy link
Member

nicolasnoble commented Nov 30, 2018

@jiangtaoli2016 now, that being said, the normal Subject should be working in this case, right? With CN=example, and overriding the hostname using ssl_target_name_override=example, even if the SAN is ignored, the CN matches.

Are we simply rejecting the whole certificate because it contains a wildcard that's too generic? Or should we rather ignore the overly-wide SAN but still validate the Subject?

@jiangtaoli2016
Copy link
Contributor

gRPC core is not rejecting TLS certificate with wildcard without domain. The ssl_transport_security just log error and treats this as mismatch. If CN matches peer hostname, it shall still work.

@nicolasnoble
Copy link
Member

Okay, so the wildcard warning is a red herring. But we're still rejecting this peer, even though the CN matches.

Hmm...

@neogeno
Copy link

neogeno commented Dec 4, 2018

For what it's worth, I had a similar problem and solved it by leaving the SSL credentials blank e.g. grpc.credentials.createSsl();

@c0b
Copy link
Author

c0b commented Dec 4, 2018

a similar problem and solved it by leaving the SSL credentials blank e.g. grpc.credentials.createSsl();

No, I tried an empty grpc.credentials.createSsl() object in this test code https://github.com/c0b/grpc-helloworld/blob/the-problem/greeter_client.js that does not work (with errors below); your case may not be similar

E1204 01:33:37.003681226    9091 ssl_transport_security.cc:1229] Handshake failed with fatal error SSL_ERROR_SSL: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed.
2018-12-04T09:33:37.004Z 'ERROR:' { Error: 14 UNAVAILABLE: Connect Failed

@KohliDev
Copy link

We are also facing a similar issue, we are able to connect to the server via CURL (REST API Call) using the cert (having CN=*.sample.com) and OpenSSL version library 1.0.2. But, when we use the same cert and same OpenSSL version library 1.0.2 with GRPC C++ client the connection is failing with below error:

E0322 08:56:35.814190388     154 ssl_transport_security.cc:1227] Handshake failed with fatal error SSL_ERROR_SSL: error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED.
D0322 08:56:35.814215477     154 security_handshaker.cc:129] Security handshake failed: {"created":"@1553244995.814200430","description":"Handshake failed","file":"src/core/lib/security/transport/security_handshaker.cc","file_line":248,"tsi_code":10,"tsi_error":"TSI_PROTOCOL_FAILURE"}
I0322 08:56:35.814223612     154 ev_posix.cc:222]            (fd-trace) fd_shutdown(9)
E0322 08:56:35.814229548     154 lockfree_event.cc:160]      LockfreeEvent::SetShutdown: 0x7f5148010280 curr=(nil) err={"created":"@1553244995.814200430","description":"Handshake failed","file":"src/core/lib/security/transport/security_handshaker.cc","file_line":248,"tsi_code":10,"tsi_error":"TSI_PROTOCOL_FAILURE"}
E0322 08:56:35.814253700     154 lockfree_event.cc:160]      LockfreeEvent::SetShutdown: 0x7f5148010288 curr=0x2 err={"created":"@1553244995.814200430","description":"Handshake failed","file":"src/core/lib/security/transport/security_handshaker.cc","file_line":248,"tsi_code":10,"tsi_error":"TSI_PROTOCOL_FAILURE"}
E0322 08:56:35.814259196     154 lockfree_event.cc:160]      LockfreeEvent::SetShutdown: 0x7f5148010290 curr=(nil) err={"created":"@1553244995.814200430","description":"Handshake failed","file":"src/core/lib/security/transport/security_handshaker.cc","file_line":248,"tsi_code":10,"tsi_error":"TSI_PROTOCOL_FAILURE"}
I0322 08:56:35.814272606     154 handshaker.cc:212]          handshake_manager 0xa33f50: error={"created":"@1553244995.814200430","description":"Handshake failed","file":"src/core/lib/security/transport/security_handshaker.cc","file_line":248,"tsi_code":10,"tsi_error":"TSI_PROTOCOL_FAILURE"} shutdown=0 index=2, args={endpoint=(nil), args=(nil) {size=0: (null)}, read_buffer=(nil) (length=0), exit_early=0}
I0322 08:56:35.814277762     154 handshaker.cc:245]          handshake_manager 0xa33f50: handshaking complete -- scheduling on_handshake_done with error={"created":"@1553244995.814200430","description":"Handshake failed","file":"src/core/lib/security/transport/security_handshaker.cc","file_line":248,"tsi_code":10,"tsi_error":"TSI_PROTOCOL_FAILURE"}
I0322 08:56:35.814282833     154 timer_generic.cc:467]       TIMER 0xa33fd0: CANCEL pending=true
I0322 08:56:35.814334483     154 resource_quota.cc:818]      RQ anonymous_pool_a33980 ipv4:23.208.223.73:443: free 8192; free_pool -> 8192
I0322 08:56:35.814339365     154 resource_quota.cc:818]      RQ anonymous_pool_a33980 ipv4:23.208.223.73:443: free 8192; free_pool -> 16384
I0322 08:56:35.814343788     154 ev_posix.cc:215]            (fd-trace) grpc_fd_orphan, fd:9 closed
I0322 08:56:35.814387357     154 connectivity_state.cc:164]  SET: 0x7f5148008300 subchannel: CONNECTING --> TRANSIENT_FAILURE [connect_failed] error=0xa47f50 {"created":"@1553244995.814376968","description":"Connect Failed","file":"src/core/ext/filters/client_channel/subchannel.cc","file_line":663,"grpc_status":14,"referenced_errors":[{"created":"@1553244995.814200430","description":"Handshake failed","file":"src/core/lib/security/transport/security_handshaker.cc","file_line":248,"tsi_code":10,"tsi_error":"TSI_PROTOCOL_FAILURE"}]}
I0322 08:56:35.814393209     154 connectivity_state.cc:190]  NOTIFY: 0x7f5148008300 subchannel: 0x7f51480108e8
I0322 08:56:35.814397204     154 subchannel.cc:668]          Connect failed: {"created":"@1553244995.814200430","description":"Handshake failed","file":"src/core/lib/security/transport/security_handshaker.cc","file_line":248,"tsi_code":10,"tsi_error":"TSI_PROTOCOL_FAILURE"}
I0322 08:56:35.814402101     154 resource_quota.cc:508]      RU shutdown 0xa33da0
I0322 08:56:35.814410485     154 subchannel_list.h:404]      [pick_first 0x7f5148003210] subchannel list 0x7f5148003650 index 0 of 3 (subchannel 0x7f5148008230): connectivity changed: state=TRANSIENT_FAILURE, error={"created":"@1553244995.814376968","description":"Connect Failed","file":"src/core/ext/filters/client_channel/subchannel.cc","file_line":663,"grpc_status":14,"referenced_errors":[{"created":"@1553244995.814200430","description":"Handshake failed","file":"src/core/lib/security/transport/security_handshaker.cc","file_line":248,"tsi_code":10,"tsi_error":"TSI_PROTOCOL_FAILURE"}]}, shutting_down=0
I0322 08:56:35.814416924     154 subchannel_list.h:332]      [pick_first 0x7f5148003210] subchannel list 0x7f5148003650 index 0 of 3 (subchannel 0x7f5148008230): stopping connectivity watch

We are getting GRPC error code 14, please guide on this issue?

@Asgoret
Copy link

Asgoret commented Apr 29, 2019

Hi! I have similar issue. I have wildcard san ssl certificate for OKD and his apps. So, when i use:
grpcurl {some_value}.apps.{some_value}.{some_value}.{some_value}:443 my.custom.server.Service/Method i got:
Failed to dial target host "{some_value}.apps.{some_value}.{some_value}.{some_value}:443": x509: certificate is valid for *.{some_value}.{some_value}, {some_value}.{some_value}, not {some_value}.apps.{some_value}.{some_value}.{some_value}
Any ideas?

UPD: Error was in NGINX

@vbar
Copy link

vbar commented Nov 21, 2019

So, what is the proper way to generate a certificate for an edge server, whose domain is not known (controlled by the customer during future deployment) when the certificate is generated?

@sukhvir148
Copy link

Is there a resolution to this issue ?

@moloch--
Copy link

moloch-- commented May 3, 2020

This still seems to be a problem, as I'm encountering the same problem. There's no way to validate a certificate and skip only hostname validation. The checkServerIdentity callback, which is how you're supposed to do this, isn't even called.

@gnarea
Copy link

gnarea commented May 5, 2020

Workaround: Instead of setting checkServerIdentity, download the certificate and pass it to grpc.credentials.createSsl().

You can download it with openssl s_client or https://nodejs.org/api/tls.html#tls_tlssocket_getpeercertificate_detailed, for example.

gnarea added a commit to relaycorp/relaynet-courier-android that referenced this issue May 9, 2020
The [Node.js gRPC client has a bug](grpc/grpc-node#663) that requires
downloading the TLS certificate before initiating the gRPC connection
[as a workaround](grpc/grpc-node#663 (comment)). When that's
done, `transportAttrs` is `null` (a behaviour that's undocumented), so it led to unhandled exceptions.
gnarea added a commit to relaycorp/relaynet-courier-android that referenced this issue May 11, 2020
The [Node.js gRPC client has a bug](grpc/grpc-node#663) that requires
downloading the TLS certificate before initiating the gRPC connection
[as a workaround](grpc/grpc-node#663 (comment)). When that's
done, `transportAttrs` is `null` (a behaviour that's undocumented), so it led to unhandled exceptions.
gnarea added a commit to relaycorp/relaynet-cogrpc-js that referenced this issue May 22, 2020
gnarea added a commit to relaycorp/relaynet-cogrpc-js that referenced this issue May 22, 2020
* wip

* fix: Download TLS certificate upfront when connecting to a courier

As a workaround for grpc/grpc-node#663

Fixes #22.

* Fix formatting issue

* refactor
@gnarea
Copy link

gnarea commented May 23, 2020

Here's an MIT-licensed implementation of the workaround I described above, in case it helps anyone: https://github.com/relaycorp/relaynet-cogrpc-js/blob/6c04ac9777ffafbbcc428b3d855e1b81db3dd866/src/lib/client.ts#L143-L176

@u007
Copy link

u007 commented Jun 11, 2020

Here's an MIT-licensed implementation of the workaround I described above, in case it helps anyone: https://github.com/relaycorp/relaynet-cogrpc-js/blob/6c04ac9777ffafbbcc428b3d855e1b81db3dd866/src/lib/client.ts#L143-L176

does not work on my end

@u007
Copy link

u007 commented Jun 11, 2020

still no solution yet? im facing same issue when using nginx as reverse grpc, even with certificate and key set on server and client. when connecting directly to my server instance with certificate, it works

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests