Extend local channel/server credentials to support TCP loopback #17370

yihuazhang · 2018-12-03T04:07:11Z

The PR enables local channel/server credentials to support both UDS and TCP loopback addresses (IPV4 and IPV6). It will be used as an substituent for insecure channel/server credentials that are used in UDS and TCP loopback connections.

This PR

Modifies grpc_security_connector_check_peer API to further take grpc_endpoint as an argument. An alternative is to pass grpc_endpoint during the handshake which requires the modification of TSI interface.
Applies getsockname to get socket information that will be used to determine if the socket address is type of either UDS or TCP loopback (IPV4 and IPV6).
Updates h2_local.cc to further validate the behavior of local credentials when used in TCP loopback addresses.

This change is

grpc-testing · 2018-12-03T05:01:08Z

Objective-C binary sizes
*****************STATIC******************
  New size                      Old size
 2,020,418      Total (>)      2,020,410

 No significant differences in binary sizes

***************FRAMEWORKS****************
  New size                      Old size
11,174,958      Total (>)     11,174,082

 No significant differences in binary sizes

grpc-testing · 2018-12-04T01:36:33Z

****************************************************************

libgrpc.so

     VM SIZE                                                                                  FILE SIZE
 ++++++++++++++ GROWING                                                                    ++++++++++++++
  +6.2%    +104 src/core/lib/security/security_connector/local/local_security_connector.cc    +104  +6.2%
      [NEW]    +592 local_check_peer                                                              +592  [NEW]
      +1.5%      +1 [Unmapped]                                                                      +1  +1.5%

 -+-+-+-+-+-+-+ MIXED                                                                      +-+-+-+-+-+-+-
  -0.0%     -40 [None]                                                                     +1.53Ki  +0.0%

  +0.0%     +64 TOTAL                                                                      +1.63Ki  +0.0%


****************************************************************

libgrpc++.so

     VM SIZE              FILE SIZE
 ++++++++++++++ GROWIN ++++++++++++++
  [ = ]       0 [None]    +152  +0.0%

  [ = ]       0 TOTAL     +152  +0.0%

grpc-testing · 2018-12-04T01:41:30Z

[trickle] No significant performance differences

grpc-testing · 2018-12-04T02:18:07Z

Objective-C binary sizes
*****************STATIC******************
  New size                      Old size
 2,020,418      Total (>)      2,020,410

 No significant differences in binary sizes

***************FRAMEWORKS****************
  New size                      Old size
11,174,416      Total (>)     11,174,077

 No significant differences in binary sizes

grpc-testing · 2018-12-04T02:29:46Z

Corrupt JSON data (indicates timeout or crash): 
    bm_call_create.BM_IsolatedFilter_ClientChannelFilter_NoOp_.counters.new: 10
    bm_call_create.BM_IsolatedFilter_ClientChannelFilter_NoOp_.counters.old: 10


[microbenchmarks] No significant performance differences

grpc-testing · 2018-12-04T05:02:02Z

****************************************************************

libgrpc.so

     VM SIZE                                                                                  FILE SIZE
 ++++++++++++++ GROWING                                                                    ++++++++++++++
  +6.2%    +104 src/core/lib/security/security_connector/local/local_security_connector.cc    +104  +6.2%
      [NEW]    +592 local_check_peer                                                              +592  [NEW]
      +1.5%      +1 [Unmapped]                                                                      +1  +1.5%

 -+-+-+-+-+-+-+ MIXED                                                                      +-+-+-+-+-+-+-
  -0.0%     -40 [None]                                                                     +1.55Ki  +0.0%

  +0.0%     +64 TOTAL                                                                      +1.66Ki  +0.0%


****************************************************************

libgrpc++.so

     VM SIZE              FILE SIZE
 ++++++++++++++ GROWIN ++++++++++++++
  [ = ]       0 [None]    +128  +0.0%

  [ = ]       0 TOTAL     +128  +0.0%

grpc-testing · 2018-12-04T05:11:13Z

[trickle] No significant performance differences

grpc-testing · 2018-12-04T05:42:09Z

Objective-C binary sizes
*****************STATIC******************
  New size                      Old size
 2,019,726      Total (>)      2,019,718

 No significant differences in binary sizes

***************FRAMEWORKS****************
  New size                      Old size
11,154,454      Total (>)     11,154,122

 No significant differences in binary sizes

grpc-testing · 2018-12-04T05:54:51Z

Corrupt JSON data (indicates timeout or crash): 
    bm_call_create.BM_IsolatedFilter_ClientChannelFilter_NoOp_.counters.new: 10
    bm_call_create.BM_IsolatedFilter_ClientChannelFilter_NoOp_.counters.old: 10


[microbenchmarks] No significant performance differences

grpc-testing · 2018-12-04T22:26:50Z

****************************************************************

libgrpc.so

     VM SIZE                                                                                  FILE SIZE
 ++++++++++++++ GROWING                                                                    ++++++++++++++
  +0.0%      +8 [None]                                                                     +1.57Ki  +0.0%
  +7.1%    +120 src/core/lib/security/security_connector/local/local_security_connector.cc    +120  +7.1%
      [NEW]    +608 local_check_peer                                                              +608  [NEW]
      +1.5%      +1 [Unmapped]                                                                      +1  +1.5%

  +0.0%    +128 TOTAL                                                                      +1.69Ki  +0.0%


****************************************************************

libgrpc++.so

     VM SIZE              FILE SIZE
 ++++++++++++++ GROWIN ++++++++++++++
  [ = ]       0 [None]    +128  +0.0%

  [ = ]       0 TOTAL     +128  +0.0%

grpc-testing · 2018-12-04T22:34:21Z

[trickle] No significant performance differences

grpc-testing · 2018-12-04T23:10:38Z

Objective-C binary sizes
*****************STATIC******************
  New size                      Old size
 2,019,726      Total (>)      2,019,718

 No significant differences in binary sizes

***************FRAMEWORKS****************
  New size                      Old size
11,154,454      Total (>)     11,154,120

 No significant differences in binary sizes

grpc-testing · 2018-12-04T23:19:57Z

Corrupt JSON data (indicates timeout or crash): 
    bm_call_create.BM_IsolatedFilter_ClientChannelFilter_NoOp_.counters.new: 10
    bm_call_create.BM_IsolatedFilter_ClientChannelFilter_NoOp_.counters.old: 10


[microbenchmarks] No significant performance differences

jiangtaoli2016

Thanks Yihua for implementation. Overall, it looks good. Some minor comments.

jiangtaoli2016 · 2018-12-04T23:49:50Z

src/core/lib/security/security_connector/local/local_security_connector.cc

@@ -209,7 +240,7 @@ grpc_security_status grpc_local_channel_security_connector_create(
  c->base.check_call_host = local_check_call_host;
  c->base.cancel_check_call_host = local_cancel_check_call_host;
  c->base.base.url_scheme =
-      creds->connect_type == UDS ? GRPC_UDS_URL_SCHEME : nullptr;
+      creds->connect_type == UDS ? GRPC_UDS_URL_SCHEME : GRPC_SSL_URL_SCHEME;


In the case of Local TCP, we shall not set as GRPC_SSL_URL_SCHEME. It is not https. Leave it as nullptr is fine -- ALTS does not have URL scheme either.

I see. Thanks for pointing it out.

jiangtaoli2016 · 2018-12-04T23:57:17Z

src/core/lib/security/security_connector/local/local_security_connector.cc

+        reinterpret_cast<grpc_sockaddr_in6*>(resolved_addr.addr);
+    // UDS
+    if (grpc_is_unix_socket(&resolved_addr)) {
+      is_endpoint_ok = true;


nit: renaming to is_endpoint_local has better readability

jiangtaoli2016 · 2018-12-05T00:03:25Z

src/core/lib/security/security_connector/local/local_security_connector.cc

                             grpc_auth_context** auth_context,
                             grpc_closure* on_peer_checked) {
+  int fd = grpc_endpoint_get_fd(ep);
+  grpc_resolved_address resolved_addr;


Is resolved_addr local address or peer address?

It corresponds to the socket address of a local endpoint. If we need to get the information about the peer's endpoint, getpeername should be used instead.

Could you please verify that if local socket address is loopback, then the connection (i.e., peer) must be local.

jiangtaoli2016 · 2018-12-05T00:23:50Z

src/core/lib/security/security_connector/local/local_security_connector.cc

@@ -178,23 +215,17 @@ grpc_security_status grpc_local_channel_security_connector_create(
        "Invalid arguments to grpc_local_channel_security_connector_create()");
    return GRPC_SECURITY_ERROR;
  }
-  // Check if local_connect_type is UDS. Only UDS is supported for now.
+  // Perform sanity check on UDS address.


Please comment that TCP local connection check is during check_peer.

jiangtaoli2016 · 2018-12-05T00:30:39Z

include/grpc/grpc_security_constants.h

 */
-typedef enum { UDS = 0 } grpc_local_connect_type;
+typedef enum { UDS = 0, TCP_LOOPBACK } grpc_local_connect_type;


rename to LOCAL_TCP?

SG. The change is applied.

jiangtaoli2016 · 2018-12-05T00:34:27Z

src/core/lib/security/security_connector/local/local_security_connector.cc

+        reinterpret_cast<grpc_sockaddr_in*>(resolved_addr.addr);
+    grpc_sockaddr_in6* addr6 =
+        reinterpret_cast<grpc_sockaddr_in6*>(resolved_addr.addr);
+    // UDS


grpc_local_connect_type in credential is not really used to check endpoint. We want to check endpoint is UDS and local_connect_type in base.channel_creds is also UDS to make sure they are consistent. Same for local TCP connections.

Good point. The change is applied.

yihuazhang · 2018-12-05T02:31:53Z

Thanks Jiangtao for the quick review. All comments are addressed. PTAL.

grpc-testing · 2018-12-05T02:34:49Z

****************************************************************

libgrpc.so

     VM SIZE                                                                                            FILE SIZE
 ++++++++++++++ GROWING                                                                              ++++++++++++++
  +0.0%    +160 [None]                                                                               +10.2Ki  +0.1%
   +26%    +432 src/core/lib/security/security_connector/local/local_security_connector.cc              +432   +26%
      [NEW]    +640 local_check_peer(grpc_security_connector*, tsi_peer, grpc_endpoint*, grpc_auth_conte    +640  [NEW]
      [NEW]    +137 local_server_check_peer                                                                 +137  [NEW]
      [NEW]    +137 local_channel_check_peer                                                                +137  [NEW]
       +35%     +23 [Unmapped]                                                                               +23   +35%

  +0.0%    +592 TOTAL                                                                                +10.6Ki  +0.1%


****************************************************************

libgrpc++.so

     VM SIZE              FILE SIZE
 ++++++++++++++ GROWIN ++++++++++++++
  [ = ]       0 [None]    +128  +0.0%

  [ = ]       0 TOTAL     +128  +0.0%

grpc-testing · 2018-12-05T02:45:13Z

[trickle] No significant performance differences

grpc-testing · 2018-12-05T03:17:33Z

Objective-C binary sizes
*****************STATIC******************
  New size                      Old size
 2,019,726      Total (>)      2,019,718

 No significant differences in binary sizes

***************FRAMEWORKS****************
  New size                      Old size
 3,948,402       Core (>)      3,947,382

11,155,141      Total (>)     11,154,127

grpc-testing · 2018-12-05T03:27:33Z

Corrupt JSON data (indicates timeout or crash): 
    bm_call_create.BM_IsolatedFilter_ClientChannelFilter_NoOp_.counters.new: 10
    bm_call_create.BM_IsolatedFilter_ClientChannelFilter_NoOp_.counters.old: 10


[microbenchmarks] No significant performance differences

grpc-testing · 2018-12-09T20:34:21Z

****************************************************************

libgrpc.so

     VM SIZE                                                                                            FILE SIZE
 ++++++++++++++ GROWING                                                                              ++++++++++++++
  +0.0%    +136 [None]                                                                               +6.93Ki  +0.1%
   +30%    +496 src/core/lib/security/security_connector/local/local_security_connector.cc              +496   +30%
      [NEW]    +696 local_check_peer(grpc_security_connector*, tsi_peer, grpc_endpoint*, grpc_auth_conte    +696  [NEW]
      [NEW]    +137 local_server_check_peer                                                                 +137  [NEW]
      [NEW]    +137 local_channel_check_peer                                                                +137  [NEW]
       +48%     +31 [Unmapped]                                                                               +31   +48%

  +0.0%    +632 TOTAL                                                                                +7.41Ki  +0.1%


****************************************************************

libgrpc++.so

     VM SIZE              FILE SIZE
 ++++++++++++++ GROWIN ++++++++++++++
  [ = ]       0 [None]     +64  +0.0%

  [ = ]       0 TOTAL      +64  +0.0%

grpc-testing · 2018-12-09T20:43:58Z

[trickle] No significant performance differences

grpc-testing · 2018-12-13T00:25:21Z

[microbenchmarks] No significant performance differences

grpc-testing · 2018-12-13T05:12:51Z

****************************************************************

libgrpc.so

     VM SIZE                                                                                            FILE SIZE
 ++++++++++++++ GROWING                                                                              ++++++++++++++
  +0.0%    +136 [None]                                                                               +10.9Ki  +0.1%
   +30%    +496 src/core/lib/security/security_connector/local/local_security_connector.cc              +496   +30%
      [NEW]    +696 local_check_peer(grpc_security_connector*, tsi_peer, grpc_endpoint*, grpc_auth_conte    +696  [NEW]
      [NEW]    +137 local_server_check_peer                                                                 +137  [NEW]
      [NEW]    +137 local_channel_check_peer                                                                +137  [NEW]
       +48%     +31 [Unmapped]                                                                               +31   +48%

  +0.0%    +632 TOTAL                                                                                +11.4Ki  +0.1%


****************************************************************

libgrpc++.so

     VM SIZE              FILE SIZE
 ++++++++++++++ GROWIN ++++++++++++++
  [ = ]       0 [None]     +48  +0.0%

  [ = ]       0 TOTAL      +48  +0.0%

grpc-testing · 2018-12-13T05:27:16Z

[trickle] No significant performance differences

grpc-testing · 2018-12-13T05:30:48Z

****************************************************************

libgrpc.so

     VM SIZE                                                                                            FILE SIZE
 ++++++++++++++ GROWING                                                                              ++++++++++++++
  +0.0%    +136 [None]                                                                               +10.9Ki  +0.1%
   +30%    +496 src/core/lib/security/security_connector/local/local_security_connector.cc              +496   +30%
      [NEW]    +696 local_check_peer(grpc_security_connector*, tsi_peer, grpc_endpoint*, grpc_auth_conte    +696  [NEW]
      [NEW]    +137 local_server_check_peer                                                                 +137  [NEW]
      [NEW]    +137 local_channel_check_peer                                                                +137  [NEW]
       +48%     +31 [Unmapped]                                                                               +31   +48%

  +0.0%    +632 TOTAL                                                                                +11.4Ki  +0.1%


****************************************************************

libgrpc++.so

     VM SIZE              FILE SIZE
 ++++++++++++++ GROWIN ++++++++++++++
  [ = ]       0 [None]     +48  +0.0%

  [ = ]       0 TOTAL      +48  +0.0%

grpc-testing · 2018-12-13T05:38:55Z

[trickle] No significant performance differences

grpc-testing · 2018-12-13T06:05:50Z

[microbenchmarks] No significant performance differences

grpc-testing · 2018-12-13T06:09:28Z

****************************************************************

libgrpc.so

     VM SIZE                                                                                            FILE SIZE
 ++++++++++++++ GROWING                                                                              ++++++++++++++
  +0.0%    +136 [None]                                                                               +10.9Ki  +0.1%
   +30%    +496 src/core/lib/security/security_connector/local/local_security_connector.cc              +496   +30%
      [NEW]    +696 local_check_peer(grpc_security_connector*, tsi_peer, grpc_endpoint*, grpc_auth_conte    +696  [NEW]
      [NEW]    +137 local_server_check_peer                                                                 +137  [NEW]
      [NEW]    +137 local_channel_check_peer                                                                +137  [NEW]
       +48%     +31 [Unmapped]                                                                               +31   +48%

  +0.0%    +632 TOTAL                                                                                +11.4Ki  +0.1%


****************************************************************

libgrpc++.so

     VM SIZE              FILE SIZE
 ++++++++++++++ GROWIN ++++++++++++++
  [ = ]       0 [None]     +48  +0.0%

  [ = ]       0 TOTAL      +48  +0.0%

grpc-testing · 2018-12-13T06:14:06Z

Objective-C binary sizes
*****************STATIC******************
  New size                      Old size
 2,020,878      Total (>)      2,020,870

 No significant differences in binary sizes

***************FRAMEWORKS****************
  New size                      Old size
 3,971,402       Core (>)      3,970,386

11,178,141      Total (>)     11,177,125

grpc-testing · 2018-12-13T06:16:05Z

[trickle] No significant performance differences

grpc-testing · 2018-12-13T06:22:27Z

[microbenchmarks] No significant performance differences

grpc-testing · 2018-12-13T06:50:55Z

Objective-C binary sizes
*****************STATIC******************
  New size                      Old size
 2,020,878      Total (>)      2,020,870

 No significant differences in binary sizes

***************FRAMEWORKS****************
  New size                      Old size
 3,971,402       Core (>)      3,970,386

11,178,135      Total (>)     11,177,132

grpc-testing · 2018-12-13T07:02:58Z

[microbenchmarks] No significant performance differences

grpc-testing · 2018-12-16T22:55:47Z

****************************************************************

libgrpc.so

     VM SIZE                                                                                            FILE SIZE
 ++++++++++++++ GROWING                                                                              ++++++++++++++
  +0.0%    +144 [None]                                                                               +3.64Ki  +0.0%
      +0.0%    +112 [Unmapped]                                                                           +3.64Ki  +0.0%
      +2.8%     +32 [None]                                                                                     0  [ = ]
  +6.6%    +192 src/core/lib/security/security_connector/local/local_security_connector.cc              +192  +6.6%
      [NEW] +1.19Ki (anonymous namespace)::local_check_peer(grpc_security_connector*, tsi_peer, grpc_end +1.19Ki  [NEW]
      [NEW]     +24 (anonymous namespace)::grpc_local_server_security_connector::check_peer                  +24  [NEW]
      [NEW]     +24 (anonymous namespace)::grpc_local_channel_security_connector::check_peer                 +24  [NEW]
  +0.1%     +16 src/core/lib/security/transport/security_handshaker.cc                                   +16  +0.1%
      +5.6%      +9 [Unmapped]                                                                                +9  +5.6%
      +3.8%      +7 check_peer_locked                                                                         +7  +3.8%

  +0.0%    +352 TOTAL                                                                                +3.84Ki  +0.0%


****************************************************************

libgrpc++.so

     VM SIZE              FILE SIZE
 ++++++++++++++ GROWIN ++++++++++++++
  [ = ]       0 [None]     +80  +0.0%

  [ = ]       0 TOTAL      +80  +0.0%

grpc-testing · 2018-12-16T23:03:44Z

[trickle] No significant performance differences

grpc-testing · 2018-12-16T23:39:44Z

Objective-C binary sizes
*****************STATIC******************
  New size                      Old size
 2,026,850      Total (>)      2,026,842

 No significant differences in binary sizes

***************FRAMEWORKS****************
  New size                      Old size
11,325,193      Total (>)     11,324,729

 No significant differences in binary sizes

grpc-testing · 2018-12-16T23:47:58Z

[microbenchmarks] No significant performance differences

yihuazhang · 2018-12-17T17:14:42Z

Code has been further revised as follows:

Synced with Move security credentials, connectors, and auth context to C++ #17291.
Refactored h2_local end2end test into h2_local_uds, h2_local_ipv4, and h2_local_ipv6, and created local_util that contains a shared util library for those three tests.

grpc-testing · 2018-12-17T17:36:18Z

****************************************************************

libgrpc.so

     VM SIZE                                                                                            FILE SIZE
 ++++++++++++++ GROWING                                                                              ++++++++++++++
  +0.0%    +144 [None]                                                                               +3.64Ki  +0.0%
      +0.0%    +112 [Unmapped]                                                                           +3.64Ki  +0.0%
      +2.8%     +32 [None]                                                                                     0  [ = ]
  +6.6%    +192 src/core/lib/security/security_connector/local/local_security_connector.cc              +192  +6.6%
      [NEW] +1.19Ki (anonymous namespace)::local_check_peer(grpc_security_connector*, tsi_peer, grpc_end +1.19Ki  [NEW]
      [NEW]     +24 (anonymous namespace)::grpc_local_server_security_connector::check_peer                  +24  [NEW]
      [NEW]     +24 (anonymous namespace)::grpc_local_channel_security_connector::check_peer                 +24  [NEW]
  +0.1%     +16 src/core/lib/security/transport/security_handshaker.cc                                   +16  +0.1%
      +5.6%      +9 [Unmapped]                                                                                +9  +5.6%
      +3.8%      +7 check_peer_locked                                                                         +7  +3.8%

  +0.0%    +352 TOTAL                                                                                +3.84Ki  +0.0%


****************************************************************

libgrpc++.so

     VM SIZE              FILE SIZE
 ++++++++++++++ GROWIN ++++++++++++++
  [ = ]       0 [None]     +80  +0.0%

  [ = ]       0 TOTAL      +80  +0.0%

grpc-testing · 2018-12-17T17:48:00Z

[trickle] No significant performance differences

grpc-testing · 2018-12-17T18:20:23Z

Objective-C binary sizes
*****************STATIC******************
  New size                      Old size
 2,026,850      Total (>)      2,026,842

 No significant differences in binary sizes

***************FRAMEWORKS****************
  New size                      Old size
11,325,193      Total (>)     11,324,734

 No significant differences in binary sizes

grpc-testing · 2018-12-17T18:28:40Z

[microbenchmarks] No significant performance differences

yihuazhang · 2018-12-17T19:19:22Z

Known failures: #13379, #17526

yihuazhang force-pushed the local_tcp_loopback branch from af418a9 to 56b8449 Compare December 3, 2018 04:12

yihuazhang force-pushed the local_tcp_loopback branch from 56b8449 to 6f0e7cd Compare December 4, 2018 01:27

yihuazhang force-pushed the local_tcp_loopback branch from 6f0e7cd to 35d7248 Compare December 4, 2018 04:55

yihuazhang added the release notes: no Indicates if PR should not be in release notes label Dec 4, 2018

yihuazhang force-pushed the local_tcp_loopback branch from 35d7248 to 516eae6 Compare December 4, 2018 22:21

yihuazhang requested review from markdroth and jiangtaoli2016 December 4, 2018 22:34

jiangtaoli2016 reviewed Dec 5, 2018

View reviewed changes

jiangtaoli2016 approved these changes Dec 7, 2018

View reviewed changes

yihuazhang force-pushed the local_tcp_loopback branch from 7320100 to 251abfc Compare December 9, 2018 20:29

yihuazhang force-pushed the local_tcp_loopback branch from 42861db to 82fe5ef Compare December 13, 2018 05:08

extend local credentials to support tcp loopback

0531d3d

yihuazhang force-pushed the local_tcp_loopback branch from d051bce to 0531d3d Compare December 16, 2018 22:50

yihuazhang added the kokoro:force-run label Dec 17, 2018

grpc-kokoro removed the kokoro:force-run label Dec 17, 2018

yihuazhang merged commit 3f7e26e into grpc:master Dec 17, 2018

yihuazhang added the lang/core label Jan 14, 2019

lock bot locked as resolved and limited conversation to collaborators Apr 14, 2019

Extend local channel/server credentials to support TCP loopback #17370

Extend local channel/server credentials to support TCP loopback #17370

Conversation

yihuazhang commented Dec 3, 2018 • edited

grpc-testing commented Dec 3, 2018

grpc-testing commented Dec 4, 2018

grpc-testing commented Dec 4, 2018

grpc-testing commented Dec 4, 2018

grpc-testing commented Dec 4, 2018

grpc-testing commented Dec 4, 2018

grpc-testing commented Dec 4, 2018

grpc-testing commented Dec 4, 2018

grpc-testing commented Dec 4, 2018

grpc-testing commented Dec 4, 2018

grpc-testing commented Dec 4, 2018

grpc-testing commented Dec 4, 2018

grpc-testing commented Dec 4, 2018

jiangtaoli2016 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yihuazhang Dec 5, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yihuazhang commented Dec 5, 2018

grpc-testing commented Dec 5, 2018

grpc-testing commented Dec 5, 2018

grpc-testing commented Dec 5, 2018

grpc-testing commented Dec 5, 2018

grpc-testing commented Dec 9, 2018

grpc-testing commented Dec 9, 2018

grpc-testing commented Dec 13, 2018

grpc-testing commented Dec 13, 2018

grpc-testing commented Dec 13, 2018

grpc-testing commented Dec 13, 2018

grpc-testing commented Dec 13, 2018

grpc-testing commented Dec 13, 2018

grpc-testing commented Dec 13, 2018

grpc-testing commented Dec 13, 2018

grpc-testing commented Dec 13, 2018

grpc-testing commented Dec 13, 2018

grpc-testing commented Dec 13, 2018

grpc-testing commented Dec 13, 2018

grpc-testing commented Dec 16, 2018

grpc-testing commented Dec 16, 2018

grpc-testing commented Dec 16, 2018

grpc-testing commented Dec 16, 2018

yihuazhang commented Dec 17, 2018

grpc-testing commented Dec 17, 2018

grpc-testing commented Dec 17, 2018

grpc-testing commented Dec 17, 2018

grpc-testing commented Dec 17, 2018

yihuazhang commented Dec 17, 2018

yihuazhang commented Dec 3, 2018 •

edited

yihuazhang Dec 5, 2018 •

edited