Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose local credentials on Python layer #19971

Merged
merged 11 commits into from
Aug 23, 2019
61 changes: 61 additions & 0 deletions src/python/grpcio/grpc/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -1744,6 +1744,64 @@ def dynamic_ssl_server_credentials(initial_certificate_configuration,
certificate_configuration_fetcher, require_client_authentication))


@enum.unique
class LocalConnectionType(enum.Enum):
"""Types of local connection for local credential creation.

Attributes:
UDS: Unix domain socket connections
LOCAL_TCP: Local TCP connections.
"""
UDS = _cygrpc.LocalConnectionType.uds
LOCAL_TCP = _cygrpc.LocalConnectionType.local_tcp


def local_channel_credentials(local_connect_type=LocalConnectionType.LOCAL_TCP):
"""Creates a local ChannelCredentials used for local connections.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're going to want a longer docstring describing what local credentials are and perhaps naming potential use cases. Without context, this won't be useful to very many users.

Copy link
Contributor

@gnossen gnossen Aug 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh. Apparently, this is still considered an experimental API within core. At any rate, we probably want some explanation about exactly what sort of authentication/authorization mechanism is used. Without looking into the code, even I'm still scratching my head about what exactly this feature is.

Edit: This comment is the most information I can find on the topic. Maybe something like "Peer authentication and channel confidentiality are established for channels using local_channel_credentials by native kernel permission checks."

CC @yihuazhang

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Lidi for exposing the local credential in OSS.

The current status is C-core, Java and Go teams agreed on the semantics of channel and call credentials (I shared with you and Lidi) which will be a building block for designing local credentials in all languages. I still need to update the existing API's in those three languages to conform to the semantics (Sorry for taking so long), and after that we can design local credentials for Java and Go and remove experimental namespace in C-core.

W.r.t explanation, TCP loopback does not provide either peer authentication or channel confidentiality while UDS provides both of them and can be treated as secure as transport security.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gnossen PTAL at the added comments.
@yihuazhang I added this because in the past users is complaining that API disparity prevents them from using secure_channel.


This is an EXPERIMENTAL API.

Local credentials are used by local TCP endpoints (e.g. localhost:10000)
also UDS connections. It allows them to create secure channel, hence
transmitting call credentials become possible.

It is useful for 1) eliminating insecure_channel usage; 2) enable unit
testing for call credentials without setting up secrets.

Args:
local_connect_type: Local connection type (either
grpc.LocalConnectionType.UDS or grpc.LocalConnectionType.LOCAL_TCP)

Returns:
A ChannelCredentials for use with a local Channel
"""
return ChannelCredentials(
_cygrpc.channel_credentials_local(local_connect_type.value))


def local_server_credentials(local_connect_type=LocalConnectionType.LOCAL_TCP):
"""Creates a local ServerCredentials used for local connections.

This is an EXPERIMENTAL API.

Local credentials are used by local TCP endpoints (e.g. localhost:10000)
also UDS connections. It allows them to create secure channel, hence
transmitting call credentials become possible.

It is useful for 1) eliminating insecure_channel usage; 2) enable unit
testing for call credentials without setting up secrets.

Args:
local_connect_type: Local connection type (either
grpc.LocalConnectionType.UDS or grpc.LocalConnectionType.LOCAL_TCP)

Returns:
A ServerCredentials for use with a local Server
"""
return ServerCredentials(
_cygrpc.server_credentials_local(local_connect_type.value))


def channel_ready_future(channel):
"""Creates a Future that tracks when a Channel is ready.

Expand Down Expand Up @@ -1913,6 +1971,7 @@ class Compression(enum.IntEnum):
'ClientCallDetails',
'ServerCertificateConfiguration',
'ServerCredentials',
'LocalConnectionType',
'UnaryUnaryMultiCallable',
'UnaryStreamMultiCallable',
'StreamUnaryMultiCallable',
Expand All @@ -1939,6 +1998,8 @@ class Compression(enum.IntEnum):
'access_token_call_credentials',
'composite_call_credentials',
'composite_channel_credentials',
'local_channel_credentials',
'local_server_credentials',
'ssl_server_credentials',
'ssl_server_certificate_configuration',
'dynamic_ssl_server_credentials',
Expand Down
5 changes: 5 additions & 0 deletions src/python/grpcio/grpc/_cython/_cygrpc/credentials.pxd.pxi
Original file line number Diff line number Diff line change
Expand Up @@ -97,3 +97,8 @@ cdef class ServerCredentials:
cdef object cert_config_fetcher
# whether C-core has asked for the initial_cert_config
cdef bint initial_cert_config_fetched


cdef class LocalChannelCredentials(ChannelCredentials):

cdef grpc_local_connect_type _local_connect_type
22 changes: 22 additions & 0 deletions src/python/grpcio/grpc/_cython/_cygrpc/credentials.pyx.pxi
Original file line number Diff line number Diff line change
Expand Up @@ -328,3 +328,25 @@ cdef grpc_ssl_certificate_config_reload_status _server_cert_config_fetcher_wrapp
cert_config.c_ssl_pem_key_cert_pairs_count)
return GRPC_SSL_CERTIFICATE_CONFIG_RELOAD_NEW


class LocalConnectionType:
uds = UDS
local_tcp = LOCAL_TCP

cdef class LocalChannelCredentials(ChannelCredentials):

def __cinit__(self, grpc_local_connect_type local_connect_type):
self._local_connect_type = local_connect_type

cdef grpc_channel_credentials *c(self) except *:
cdef grpc_local_connect_type local_connect_type
local_connect_type = self._local_connect_type
return grpc_local_credentials_create(local_connect_type)

def channel_credentials_local(grpc_local_connect_type local_connect_type):
return LocalChannelCredentials(local_connect_type)

def server_credentials_local(grpc_local_connect_type local_connect_type):
cdef ServerCredentials credentials = ServerCredentials()
credentials.c_credentials = grpc_local_server_credentials_create(local_connect_type)
return credentials
12 changes: 12 additions & 0 deletions src/python/grpcio/grpc/_cython/_cygrpc/grpc.pxi
Original file line number Diff line number Diff line change
Expand Up @@ -584,6 +584,12 @@ cdef extern from "grpc/grpc_security.h":

void grpc_auth_context_release(grpc_auth_context *context)

grpc_channel_credentials *grpc_local_credentials_create(
grpc_local_connect_type type)
grpc_server_credentials *grpc_local_server_credentials_create(
grpc_local_connect_type type)


cdef extern from "grpc/compression.h":

ctypedef enum grpc_compression_algorithm:
Expand Down Expand Up @@ -624,3 +630,9 @@ cdef extern from "grpc/impl/codegen/compression_types.h":

const char *_GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY \
"GRPC_COMPRESSION_REQUEST_ALGORITHM_MD_KEY"


cdef extern from "grpc/grpc_security_constants.h":
ctypedef enum grpc_local_connect_type:
UDS
LOCAL_TCP
4 changes: 3 additions & 1 deletion src/python/grpcio_tests/commands.py
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,9 @@ class TestGevent(setuptools.Command):
'channelz._channelz_servicer_test.ChannelzServicerTest.test_many_subchannels_and_sockets',
'channelz._channelz_servicer_test.ChannelzServicerTest.test_streaming_rpc',
# TODO(https://github.com/grpc/grpc/issues/15411) enable this test
'unit._cython._channel_test.ChannelTest.test_negative_deadline_connectivity'
'unit._cython._channel_test.ChannelTest.test_negative_deadline_connectivity',
# TODO(https://github.com/grpc/grpc/issues/15411) enable this test
'unit._local_credentials_test.LocalCredentialsTest',
)
BANNED_WINDOWS_TESTS = (
# TODO(https://github.com/grpc/grpc/pull/15411) enable this test
Expand Down
1 change: 1 addition & 0 deletions src/python/grpcio_tests/tests/tests.json
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@
"unit._interceptor_test.InterceptorTest",
"unit._invalid_metadata_test.InvalidMetadataTest",
"unit._invocation_defects_test.InvocationDefectsTest",
"unit._local_credentials_test.LocalCredentialsTest",
"unit._logging_test.LoggingTest",
"unit._metadata_code_details_test.MetadataCodeDetailsTest",
"unit._metadata_flags_test.MetadataFlagsTest",
Expand Down
1 change: 1 addition & 0 deletions src/python/grpcio_tests/tests/unit/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ GRPCIO_TESTS_UNIT = [
"_interceptor_test.py",
"_invalid_metadata_test.py",
"_invocation_defects_test.py",
"_local_credentials_test.py",
"_logging_test.py",
"_metadata_code_details_test.py",
"_metadata_test.py",
Expand Down
3 changes: 3 additions & 0 deletions src/python/grpcio_tests/tests/unit/_api_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,9 @@ def testAll(self):
'ServiceRpcHandler',
'Server',
'ServerInterceptor',
'LocalConnectionType',
'local_channel_credentials',
'local_server_credentials',
'unary_unary_rpc_method_handler',
'unary_stream_rpc_method_handler',
'stream_unary_rpc_method_handler',
Expand Down
70 changes: 70 additions & 0 deletions src/python/grpcio_tests/tests/unit/_local_credentials_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Copyright 2019 The gRPC Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Test of RPCs made using local credentials."""

import unittest
from concurrent.futures import ThreadPoolExecutor
import grpc


class _GenericHandler(grpc.GenericRpcHandler):

def service(self, handler_call_details):
return grpc.unary_unary_rpc_method_handler(
lambda request, unused_context: request)


class LocalCredentialsTest(unittest.TestCase):

def _create_server(self):
server = grpc.server(ThreadPoolExecutor())
server.add_generic_rpc_handlers((_GenericHandler(),))
return server

def test_local_tcp(self):
server_addr = 'localhost:{}'
channel_creds = grpc.local_channel_credentials(
grpc.LocalConnectionType.LOCAL_TCP)
server_creds = grpc.local_server_credentials(
grpc.LocalConnectionType.LOCAL_TCP)

server = self._create_server()
port = server.add_secure_port(server_addr.format(0), server_creds)
server.start()
with grpc.secure_channel(server_addr.format(port),
channel_creds) as channel:
self.assertEqual(b'abc',
channel.unary_unary('/test/method')(
b'abc', wait_for_ready=True))
server.stop(None)

def test_uds(self):
server_addr = 'unix:/tmp/grpc_fullstack_test'
channel_creds = grpc.local_channel_credentials(
grpc.LocalConnectionType.UDS)
server_creds = grpc.local_server_credentials(
grpc.LocalConnectionType.UDS)

server = self._create_server()
server.add_secure_port(server_addr, server_creds)
server.start()
with grpc.secure_channel(server_addr, channel_creds) as channel:
self.assertEqual(b'abc',
channel.unary_unary('/test/method')(
b'abc', wait_for_ready=True))
server.stop(None)


if __name__ == '__main__':
unittest.main()