[WIP] implement metadata for asynchronous unaryunary callable #10

ZHmao · 2019-09-03T10:56:49Z

Implement metadata for Aio UnaryUnary Call .

Fix grpc#20144

TODO

add tests.

Implement the minimal stuff for making a unary call with the new experimental gRPC Python implementation for Asyncio, called Aio. What has been added: - Minimal iomgr code for performing the required network and timer calls. - Minimal Cython code implementing the channel, call and the callback context. - Minimal Python code that mimics the synchronous implementation but designed to be asynchronous. Not solved yet: - Tests have to be executed using the `GRPC_ENABLE_FORK_SUPPORT=0` environment variable for skipping the fork handles installed by the core library. Co-authored-by: Manuel Miranda <manuel.miranda@skyscanner.net> Co-authored-by: Mariano Anaya <mariano.anaya@skyscanner.net> Co-authored-by: Zhanghui Mao <zhanghui.mao@skyscanner.net>

…orted

…lement-metadata-for-call

argaen · 2019-09-24T13:26:10Z

src/python/grpcio_tests/tests_aio/unit/metadata_test.py

+)
+
+
+class TestMetadata(test_base.AioTestBase):


I would put the test in the same file where we have the other tests, in the end we are testing the client with metadata

+1 just as a new test_unary_unary_metadata

argaen · 2019-09-24T13:27:32Z

src/python/grpcio_tests/tests_aio/unit/test_common.py

+DETAILS = 'test details'
+
+
+def metadata_transmitted(original_metadata, transmitted_metadata):


I don't understand this test, what is the purpose of it? It is not importing anything from gRPC

It's a function used by TestMetadata , this function help it check if the metadata of response is equal to original metadata.

argaen · 2019-09-24T13:29:27Z

src/python/grpcio/grpc/experimental/aio/_channel.py

+
+        return state.response
+
+    async def with_state(self,


what is this method used for? Why did you add it here?

I should use the name of sync version (with_call) .
When we call this function instead of __call__ , we can return more detail information (like metadata), because __call__ only return response.

I copied it from sync version. This is with_call in sync version: https://github.com/grpc/grpc/blob/master/src/python/grpcio/grpc/_channel.py#L606

And sync version use _end_unary_response_blocking to check which parameters should be returned:
https://github.com/grpc/grpc/blob/master/src/python/grpcio/grpc/_channel.py#L615
https://github.com/grpc/grpc/blob/master/src/python/grpcio/grpc/_channel.py#L498-L502

All of the code that is behind with_call will be in some how refactorized for accommodating this proposal grpc#20001.

It's true that what you have implemented can be reused later for implementing that proposal, so IMO I would keep this method - please rename it - but I would put a disclaimer like.

# TODO(https://github.com/grpc/grpc/issues/20001) This method will be removed and its logic will be used for achieving the new unified version of __call__, future and with_call ``

OK, I see 👍

argaen · 2019-09-24T13:32:15Z

src/python/grpcio/grpc/experimental/aio/_channel.py

+        self.trailing_metadata = trailing_metadata
+
+
+def _handle_call_result(operations, state, response_deserializer):


where did you get this logic from? Is it from the sync version? Can you link it? :) Gives the impression we are implementing more things than needed (for now)?

I got it from here: https://github.com/grpc/grpc/blob/master/src/python/grpcio/grpc/_channel.py#L119

I think we may use this function construct more information in the future.

argaen · 2019-09-24T13:33:52Z

src/python/grpcio_tests/tests_aio/unit/metadata_test.py

+              '/grpc.testing.TestService/UnaryCall',
+              request_serializer=messages_pb2.SimpleRequest.SerializeToString,
+              response_deserializer=messages_pb2.SimpleResponse.FromString)
+            state = await hi.with_state(messages_pb2.SimpleRequest(),


why? Why not call hi directly as we do in the other tests?

If we want to get more information(like metadata) from __call__, I need to change the return value of it. So I add a new method to return more details.

pfreixes · 2019-09-27T09:01:00Z

src/python/grpcio_tests/tests_aio/unit/metadata_test.py

+)
+
+
+class TestMetadata(test_base.AioTestBase):


+1 just as a new test_unary_unary_metadata

pfreixes · 2019-09-27T09:31:52Z

src/python/grpcio/grpc/experimental/aio/_channel.py

+
+        return state.response
+
+    async def with_state(self,


All of the code that is behind with_call will be in some how refactorized for accommodating this proposal grpc#20001.

It's true that what you have implemented can be reused later for implementing that proposal, so IMO I would keep this method - please rename it - but I would put a disclaimer like.

# TODO(https://github.com/grpc/grpc/issues/20001) This method will be removed and its logic will be used for achieving the new unified version of __call__, future and with_call ``

pfreixes and others added 7 commits August 29, 2019 17:38

Use get_event_loop rather than use get_running_loop

35f1587

Move into other files declarations and utilities for iomgr

72aa479

Choose the localhost name for testing depending on the IP stacks supp…

26ce172

…orted

Syntax and linter

34242b2

Update BAZEL builf file

50f0b27

implement metadata for asynchronous unaryunary callable

e7b69a6

pfreixes force-pushed the unary_unary_client_aio branch 2 times, most recently from 3f6df55 to 799ced6 Compare September 5, 2019 14:08

pfreixes force-pushed the unary_unary_client_aio branch 2 times, most recently from 615b846 to a44e6d7 Compare September 11, 2019 18:39

ZHmao added 3 commits September 14, 2019 15:14

Merge branch 'unary_unary_client_aio' into unary_unary_client_aio-imp…

945344f

…lement-metadata-for-call

add test for metadata

8b0e474

fix _GrpcState when call it

7b9b376

argaen reviewed Sep 24, 2019

View reviewed changes

pfreixes suggested changes Sep 27, 2019

View reviewed changes

ZHmao closed this Nov 30, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] implement metadata for asynchronous unaryunary callable #10

[WIP] implement metadata for asynchronous unaryunary callable #10

ZHmao commented Sep 3, 2019

argaen Sep 24, 2019

pfreixes Sep 27, 2019

rmariano Oct 9, 2019

argaen Sep 24, 2019

ZHmao Sep 25, 2019

argaen Sep 24, 2019

ZHmao Sep 25, 2019

pfreixes Sep 27, 2019

ZHmao Sep 29, 2019

argaen Sep 24, 2019

ZHmao Sep 25, 2019

argaen Sep 24, 2019

ZHmao Sep 25, 2019

pfreixes Sep 27, 2019

pfreixes Sep 27, 2019

		DETAILS = 'test details'


		def metadata_transmitted(original_metadata, transmitted_metadata):

		self.trailing_metadata = trailing_metadata


		def _handle_call_result(operations, state, response_deserializer):

		)


		class TestMetadata(test_base.AioTestBase):

		)


		class TestMetadata(test_base.AioTestBase):

[WIP] implement metadata for asynchronous unaryunary callable #10

[WIP] implement metadata for asynchronous unaryunary callable #10

Conversation

ZHmao commented Sep 3, 2019

TODO

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment