Subscription mode not implemented in Python

[7AC]

I wrote a client in Python that uses the following rpc:

rpc Subscribe(stream SubscribeRequest) returns (stream SubscribeResponse);

The SubscribeRequest has some options related to choosing between "streaming", "once" and "poll" and the comments mention that "streaming" should be the default. I don't think that's what's happening though since there are a few suspicious things happening.

This log message is displayed when the subscription starts:

D0630 22:29:37.999044000 140735169146880 ev_posix.c:101] Using polling engine: poll

Attempting to manipulate any "mode" options (e.g., setting once=False or poll_interval) results in this error:

Traceback (most recent call last):
  File "./openconfig_client.py", line 68, in <module>
    run()
  File "./openconfig_client.py", line 62, in run
    subscribe(stub, args.subscribe)
  File "./openconfig_client.py", line 33, in subscribe
    for response in stub.Subscribe([subscribe_request], _TIMEOUT_SECONDS):
  File "/usr/local/lib/python2.7/site-packages/grpc/beta/_client_adaptations.py", line 150, in next
    return self._next()
  File "/usr/local/lib/python2.7/site-packages/grpc/beta/_client_adaptations.py", line 144, in _next
    raise _abortion_error(rpc_error_call)
grpc.framework.interfaces.face.face.LocalError: LocalError(code=StatusCode.UNIMPLEMENTED, details="Subscription mode not implemented yet")
E0630 22:29:38.104283000 140735169146880 chttp2_transport.c:1810] close_transport: {"created":"@1467350978.104255000","description":"FD shutdown","file":"src/core/lib/iomgr/ev_poll_posix.c","file_line":427}

After a few responses (I suspect one per path, i.e., the definition of "once" which should be False by default) we hit EOF:

Traceback (most recent call last):
  File "./openconfig_client.py", line 69, in <module>
    run()
  File "./openconfig_client.py", line 63, in run
    subscribe(stub, args.subscribe)
  File "./openconfig_client.py", line 33, in subscribe
    for response in stub.Subscribe([subscribe_request], _TIMEOUT_SECONDS):
  File "/usr/local/lib/python2.7/site-packages/grpc/beta/_client_adaptations.py", line 150, in next
    return self._next()
  File "/usr/local/lib/python2.7/site-packages/grpc/beta/_client_adaptations.py", line 144, in _next
    raise _abortion_error(rpc_error_call)
grpc.framework.interfaces.face.face.AbortionError: AbortionError(code=StatusCode.OUT_OF_RANGE, details="EOF")
E0630 22:32:44.359706000 140735169146880 chttp2_transport.c:1810] close_transport: {"created":"@1467351164.359684000","description":"FD shutdown","file":"src/core/lib/iomgr/ev_poll_posix.c","file_line":427}

The Go client using the same .proto goes on forever, which is what made me thing this might be a Python issue. Does this sound plausible at all? If not how could I debug this further?

[soltanmm]

Could you provide a link to example code (both Python and Go if possible)? It'd be helpful to be able to compare the Go and Python clients to debug the situation.

[soltanmm]

Could you provide a link to example code (both Python and Go if possible)? It'd be helpful to be able to compare the Go and Python clients to debug the situation.

[7AC]

Sure! Here it is.

Proto definitions: openconfig.proto. Both stubs have been generated with protobuf 3.0.0-beta-3.

Python:

• generated code
• subscription code (then used here)
• using grpcio 0.15.0 (I was having different issues with 0.14 which look suspiciously similar to this)

Go:

• generated code
• subscription code
• using grpc/grpc-go@e78224b

Any of the following diffs in the Python code results in:

grpc.framework.interfaces.face.face.LocalError: LocalError(code=StatusCode.UNIMPLEMENTED, details="Subscription mode not implemented yet")

• setting once=True (or False, which should be the default anyways)

@@ -24,5 +24,5 @@ def make_subscribe_request(path_str='/'):
     """Create a subscribe request from a string path"""
     path = make_path(path_str)
     subscription = openconfig_pb2.Subscription(path=path)
-    subscription_list = openconfig_pb2.SubscriptionList(subscription=[subscription])
+    subscription_list = openconfig_pb2.SubscriptionList(subscription=[subscription], once=True)
     return openconfig_pb2.SubscribeRequest(subscribe=subscription_list)

• attempting to set poll_interval

@@ -24,5 +24,5 @@ def make_subscribe_request(path_str='/'):
     """Create a subscribe request from a string path"""
     path = make_path(path_str)
     subscription = openconfig_pb2.Subscription(path=path)
-    subscription_list = openconfig_pb2.SubscriptionList(subscription=[subscription])
+    subscription_list = openconfig_pb2.SubscriptionList(subscription=[subscription], poll_interval=100)
     return openconfig_pb2.SubscribeRequest(subscribe=subscription_list)

[7AC]

Sure! Here it is.

Proto definitions: openconfig.proto. Both stubs have been generated with protobuf 3.0.0-beta-3.

Python:

• generated code
• subscription code (then used here)
• using grpcio 0.15.0 (I was having different issues with 0.14 which look suspiciously similar to this)

Go:

• generated code
• subscription code
• using grpc/grpc-go@e78224b

Any of the following diffs in the Python code results in:

grpc.framework.interfaces.face.face.LocalError: LocalError(code=StatusCode.UNIMPLEMENTED, details="Subscription mode not implemented yet")

• setting once=True (or False, which should be the default anyways)

@@ -24,5 +24,5 @@ def make_subscribe_request(path_str='/'):
     """Create a subscribe request from a string path"""
     path = make_path(path_str)
     subscription = openconfig_pb2.Subscription(path=path)
-    subscription_list = openconfig_pb2.SubscriptionList(subscription=[subscription])
+    subscription_list = openconfig_pb2.SubscriptionList(subscription=[subscription], once=True)
     return openconfig_pb2.SubscribeRequest(subscribe=subscription_list)

• attempting to set poll_interval

@@ -24,5 +24,5 @@ def make_subscribe_request(path_str='/'):
     """Create a subscribe request from a string path"""
     path = make_path(path_str)
     subscription = openconfig_pb2.Subscription(path=path)
-    subscription_list = openconfig_pb2.SubscriptionList(subscription=[subscription])
+    subscription_list = openconfig_pb2.SubscriptionList(subscription=[subscription], poll_interval=100)
     return openconfig_pb2.SubscribeRequest(subscribe=subscription_list)

[remingtonc]

Hi all, any update regarding whether or not subscription is implemented in Python?

[remingtonc]

Hi all, any update regarding whether or not subscription is implemented in Python?

[nathanielmanistaatgoogle]

Hmm, it looks like many of the Python links that @7AC shared here are no longer valid.

@remingtonc: I'm still unclear on what subscription mode is - can you share the code you've written that you expect to work but that doesn't because of this issue?

[nathanielmanistaatgoogle]

Hmm, it looks like many of the Python links that @7AC shared here are no longer valid.

@remingtonc: I'm still unclear on what subscription mode is - can you share the code you've written that you expect to work but that doesn't because of this issue?

[remingtonc]

@nathanielmanistaatgoogle My personal comment was a misunderstanding on my part. Subscription mode works correctly given its purpose is to subscribe to channel connectivity event changes and not new data in a stream/channel. My initial thought was that subscribe would "subscribe" to a long-term stream/channel and perform a callback when new data arrived. I am uncertain what the original issue is either now that I understand that an iterable is returned that does not necessarily require a "subscription" callback for new data. @7AC will need to clarify their problem further if it is still an issue.

[remingtonc]

@nathanielmanistaatgoogle My personal comment was a misunderstanding on my part. Subscription mode works correctly given its purpose is to subscribe to channel connectivity event changes and not new data in a stream/channel. My initial thought was that subscribe would "subscribe" to a long-term stream/channel and perform a callback when new data arrived. I am uncertain what the original issue is either now that I understand that an iterable is returned that does not necessarily require a "subscription" callback for new data. @7AC will need to clarify their problem further if it is still an issue.

[7AC]

My bad, here are the corrected links:

https://github.com/aristanetworks/pyopenconfig/blob/master/pyopenconfig/openconfig.proto

https://github.com/aristanetworks/pyopenconfig/blob/master/pyopenconfig/pb2.py

[7AC]

My bad, here are the corrected links:

https://github.com/aristanetworks/pyopenconfig/blob/master/pyopenconfig/openconfig.proto

https://github.com/aristanetworks/pyopenconfig/blob/master/pyopenconfig/pb2.py

[nathanielmanistaatgoogle]

@7AC: Thanks for the updated links. For the circumstance in which you're seeing the undesirable behavior, is the client code written in Python and the server code written in Go? Or some other pairing? Also please ensure that you're using grpcio 1.0.0rc1 (or if you're working from our master source that's fine too).

[nathanielmanistaatgoogle]

@7AC: Thanks for the updated links. For the circumstance in which you're seeing the undesirable behavior, is the client code written in Python and the server code written in Go? Or some other pairing? Also please ensure that you're using grpcio 1.0.0rc1 (or if you're working from our master source that's fine too).

[jfrabaute]

Hi,

I was investigating a similar problem with a java client instead of python client and found a bug in the server-side code. So, it's highly likely that this is a problem with the specific server-side go code.
I'll validate with @7AC and close this issue if that's the case.

More detail about the bug in case someone runs into the same problem:
Reminder: This is a bidirectionnal rpc method call.
The go server-side code closes the connection when the client RPC stream ends (an EOF is received on the server-side).
On the server-side the goroutine sending the stream will be interrupted as well.
But actually, only the stream is closed and the server should NOT close the connection and continue to stream data to the client.

[jfrabaute]

Hi,

I was investigating a similar problem with a java client instead of python client and found a bug in the server-side code. So, it's highly likely that this is a problem with the specific server-side go code.
I'll validate with @7AC and close this issue if that's the case.

More detail about the bug in case someone runs into the same problem:
Reminder: This is a bidirectionnal rpc method call.
The go server-side code closes the connection when the client RPC stream ends (an EOF is received on the server-side).
On the server-side the goroutine sending the stream will be interrupted as well.
But actually, only the stream is closed and the server should NOT close the connection and continue to stream data to the client.

[7AC]

I can confirm that keeping the connection open on the server side fixed the issue, the Go server can now stream to the Python client continuously.

[7AC]

I can confirm that keeping the connection open on the server side fixed the issue, the Go server can now stream to the Python client continuously.