Runtime error when using send_remote

[dcam0050]

I have set up a scenario with 2 python processes that run as separate (remote) instances to test communication between two python Pyrlang processes registered as sender and receiver

From controller, I am executing

self.node.send_remote(sender=self.pid_, dst_node="receiver@127.0.0.1", receiver="receiver", message=Atom("try"))

Whenever I execute this command, I get the following traceback on sender:

Registering process - sender@127.0.0.1
Task exception was never retrieved
future: <Task finished coro=<_generic_async_loop() done, defined at /home/icub/user_files/project/venv/Pyrlang/pyrlang/async/asyncio_engine.py:169> exception=RuntimeError('This event loop is already running',)>
Traceback (most recent call last):
  File "/home/icub/user_files/project/venv/Pyrlang/pyrlang/async/asyncio_engine.py", line 170, in _generic_async_loop
    while loop_fn():
  File "/home/icub/user_files/project/project/project/BaseGlue.py", line 61, in process_loop
    self.module_loop()
  File "/home/icub/user_files/project/project/project/Modules/Controller.py", line 75, in module_loop
    self.__node__._send_remote(sender=self.pid_, dst_node="receiver@127.0.0.1", receiver="receiver", message=Atom("try"))
  File "/home/icub/user_files/project/venv/Pyrlang/pyrlang/node.py", line 314, in _send_remote
    message=m)
  File "/home/icub/user_files/project/venv/Pyrlang/pyrlang/node.py", line 350, in _dist_command
    self.engine_.sleep(0.1)
  File "/home/icub/user_files/project/venv/Pyrlang/pyrlang/async/asyncio_engine.py", line 64, in sleep
    self.loop_.run_until_complete(asyncio.sleep(seconds))
  File "/usr/lib/python3.6/asyncio/base_events.py", line 455, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.6/asyncio/base_events.py", line 409, in run_forever
    raise RuntimeError('This event loop is already running')
RuntimeError: This event loop is already running
Socket closed

and the following error in receiver:

Error: Dist protocol version have: (5, 5) got: (5, 5) (state recvname)
Protocol requested to disconnect the socket

Any feedback would be greatly appreciated!

[kvakvs]

Not sure how to interpret the error, but I think you should be using send not send_remote and the destination should be a tuple with (Atom('othernode@127.0.0.1'), Atom('receiver')). The process with name receiver must exist and be registered on the destination node. Otherwise this looks like a bug: send to remote should be marked as private and destination as a string could also be handled for convenience.

node.send(sender=self.pid_,
          receiver=(Atom('erl@127.0.0.1'), Atom('shell')),
          message=Atom('hello'))

P.S. Do you accidentally have 2 Node's running on 1 python? This is not tested at all and could even be the problem which you observe.

[dcam0050]

Hey kvakvs,

I tried node.send initially and I get the same error. I believe the error is with self.engine_.sleep(0.1) called within node.py. I tried commenting it but it just loops infinitely. My guess is that it needs to call the engine to run for a while before checking exit condition.

The issue maybe arises because of event_engine.loop_forever() is performing the loop and message handling functions of Process and then self.loop_.run_until_complete(asyncio.sleep(seconds)) in asyncio_engine.py seems to raise an error because loop is already running.

[kvakvs]

can you post a minimal setup code to reproduce this? and i can have a look today.

[kvakvs]

can you post a minimal setup code to reproduce this? and i can have a look today.

[dcam0050]

Will do in an hour

[dcam0050]

Here is the minimal example.

2 files: a receiver that is a registered process that prints all messages it receives and a sender which sends a message to the receiver every once in a while

minimal_python_comms.zip

[kvakvs]

Ok i just had a look.

You've caught me here with creating Process with passive=False. What this means, that Process.process_loop will be called from event loop and no other event loop functions such as sleep, also send and connect can not be called here. It should be possible but it isn't. So this needs some work, which can take a few days (or weeks).

Why this is not done yet? Main effort was on driving Pyrlang from Erlang, so your infinite loop would be running in Python, and incoming messages over the network would wake your processes up and they would do some work. Now a scenario with Python being active and counting time and doing something, this is where I am not prepared. Essentially there should be a model for running your process loops, like you have now, but it is not async def, it is very limited and you cannot even connect another node from there.

I have no information whether there are gevent users of this library and how many, might make it work nicely with asyncio though. There is some fear and uncertainty and doubt here.

[dcam0050]

Hey @kvakvs great explanation. For now I believe I can make do with an erlang node that acts as a router in between pyrlang nodes since I can send and receive via erlang just fine.

Thanks for the quick reply as always and keep up the good work!