Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Strange interaction between RPyC and pyqtgraph #517

Open
sidneycadot opened this issue Nov 28, 2022 · 3 comments
Open

Strange interaction between RPyC and pyqtgraph #517

sidneycadot opened this issue Nov 28, 2022 · 3 comments
Assignees

Comments

@sidneycadot
Copy link

sidneycadot commented Nov 28, 2022

Using Python 3.11 and RPyC 5.3.0, I see a very puzzling interaction when talking to an RPyc server from a client that uses a pyqtgraph PlotWindow widget. (Pyqtgraph is a well-known framework for making plots in python programs that implement a Qt-based GUI).

I am testing under Linux, with PyQt5 but the issue is also present when using PySide6.

The server code is as plain and simple as can be:

#! /usr/bin/env python3

import logging
import rpyc

# Configure the logging infrastructure.
logging.basicConfig(level=logging.DEBUG)

class MyService(rpyc.Service):
    def __init__(self):
        super().__init__()
        self.exposed_value = 42

server = rpyc.utils.server.ThreadedServer(MyService(), port=30000)
server.start()

The client side is somewhat more involved. I open a connection to the server and interact with it. Next, I run a simple GUI application that stops when the user closes the window. Finally, I close the connection:

#! /usr/bin/env python3

import sys
import logging
import time

import rpyc

from PyQt5.QtWidgets import QApplication, QWidget
import pyqtgraph as pg

# Configure the logging infrastructure.
logging.basicConfig(level=logging.DEBUG)

# Connect to the server's MyService instance.
connection = rpyc.connect("localhost", 30000)

# Obtain a value.
# If we don't do this, the server-side issue disappears.

value = connection.root.value
print(value)

# Make a trivial Qt application and run it.
app = QApplication(sys.argv)
main_window = pg.PlotWidget()  # If we use this, the server will emit an error upon closing the window.
#main_window = QWidget()       # If we use this instead, the server will be happy.
main_window.show()
exitcode = app.exec()

# As soon as we return from the application, the server-side generates a strange debug message.

# Wait a bit (to establish when the server-side error is emitted), then close the connection.
time.sleep(10)
connection.close()

The issue is that as soon as I close the GUI application, the server-side emits a debugging message that strongly suggests something has gone wrong. This is before the connection is closed:

sidney@zeeman:~/rpc_pyqtgraph_issue$ ./server.py 
INFO:MY/30000:server started on [0.0.0.0]:30000
INFO:MY/30000:accepted ('127.0.0.1', 43422) with fd 4
INFO:MY/30000:welcome ('127.0.0.1', 43422)
DEBUG:MY/30000:Exception caught
Traceback (most recent call last):
  File "/home/sidney/local_python/root/lib/python3.11/site-packages/rpyc/core/protocol.py", line 356, in _dispatch_request
    res = self._HANDLERS[handler](self, *args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sidney/local_python/root/lib/python3.11/site-packages/rpyc/core/protocol.py", line 853, in _handle_getattr
    return self._access_attr(obj, name, (), "_rpyc_getattr", "allow_getattr", getattr)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sidney/local_python/root/lib/python3.11/site-packages/rpyc/core/protocol.py", line 780, in _access_attr
    name = self._check_attr(obj, name, param)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sidney/local_python/root/lib/python3.11/site-packages/rpyc/core/protocol.py", line 770, in _check_attr
    raise AttributeError(f"cannot access {name!r}")
AttributeError: cannot access '__class__'
DEBUG:MY/30000:Exception caught
Traceback (most recent call last):
  File "/home/sidney/local_python/root/lib/python3.11/site-packages/rpyc/core/protocol.py", line 356, in _dispatch_request
    res = self._HANDLERS[handler](self, *args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sidney/local_python/root/lib/python3.11/site-packages/rpyc/core/protocol.py", line 853, in _handle_getattr
    return self._access_attr(obj, name, (), "_rpyc_getattr", "allow_getattr", getattr)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sidney/local_python/root/lib/python3.11/site-packages/rpyc/core/protocol.py", line 780, in _access_attr
    name = self._check_attr(obj, name, param)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sidney/local_python/root/lib/python3.11/site-packages/rpyc/core/protocol.py", line 770, in _check_attr
    raise AttributeError(f"cannot access {name!r}")
AttributeError: cannot access '__class__'
INFO:MY/30000:goodbye ('127.0.0.1', 43422)
^C
WARNING:MY/30000:keyboard interrupt!
INFO:MY/30000:server has terminated
INFO:MY/30000:listener closed

There's a couple of ways I can make this debugging noise disappear.

(1) If I don't interact from the client with the server other than opening/closing it, the issue disappears.

(2) If I use a plain QWidget rather than a pyqtgraph PlotWidget, the issue disappears.

My conclusion is that there is some kind of a funky interaction between pyqtgraph and rpyc; but I am not well-enough versed in both their codebases to investigate any further -- so I am hoping someone here can reproduce the issue, and see what's going on.

@ghost
Copy link

ghost commented Jan 5, 2023

I'm working with @sidneycadot on an RPyc application and did some further debugging on this issue. I found that the issue is triggered because pyqtgraph calls isinstance() on an RPyc netref.

During application cleanup, pyqtgraph runs a piece of cleanup code (link), which slightly simplified looks like:

    for o in gc.get_objects():
        if isinstance(o, ...):
            ...

This cleanup code hits all live Python objects, including any RPyc netrefs. When isinstance() is called on an instance of rpyc.core.netref.__main__.MyService, the warning appears in the RPyc server.

The following simplified client program reproduces the issue without involving pyqtgraph:

import sys
import logging
import time

import rpyc

# Configure the logging infrastructure.
logging.basicConfig(level=logging.DEBUG)

# Connect to the server's MyService instance.
connection = rpyc.connect("localhost", 30000)
print("Connected.")

time.sleep(1)
print("Now triggering the issue ...")

# Trigger the issue by running "isinstance()" on the remote object.
print(isinstance(connection.root, int))

print("Triggered the issue.")
time.sleep(2)

connection.close()

To summarize:

  1. RPyc netref objects by default do not support calls to isinstance().
  2. During application cleanup, pyqtgraph calls isinstance() on all live Python objects.

I'm not sufficiently experienced with RPyc to know what the correct solution is here.

One may take the opinion that pyqtgraph should not be calling isinstance() on objects beyond its own control. However, presumably pyqtgraph requires this cleanup mechanism. Asking them to avoid a useful mechanism because it interacts badly with an intrusive RPC framework seems unreasonable.

One may also take the opinion that RPyc objects not supporting isinstance() is inherently unpythonic.

The warning can be avoided by explicitly exposing the __class__ attribute in the server script:

class MyService(rpyc.Service):
    def __init__(self):
        super().__init__()
        self.exposed_value = 42
        self.exposed___class__ = self.__class__

Again, I don't know enough about RPyc to understand whether this is the intended way to enable access to the __class__ attribute. Any advice would be appreciated.

@sidneycadot
Copy link
Author

sidneycadot commented Jan 5, 2023

@jorisvr

It's a bit of a difficult question how one would /want/ isinstance() to behave. At the client side, do we want to do the check on the proxy itself, or on the remote object? Both could be useful I think, depending on circumstances.

A technical alternative could be to override the __instancecheck__ class method.

@comrumino comrumino self-assigned this Jan 14, 2023
@sidneycadot
Copy link
Author

sidneycadot commented Feb 9, 2023

Okay @jorisvr figured out the root cause, which is that 'isinstance' on netrefs doesn't work as expected under some circumstances.

In my particular (PyQt) application it's a bit hard to work around that, because both PyQtGraph and my application use an "aboutToQuit" handler, the calling-order at program end of which is undefined.

We (@jorisvr) and myself found two workarounds for the original problem:

On the pyrpc side, setting the protocol config attribute "allow_all_attrs" to True makes the isinstance() work, suppressing the issue.

On the PyQtGraph side, the garbage collection sweep that triggers the isinstance() calls can be disabled by setting the config option 'exitCleanup' to False, for example by executing:

pyqtgraph.setConfigOptions(exitCleanup=False)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants