Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Unicode-only strings in IronPython make rpyc.utils.classic.deliver() fail if the server is Python 2 #251
When connecting an IronPython (2.7) client to a server running Python 2,
On the server:
&> python -V && python rpyc_classic.py Python 2.7.13 :: Continuum Analytics, Inc. INFO:SLAVE/18812:server started on [127.0.0.1]:18812 INFO:SLAVE/18812:accepted ('127.0.0.1', 52319) with fd 868 INFO:SLAVE/18812:welcome ('127.0.0.1', 52319) DEBUG:SLAVE/18812:Exception caught Traceback (most recent call last): File "c:\tools\Anaconda3\envs\rhinoremote\lib\site-packages\rpyc\core\protocol.py", line 347, in _dispatch_request res = self._HANDLERS[handler](self, *args) File "c:\tools\Anaconda3\envs\rhinoremote\lib\site-packages\rpyc\core\protocol.py", line 624, in _handle_call return self._local_objects[oid](*args, **dict(kwargs)) TypeError: loads() argument 1 must be string, not unicode
On the client:
IronPython 2.7.5 (126.96.36.199) on .NET 4.0.30319.42000 (64-bit) Type "help", "copyright", "credits" or "license" for more information. >>> import rpyc >>> rpyc.__version__ (3, 4, 4) >>> conn = rpyc.classic.connect('localhost') >>> rpyc.utils.classic.deliver(conn, 'a') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Program Files (x86)\IronPython 2.7\Lib\rpyc\utils\classic.py", line 289, in deliver File "C:\Program Files (x86)\IronPython 2.7\Lib\rpyc\core\netref.py", line 199, in __call__ File "C:\Program Files (x86)\IronPython 2.7\Lib\rpyc\core\netref.py", line 72, in syncreq File "C:\Program Files (x86)\IronPython 2.7\Lib\rpyc\core\protocol.py", line 523, in sync_request TypeError: loads() argument 1 must be string, not unicode ========= Remote Traceback (1) ========= ...
I have identified the root of the problem to be in the
One way to solve this is to force plain ascii strings to be sent as regular strings by brine by changing
import sys if sys.platform == 'cli': @register(_dump_registry, unicode) def _dump_unicode(obj, stream): try: _dump_str(obj.decode('ascii'), stream) except UnicodeDecodeError: stream.append(TAG_UNICODE) _dump_str(obj.encode("utf8"), stream) else: @register(_dump_registry, unicode) def _dump_unicode(obj, stream): stream.append(TAG_UNICODE) _dump_str(obj.encode("utf8"), stream)
But this might impact performance, and it might be better to send a flag to the server for when the string should not be considered unicode. However, the question is then to define what should be unicode or not... Another way would be to force the server to convert to
I know that IronPython 2 is an almost-dead project that you are not officially supporting, so I'm fine if you do not want to include this in the main branch; I just thought I'd let you know. This might have also been what was causing part of #10.
Hi, thanks for reporting.
For clarification, so in Ironpython we have
if sys.platform == 'cli':
IronPython shows as
def _dump_unicode(obj, stream): try: _dump_str(obj.decode('ascii'), stream) except UnicodeDecodeError: stream.append(TAG_UNICODE) _dump_str(obj.encode("utf8"), stream)
I think you mean
The problem with this is that it opens the door for a lot more unicode/str related problems as were common on py2 whenever mixing unicode with bytes objects (i.e. a certain variable in a certain function can be either unicode or bytes) and that manifest as exceptions only once the program is used with true unicode data but may go unnoticed a long time during testing.
To be honest, given that IronPython handles
(Otherwise, I would be willing to merge a working solution, if it exists and is simple enough)
added a commit
Jan 16, 2018
Hi, and sorry for not replying sooner.
I agree with all your comments here, and the solution in b02e7e7. I had actually implemented something very similar before I saw it, starting from your comment that
There is just one thing to add to solve the original problem:
deliver(conn, localobj): return conn.modules["rpyc.lib.compat"].pickle.loads(bytes(pickle.dumps(localobj)))
works for my case (IronPython to Python 2). It should also be fine in regular Python since