Skip to content
This repository has been archived by the owner on Jun 4, 2023. It is now read-only.

maximum recursion depth exceeded #149

Closed
zxpatric opened this issue Feb 10, 2017 · 19 comments
Closed

maximum recursion depth exceeded #149

zxpatric opened this issue Feb 10, 2017 · 19 comments

Comments

@zxpatric
Copy link

what to achieve:

Expect to use pyro to share 2M data acquired in Jython (client) to Python. Tried file but it was too slow. Open to suggestion of other ways.

Client using pyrolite-4.15.jar:

self._pyroP.call("saveLeft", bytearray('123'))

Server Pyro 4.53 :

@Pyro4.expose
def saveLeft(self, data):
    dataLen = len(data)
    self._leftData = bytearray(dataLen)
    self._leftData[0:dataLen] = data

Error Message:

Exception in thread Thread-1:Traceback (most recent call last):
File , line 222, in _Thread__bootstrap
self.run()
File , line 149, in run
self._pyroP.call("saveLeft", bytearray('123'))
RuntimeError: maximum recursion depth exceeded (Java StackOverflowError)
@irmen
Copy link
Owner

irmen commented Feb 10, 2017 •
edited
Loading

Can you paste the full java stacktrace. Also: what happens when you upgrade to the latest Pyrolite release.

@zxpatric
Copy link
Author

I tried the latest build pyrolite-4.18.jar and upgraded Pyro4 to Pyro4-4.54. Still same thing.

Jython doesn't give the full java stacktrace or I did find howto. I wrote a pure java application but still struggle to find out java equivalent of jython call "self._pyroP.call("saveLeft", bytearray('123'))"

I tried byte[] data but rejected by serpent. I tried string but it doesn't reproduce the recursion error. Please suggest.

        String _uri = "PYRO:obj_221d338e088844cb934ce386c8056465@localhost:57892";
        System.out.println(_uri.getClass().getName());
        PyroProxy _pyroP = new PyroProxy(new PyroURI(_uri));
        System.out.println(_pyroP.call("greeting"));

// byte[] CDRIVES = hexStringToByteArray("e04fd020ea3a6910a2d808002b30309d");
// byte[] data = "Any String you want".getBytes();
// System.out.println(data.getClass().getName());
String data = "Any String you want";
_pyroP.call("saveLeft", data);

        System.out.println(_pyroP.call ("getLeft"));

        _pyroP.close();

@irmen
Copy link
Owner

irmen commented Feb 10, 2017

Here's some Java example code that uses the native Java API to Pyrolite
https://github.com/irmen/Pyrolite/blob/master/java/src/test/java/net/razorvine/examples/EchoExample.java

A bytearray is obviously just new byte[] {0x12, 0x34, 0x45} in Java....? Does that crash Pyrolite if you pass that as an argument to call?

@irmen
Copy link
Owner

irmen commented Feb 11, 2017

I can't reproduce your problem.

Here are some tests I did with Jython. (First I started the built-in Pyro test echoserver, python -m Pyro4.test.echoserver)

$ jython
Jython 2.7.1b3 (default:df42d5d6be04, Feb 3 2016, 03:22:46)
[OpenJDK 64-Bit Server VM (Azul Systems, Inc.)] on java1.8.0_102
Type "help", "copyright", "credits" or "license" for more information.
>>> import serpent
>>> serpent.dumps(bytearray('123'))
"# serpent utf-8 python2.6\n{'data':'MTIz','encoding':'base64'}"
>>> import Pyro4
>>> p=Pyro4.Proxy("PYRONAME:test.echoserver")
>>> p.echo("test")
u'test'
>>> p.echo(bytearray('1234'))
{u'data': u'MTIzNA==', u'encoding': u'base64'}
>>> _["data"].decode("base-64")
'1234'

It is by design that serpent encodes binary data in a base-64 encoded string structure. That is easily converted back to the actual bytes as you can see.

Now for the native Pyrolite Java API, there's no problem what I can see with giving it bytes:

package net.razorvine.examples;

import java.io.IOException;
import java.util.Map;

import javax.xml.bind.DatatypeConverter;

import net.razorvine.pyro.NameServerProxy;
import net.razorvine.pyro.PyroProxy;
import net.razorvine.pyro.PyroURI;

public class EchoBytes
{
	public static void main(String[] args) throws IOException
	{
		// First start Pyro's test echoserver in a terminal:  python -m Pyro4.test.echoserver -N
		
		// get proxy via name server:
//		NameServerProxy ns = NameServerProxy.locateNS(null);
//		PyroProxy p = new PyroProxy(ns.lookup("test.echoserver"));
//		ns.close();
		// or get it directly:
		PyroProxy p = new PyroProxy(new PyroURI("PYRO:test.echoserver@localhost:50205"));
		
		byte[] data = new byte[]{0x12, 0x34, 0x45};
		
		Object result = p.call("echo", data);
		System.out.println("return value:");
		System.out.println("[class: "+result.getClass()+"]");
		System.out.println(result);
		
		// bytes/bytearry is serialized in Serpent as a structure with encoding (base64) and the data (ascii-encoded bytes)
		@SuppressWarnings("unchecked")
		Map<String, String> h = (Map<String,String>)result;
		if(h.get("encoding").equals("base64")) {
			byte[] resultdata = DatatypeConverter.parseBase64Binary(h.get("data"));
			System.out.println("result data:");
			for(byte b: resultdata)
			{
				System.out.println("byte 0x"+Integer.toHexString(b));
			}
			
		} else {
			System.err.println("unexpected encoding: "+h.get("encoding"));
		}
				
		p.close();
	}
}

When running this, it prints:

return value:
[class: class java.util.HashMap]
{encoding=base64, data=EjRF}
result data:
byte 0x12
byte 0x34
byte 0x45

So I'm closing this issue.
If you can't solve your problem you can reopen it but you have to provide a minimal set of working program code that I can run to reproduce the problem.

@irmen irmen closed this as completed Feb 11, 2017
@zxpatric
Copy link
Author

I still can reproduce the error in my project. Have uploaded a zip of PyCharm project at https://drive.google.com/open?id=0B185R70hbZHkZVkybi1sWnhCQzQ

Let me know if it runs for you. Thanks.

@zxpatric
Copy link
Author

zxpatric commented Feb 13, 2017 via email

@irmen
Copy link
Owner

irmen commented Feb 13, 2017

I'm willing to look at it further but only if you trim down the code a lot. I don't have the time to try to get this full 30 mb project to work and debug all the code of it....
Please create a smaller example that shows the issue, and tell us exactly what to run to reproduce the problem?

@irmen irmen reopened this Feb 13, 2017
@zxpatric
Copy link
Author

zxpatric commented Feb 13, 2017 via email

@zxpatric
Copy link
Author

zxpatric commented Feb 13, 2017 via email

@irmen
Copy link
Owner

irmen commented Feb 13, 2017

It is not going to run as provided on my system.
Please get rid of unneeded code and dependencies (it crashes on numpy and pyqt) that are of no consequence to reproducing the issue, and I'm willing to take another look.

@zxpatric
Copy link
Author

zxpatric commented Feb 13, 2017 via email

@irmen
Copy link
Owner

irmen commented Feb 13, 2017

Thanks, I'll have a look.

In the meantime, I noticed that jython has its own share of classes representing various Python types. See http://javadox.com/org.python/jython-standalone/2.7-rc1/org/python/core/PyObject.html and its derived classes
I suspect 2 things:

  • the problem is specific to Serpent, not Pyro or Pyrolite. If I have proof this issue should reallly be moved to the serpent project ( https://github.com/irmen/Serpent )
  • the problem is because Serpent doesn't know about these Jython classes, and when calling the serializer from Jython code, it somehow gets stuck in a loop.

@zxpatric
Copy link
Author

zxpatric commented Feb 13, 2017 via email

@irmen
Copy link
Owner

irmen commented Feb 13, 2017

question: is there any need to use Jython at all? Can't you use Python itself for client as well?

@zxpatric
Copy link
Author

zxpatric commented Feb 13, 2017 via email

@irmen
Copy link
Owner

irmen commented Feb 14, 2017

I see, but have you tried to use plain Java plus the native Pyrolite java api, instead of jython? That is a lot faster. See my comment above on some tips to use it, have you tried that? #149 (comment)

@irmen irmen mentioned this issue Feb 14, 2017
Closed
@irmen
Copy link
Owner

irmen commented Feb 14, 2017

Please try a new Serpent jar build. (no need to upgrade pyrolite). Hopefully it fixes the issue with the bytearray. Please let me know

@zxpatric
Copy link
Author

zxpatric commented Feb 14, 2017 via email

@zxpatric
Copy link
Author

Yes. The "maximum recursion depth exceeded" has been fixed after upgrading the serpent package. I have also moved my codes away from using Jython.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@irmen @zxpatric