PyuTp under Snow Leopard

[roverso]

I'm currently trying to test your implementation of uTP using the Python wrapper you provided. In doing so I encountered two problems: the first being that the struct.unpack call would throw this exception:

IPAddr(struct.unpack("L", socket.inet_aton(ip))[0])
struct.error: unpack requires a string argument of length 8

This at "sockaddr_types.py" line 80.

However, I managed to solve that by changing the format of the unpack (fmt) to network(big endian) "!L".

After that, I did not get that message anymore BUT python crashes due to a segmentation fault in the underlying utp implementation. I'm particularly expert in
programming python applications which interface with C/C++ but I guess a wrong pointer is passed to the C uTP library. The library crashes when calling:

utp.UTP_SetCallbacks(self.utp_socket,
ctypes.byref(f), ctypes.py_object(self))

at line 92 in the Socket class in file utp_socket.py. I attach to this e-mail the stack trace of the error.

I'm currently using the pre-installed Python 2.6 under Mac OS Snow Leopard latest version. I tried also the last Python 2.7 from the official releases and I got
the same kind of behavior. Then I tried to install it in a Linux machine and all went fine. However, I do really need to get this working on all platforms, windows
included.

Thank you very much. I really appreciate your work.

[ghazel]

I don't think changing the struct format to big endian is the correct thing to do there. socket.inet_aton already returns network byte order, so adding the "!" to the struct unpack essentially flips it back.

I find it very strange that struct.unpack("L", '\x7f\x00\x00\x01') would throw "struct.error: unpack requires a string argument of length 8", since the format size for "L" is clearly defined as 4: http://docs.python.org/library/struct.html#format-characters
What does struct.calcsize("L") return?

As for the crash, UTP_SetCallbacks is a very simple function, so my only guess is that the utp_socket is not set or created properly. Can you provide a minimal example which reproduces the crash?

[roverso]

You can reproduce the first issue in Mac using this:
import socket
import struct
ip = '127.0.0.1'
size = struct.calcsize("L")
print "struct calcsize returns:"+str(size)
print "socket.inet_aton returns:"+str(len(socket.inet_aton(ip)))

The return size of the struct and of the socket.inet_aton will be 8 and 4 bytes respectively. No Match. I found out in a forum (http://code.activestate.com/recipes/66517-ip-address-conversion-functions-with-the-builtin-s/) that the interpretation via the struct-module is not correct on most platforms and consequently you have to force the format manually using "!L" instead of only "L".

Regarding the crash, I tried to debug a minimal python program that uses pyutp together with gdb for stepping through C++ libutp code. I found out that the program crashes when trying to access the UTP_Socket in the UTP_SetCallbacks function. It looks as if python provides a wrong pointer to the libutp C++ code in the latter.

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000002182160
0x00000001015e45a5 in UTP_SetCallbacks ()

I choose a stripped down version of WriteDataTestCase to reproduce the issue but any of the default tests where an actual connection is triggered should cause the error.

[ghazel]

That forum post is referring to endianness. I believe the endianness is correct in libutp, it's the Python struct module size which is wrong. Looking closer at the Python struct documentation I linked to, I think this is a "native" vs. "standard" size problem. Try the format string "=L".

[roverso]

I tried the format string "=L" but this does not solve the crash I mentioned earlier. Can you reproduce the issue on your end?

[ghazel]

Does "=L" solve the struct.error?

I do not have access to OS X at the moment since I have a PC. I'll see if I can borrow a Mac later this week.

[roverso]

As I mentioned in the previous post, the "=L" has not fixed the bug. Have you been able to reproduce the issue on a Mac machine?