multiple errors in test_socket on OpenBSD #57114
Comments
|
Hi, I have few problems in test_socket.py.
Traceback (most recent call last):
File "Lib/test/test_socket.py", line 753, in test_sendall_interrupted_with_timeout
self.check_sendall_interrupted(True)
File "Lib/test/test_socket.py", line 742, in check_sendall_interrupted
self.assertRaises(socket.timeout, c.sendall, b"x" * (1024**2))
File "/usr/ports/pobj/Python-3.2.2/Python-3.2.2/Lib/unittest/case.py", line 557, in assertRaises
callableObj(*args, **kwargs)
socket.error: [Errno 0] Error======================================================================
Traceback (most recent call last):
File "Lib/test/test_socket.py", line 1707, in testInterruptedTimeout
foo = self.serv.accept()
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "Lib/test/test_socket.py", line 1707, in testInterruptedTimeout
foo = self.serv.accept()
Alarm
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "Lib/test/test_socket.py", line 1721, in testInterruptedTimeout
self.fail("got Alarm in wrong place")
AssertionError: got Alarm in wrong place====================================================================== Thanks a lot, Remi. |
|
I add bt output with gdb 7, to have more informations. |
|
I hope that this issue is not related to threads+signals. We got many threads+signals issues on FreeBSD 6. |
|
Issue bpo-12903 is similar to this one. |
Yep. @rémi """ If it works, you could try creating a dummy thread before setting up the signal handler, something like: And retry. |
Hi, it blocks too: $ gdb -args ./python
[...]
(gdb) run
Starting program: /usr/ports/pobj/Python-3.2.2/Python-3.2.2/python
Python 3.2.2 (default, Sep 5 2011, 21:21:34)
[GCC 4.2.1 20070719 ] on openbsd5
Type "help", "copyright", "credits" or "license" for more information.
>>> import signal, socket
>>> c, s = socket.socketpair()
>>> signal.signal(signal.SIGALRM, lambda x,y: 0)
0
>>> signal.alarm(1)
0
>>> [New process 28830]
c.sendall(b"x" * (1024**2))
^C
Program received signal SIGINT, Interrupt.
Cannot access memory at address 0x98
(gdb) bt
#0 0x000000020125678a in poll () from /usr/lib/libc.so.60.1
#1 0x000000020619b4aa in _thread_kern_poll (wait_reqd=<value optimized out>)
at /usr/src/lib/libpthread/uthread/uthread_kern.c:780
#2 0x000000020619c3a8 in _thread_kern_sched (scp=0x0)
at /usr/src/lib/libpthread/uthread/uthread_kern.c:382
#3 0x0000000206190ade in sendto (fd=9, msg=0x2032d6020, len=1044480, flags=0, to=0x0, to_len=0)
at /usr/src/lib/libpthread/uthread/uthread_sendto.c:63
#4 0x00000002105f670a in sock_sendall ()
from /usr/ports/pobj/Python-3.2.2/Python-3.2.2/build/lib.openbsd-5.0-amd64-3.2/_socket.so
#5 0x000000020668b172 in PyEval_EvalFrameEx () from /usr/local/lib/libpython3.2m.so.1.0
#6 0x000000020668bf66 in PyEval_EvalCodeEx () from /usr/local/lib/libpython3.2m.so.1.0
#7 0x000000020668c22b in PyEval_EvalCode () from /usr/local/lib/libpython3.2m.so.1.0
#8 0x00000002066a93d7 in run_mod () from /usr/local/lib/libpython3.2m.so.1.0
#9 0x00000002066abcae in PyRun_InteractiveOneFlags () from /usr/local/lib/libpython3.2m.so.1.0
#10 0x00000002066abf3e in PyRun_InteractiveLoopFlags () from /usr/local/lib/libpython3.2m.so.1.0
#11 0x00000002066ac04c in PyRun_AnyFileExFlags () from /usr/local/lib/libpython3.2m.so.1.0
#12 0x00000002066bc9f4 in Py_Main () from /usr/local/lib/libpython3.2m.so.1.0
#13 0x0000000000400e75 in main ()Thanks for your help, Remi. |
Oops, I just realized there was a typo in the sample test. and not lambda x,y: 0 |
|
I tried the following script on OpenBSD 5 with Python 3.3: s = signal.SIGALRM
signal.signal(s, lambda x,y: 1/0)
signal.alarm(1)
signal.siginterrupt(s, True)
sys.stdin.read()The C signal handler is called, but the system call (read in this case) is not interrupted. |
Bad news: the script doesn't hang if Python is build without threads. Short C program to test interrupted syscalls:
-------------
#include <signal.h>
#include <stdio.h>
#include <pthread.h>
void
handler(int signum)
{
printf("HANDLER!\n");
}
void _noop()
{
}
int main()
{
int s = SIGALRM;
char buffer[1024];
int n;
static int dummy = 0;pthread_t thread1; signal(s, handler);
siginterrupt(s, 1);
alarm(1);
printf("read...\n");
n = read(0, buffer, 1024);
printf("read->%i\n", n);
return 0;
}read() is interrupted after 1 second, it works. |
|
Oh, siginterrupt(SIGALRM, 0) doesn't work in a program linked to pthread. Example: #include <signal.h>
#include <stdio.h>
void
handler(int signum)
{ printf("HANDLER!\n"); }
int main()
{
int s = SIGALRM;
char buffer[1024];
int n;
signal(s, handler);
siginterrupt(s, 0);
alarm(1);
printf("read...\n");
n = read(0, buffer, 1024);
printf("read->%i\n", n);
return 0;
}This program ends after 1 second with "read->-1" if it is linked to pthread (bug!), it hangs if it is not linked to pthread (ok). |
That's what I thought...
Makes sense. When linked with pthread, all I/O syscalls are actually non-blocking.
Hmmm... Also, one difference is that Python uses sigaction to setup the signal handler. There might be subtle semantics change/bugs between signal/sigaction.
You could try with sigaction/SA_RESTART. But OpenBSD's pthread implementation has severe limitations/bugs. |
Yes, the read() is also interrupted as expected if no thread is created.
If the handler is installed using the following code, read() is interrupted: Using sa.sa_flags=SA_RESTART, read() hangs (it is not interrupted). Python uses sigaction with flags=0.
Using SA_RESTART, read() is not interrupted. But if the program is linked to pthread, read() is always interrupted: with sa_flags=0 or sa_flags=SA_RESTART.
rthread doc contains: "Future work: Quite simply, signal handling is one the most complicated aspects of threads to get right. (...)" |
Ouch...
This paper dates back to 2005, I was hoping they would have solved As for the original problem, IIUC you don't reproduce it with your C |
|
It looks like Python cannot do much to workaround OpenBSD issues. IMO the best fix is just to skip these tests on OpenBSD, until OpenBSD handles correctly signals in programs linked to pthread. The same "fix" can be used for bpo-12903. |
Agreed. |
|
Rémi, do you want to submit a patch to skip those tests on OpenBSD? |
|
I think we could close this bug because it's du to the pthread library on OpenBSD and not Python. Thanks for your help and sorry for the delay. |
Indeed. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: