OSError: [Errno 4] Interrupted system call

[tryerrorman]

When I use pytorch to train a small network in a multi-user Ubuntu 16.04 + cuda 8.0 + python2.7, I came across the OSError. It happens accidentally. Some time within 1 epoch, some time 4 epoch. The log is:

Traceback (most recent call last):
File "autoencodertrain.py", line 43, in
data = dataloader.get_next_iter()
File "/home/pytorch/codes/gan/xgan/data/newdata_loader.py", line 45, in get_next_iter
dataB = self.dataLoaderB.iter().next()
File "/usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.py", line 254, in next
idx, batch = self._get_batch()
File "/usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.py", line 233, in _get_batch
return self.data_queue.get()
File "/usr/lib/python2.7/multiprocessing/queues.py", line 378, in get
return recv()
File "/usr/local/lib/python2.7/dist-packages/torch/multiprocessing/queue.py", line 22, in recv
return pickle.loads(buf)
File "/usr/lib/python2.7/pickle.py", line 1388, in loads
return Unpickler(file).load()
File "/usr/lib/python2.7/pickle.py", line 864, in load
dispatchkey
File "/usr/lib/python2.7/pickle.py", line 1139, in load_reduce
value = func(*args)
File "/usr/local/lib/python2.7/dist-packages/torch/multiprocessing/reductions.py", line 68, in rebuild_storage_fd
fd = multiprocessing.reduction.rebuild_handle(df)
File "/usr/lib/python2.7/multiprocessing/reduction.py", line 170, in rebuild_handle
new_handle = recv_handle(conn)
File "/usr/lib/python2.7/multiprocessing/reduction.py", line 85, in recv_handle
return _multiprocessing.recvfd(conn.fileno())
OSError: [Errno 4] Interrupted system call

Can anyone help solve this problem. Or give some hints on how to solve it.

[tryerrorman]

I have refer to https://stackoverflow.com/questions/14136195/what-is-the-proper-way-to-handle-in-python-ioerror-errno-4-interrupted-syst and change the function recv_handle() in reduction.py as follow

def recv_handle(conn):
    while True:
        try:
            _fl_data = _multiprocessing.recvfd(conn.fileno())
            break
        except OSError, num:
            print num
            if num==4:
                continue
    return _fl_data
`
it work, but I don't know which program trigger this Interrupt.

[colesbury]

This looks like it's specific to Python 2.7. I think Python fixed it internally (https://www.python.org/dev/peps/pep-0475/#status-in-python-3-4) in Python 3.4.

We should use the strategy provided in the linked stackoverflow post (retry on IOError with e.errorno == errno.EINTR)

[ssnl]

closed by bc62810