idle: Errno 10035 a non-blocking socket operation could not be completed immediately

[mj]

BPO	26739
Nosy	@terryjreedy, @pfmoore, @kristjanvalur, @tjguk, @zware, @zooba
Files	Skier: Game

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2016-05-31.21:42:30.707>
created_at = <Date 2016-04-11.21:24:19.859>
labels = ['expert-IDLE', 'type-bug', 'OS-windows']
title = 'idle: Errno 10035 a non-blocking socket operation could not be completed immediately'
updated_at = <Date 2016-06-01.09:19:53.665>
user = 'https://bugs.python.org/MJ'

bugs.python.org fields:

activity = <Date 2016-06-01.09:19:53.665>
actor = 'kristjan.jonsson'
assignee = 'none'
closed = True
closed_date = <Date 2016-05-31.21:42:30.707>
closer = 'zach.ware'
components = ['IDLE', 'Windows']
creation = <Date 2016-04-11.21:24:19.859>
creator = 'MJ'
dependencies = []
files = ['42436']
hgrepos = []
issue_num = 26739
keywords = ['3.4regression']
message_count = 15.0
messages = ['263208', '263521', '263662', '263676', '263689', '263697', '264104', '264105', '264117', '264173', '266786', '266787', '266788', '266790', '266811']
nosy_count = 7.0
nosy_names = ['terry.reedy', 'paul.moore', 'kristjan.jonsson', 'tim.golden', 'zach.ware', 'steve.dower', 'MJ']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue26739'
versions = ['Python 2.7']

[mj]

So far I've got past the "bug in program" stage of debugging, but this came up:
IDLE internal error in runcode()
Traceback (most recent call last):
  File "C:\Python27\lib\idlelib\rpc.py", line 235, in asyncqueue
    self.putmessage((seq, request))
  File "C:\Python27\lib\idlelib\rpc.py", line 332, in putmessage
    n = self.sock.send(s[:BUFSIZE])
error: [Errno 10035] A non-blocking socket operation could not be completed immediately
I have no idea what a "socket" is so if you know please tell me!

[terryjreedy]

Michael, when you open an issue, you need to stay nosy so we can ask questions. This is especially true when the issue involved 3rd party modules, like pygame, and even more, when personal files, like the images, are required. In this case:

0. Which patch release of 2.7 are you running? If not 2.7.11, please upgrade and retry. There was a patch for Error 10035 in March 19, 2013, which would be about the time of 2.7.6 or so.
   https://hg.python.org/cpython/rev/8ec39bfd1f01 bpo-9090

1.Does your program run correctly when you run it directly with python from the console, with "python pathto/Skier"?

2. Were you running it from the IDLE editor? If so, does it fail immediately? or after the game runs a bit? Does it seem to always fail in exactly the same way, at the same time?

[A 'socket' is a class abstraction used to communicate between different processes. It is usually used for processes on different machines, as with the browser process on your machine and a server process on another machine, but can be used for two processes on the same machine.]

--
This appears to be a socket issue, not an IDLE issue as such, just as bpo-25332 was not a urllib issue. This is the 10th issue found searching the tracker for 'Error 10035', but the first, I think, involving IDLE. I know essentially nothing about the subject.

[mj]

0. I have Python 2.7.5
1. I don't know what you mean.
2. It fails right when I run it.
   IDLE stands for Python's Integrated DeveLopment Environment
   I am running IDLE / Python on a Windows Vista, Service pack 2

[terryjreedy]

0. OK, 2.7.5 came out in 2013 May, after the 2.7 backport. Kristján, you did the backport. Do you have any idea about this?

1. Open a Command Prompt window, the command line console. For Win 7, it can be found find it on the Start menu under Admin or something or started on the run line with 'cmd.exe'. You should learn how to get the console.

On the command line, after the ...> prompt, enter "python patth/to/Skier.py".

Or, maybe, find Skier.py in Windows Explorer, and either double click or right-click and select "Run". (I am not sure if this works with 2.7.5 on Vista.)

[mj]

Is the command prompt itself admin?
I don't have access to the admin command prompt.

[terryjreedy]

No, admin privileges should not be needed to run Command Prompt. Ditto for python if you did not already need same to run IDLE.

[kristjanvalur]

Hi there.
I don't think this is in relation to issue bpo-9090.
That one had to do with the internal mechanisms of doing blocking IO with timeout. this is done internally by using non-blocking sockets and select(), and the backport dealt with some edge cases on windows where select() could falsely indicate that data were ready.

From what I can see in this error description, we are dealing with real non-blocking IO, i.e. an application is using select and non-blocking sockets.

It is possible that this windows edge case is now being elevated into the application code and whatever select() logic being used in rpc.py needs to be aware of it, or that for some reason this socket is supposed to be blocking, but isn't.

I'll have a quick look at idlelib and see if I can see anything.

[kristjanvalur]

Caveat emptor: I know nothing of IDLE, and I even suspect it to be dead or dying code. Non the less, it could be patched.

I found this in the code:
def putmessage(self, message):
self.debug("putmessage:%d:" % message[0])
try:
s = pickle.dumps(message)
except pickle.PicklingError:
print >>sys.__stderr__, "Cannot pickle:", repr(message)
raise
s = struct.pack("<i", len(s)) + s
while len(s) > 0:
try:
r, w, x = select.select([], [self.sock], [])
n = self.sock.send(s[:BUFSIZE])
except (AttributeError, TypeError):
raise IOError, "socket no longer exists"
except socket.error:
raise
else:
s = s[n:]

If the socket were non-blocking, this would be the place to add a handler to catch socket.error with errno=errno.EWOULDBLOCK

However, I can't see that this socket is non-blocking. Perhaps I have some blindness, but the select calls seem to be redundant to me, I can't see any sock.setblocking(False) or sock.settimeout(0.0) being done anywhere.

Having said that, the following change can be made (which is the prudent way to use select/send anyway)
while len(s) > 0:
try:
while True:
r, w, x = select.select([], [self.sock], [])
try:
n = self.sock.send(s[:BUFSIZE])
break
except socket.error as e:
import errno # should be done at the top
if e.errno != errno.EWOULDBLOCK:
raise
except (AttributeError, TypeError):
raise IOError, "socket no longer exists"
except socket.error:
raise
else:
s = s[n:]

[terryjreedy]

Kristján, thank you for the response. The socket communication part of IDLE is pretty much a black box to me. Just to clarify, you are saying that select polling is only needed for non-blocking sockets; sockets are blocking by default; and you see no override of the default. Me neither. It would be nice to know. Idlelib.PyShell has the following lines:
<445> self.rpcclt.listening_sock.settimeout(10)
<561> response = clt.pollresponse(self.active_seq, wait=0.05)
The wait is ultimately passed to rpc.py, line 353:
r, w, x = select.select([self.sock.fileno()], [], [], wait)

(IDLE is very much in use, and I hope to modernize the 3.x version, and perhaps replace the sockets with non-blocking use of pipes.)

Michael: I cannot change 2.7.5 and I am not inclined to change 2.7.12+ until I know there is a problem with the current 2.7 release. Numerous people are using IDLE with 2.7.11 (and 3.x) on Windows without problems.

[kristjanvalur]

I think that the select.select calls there are a red herring, since I see no evidence that the rpc socket is ever put in non-blocking mode.
But the line
self.rpcclt.listening_sock.settimeout(10)
indicates that the socket is in timeout mode, and so, the error could be expected if it weren't for the backported fix for issue bpo-9090

I'll have another look at that code and see if thera are any loopholes.

Also, Micahel could try commenting out this line in
C:\Python27\Lib\idlelib\PyShell.py:
self.rpcclt.listening_sock.settimeout(10)

and see if the problem goes away.

[mj]

Kristján, when you say 'dead or dying code', it worries me a lot. Can you tell me what you mean because I do NOT want to reinstall IDLE.

[mj]

I also do not want to change something in the /lib folder because that is like going into the Windows Registry and 'fixing' a bunch of stuff.

I've made other games with PyGame and they generate errno 10035, too.
I've also installed the same version of Python, on a newer computer.
It is a Windows 10, ex-8.1
I haven't finished the program on my newer computer.

[zware]

If you're still using 2.7.5, we can't help you. If you can reproduce the issue with 2.7.11 or 3.5.1, please reopen.

[terryjreedy]

Michael, IDLE is not dead, and I am working steadily to modernize it inside and out for 3.6. There have even been improvements for 2.7 since 2.7.5. For his work, Kristján has had no reason to keep up to date on IDLE's progress.

[kristjanvalur]

Hi there, everyone.
I'm sorry for my rash remarks about the state of IDLE, I'm sure it is alive and well and its good to see that fine people like Terry are working on keeping it up to date.

Michael, please understand that python developers are volunteers and sometimes need help to fix things. In this case, we have not been able to reproduce the problem, and are not sure what can be causing it. My suggestion for you to modify code would be a step in identifying and diagnosing the problem. Without such feedback it is hard to accomplish anything.

Cheers!