Blocking sockets take entirely too long to timeout

[khiltd]

BPO	2132
Nosy	@loewis

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2008-03-19.22:40:33.864>
created_at = <Date 2008-02-17.18:56:53.901>
labels = ['type-bug', 'library']
title = 'Blocking sockets take entirely too long to timeout'
updated_at = <Date 2008-03-19.22:40:33.862>
user = 'https://bugs.python.org/khiltd'

bugs.python.org fields:

activity = <Date 2008-03-19.22:40:33.862>
actor = 'jafo'
assignee = 'none'
closed = True
closed_date = <Date 2008-03-19.22:40:33.864>
closer = 'jafo'
components = ['Library (Lib)']
creation = <Date 2008-02-17.18:56:53.901>
creator = 'khiltd'
dependencies = []
files = []
hgrepos = []
issue_num = 2132
keywords = []
message_count = 3.0
messages = ['62498', '62503', '64116']
nosy_count = 3.0
nosy_names = ['loewis', 'jafo', 'khiltd']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue2132'
versions = ['Python 2.5']

[khiltd]

The following code:

import smtplib

test = smtplib.SMTP('mail.host.com')

will hang the entire script for about ten minutes when run on a machine
which is connected to the internet via an ISP who blocks port 25 (which is
pretty much all of them these days). Closer inspection of the smtplib
sources reveals that it is making use of blocking sockets, which is all
well and good, but I do not believe they should be allowed to block for
such a ridiculously lengthy period of time.

My task is to walk the list of MX records provided by a DNS query, connect
to each server listed and check a user-supplied address for validity. If I
have to wait 10 minutes for an exception to be thrown for each
unresponsive server, this process could easily take hours per address,
which is completely unacceptable.

My workaround is to ditch smtplib entirely and write my own socket code,
but setting a timeout on a socket throws it into non-blocking mode, which
is not entirely what I want, either. PHP allows one to apply a perfectly
sane timeout to a blocking socket without any difficulty, and this is
something I'm frankly rather surprised to see Python choke on.

What I'd expect to see is an exception after less than a minute's worth of
repeated failures so the caller can catch it and try another port.

[loewis]

I recommend that you stay with non-blocking sockets, and use select/poll
on all sockets. Then you can simultaneously check multiple servers, and
select will tell you which ones you got connected to. For this
application, putting a time-out on the socket and doing the connections
sequentially seems unreasonable - that's exactly what select was
invented for.

I don't understand the "setting a timeout ... is not entirely what I
want, either" remark. The only way to specify a timeout for connect *is*
to set it into non-blocking mode. I'm sure PHP does the same.

If you want to see timeouts for smtplib, a work-around is to set a
global timeout for all sockets, through socket.setdefaulttimeout. This
will transparently apply to smtplib as well.

[jafo]

smtplib is for sending messages via SMTP, not for testing to see if a
user is behind an ISP that is incorrectly blocking outgoing SMTP
connections. I would argue the "incorrectly" because they are dropping
rather than rejecting the connection packets.

One mechanism would be to use an alarm(30) call which would limit the
whole transaction to 30 seconds (connection, EHLO, RCPT, QUIT), not just
the connection. My experience with dealing with remote machines in a
time-sensitive manner is that this approach is much more usable than
using socket timeouts.

A better mechanism is probably to use non-blocking socket I/O directly
and make connections to all or many of the remote servers and test them
in parallel, meaning all servers could be tested in 30 seconds (or
whatever your timeout is) rather than 5 minutes (you imply in your
message that you may be doing more than a dozen requests).

So, I believe the current functionality in the smtplib of being simple
and conservative at trying to get messages through, rather than trying
to be optimized for checking for account existance, is reasonable. And
that in any case, using alarm() is a better solution than socket timeouts.