during a https-request the READ TIMEOUT not workunder win32 [rt.cpan.org #3316] #36
Description
Migrated from rt.cpan.org#3316 (status was 'open')
Requestors:
From on 2003-08-22 12:59:14:
my $ua = new LWP::UserAgent;
# set read timeout
$ua->timeout(3);
...
my $result = $ua->request(...);
on an http-request the timeout works
on an https-request the timeout NOT works
From on 2004-06-12 05:27:49:
[guest - Fri Aug 22 08:59:14 2003]:
> my $ua = new LWP::UserAgent;
>
> # set read timeout
> $ua->timeout(3);
> ...
> my $result = $ua->request(...);
>
> on an http-request the timeout works
> on an https-request the timeout NOT works
I've noticed the same problem, under Fedora 1 (up2date to June 2004),
with the latest (5.79) libwww installed.
I'm not 100% certain, but I think this problem is not limited to https,
but instead to persistent connections and servers that send incorrect
Content-Length headers. This could do with some more investigation though.
I've tracked the problem down to LWP::Protocol::http::SocketMethods, in
the sysread() method. The code in question is this:
if (my $timeout = ${*$self}{io_socket_timeout}) {
die "read timeout" unless $self->can_read($timeout);
}
else {
...
I can't find any mention of {io_socket_timeout} anywhere in the LWP
module. As a result, I don't think timeouts are ever actually used
correctly. I kludged a solution by prefixing the above code with:
${*$self}{io_socket_timeout}=180;
...and found that https downloads are now considerably more reliable,
and (I think) have stopped hanging. I never could find a specific site
that repeatably caused a problem, so it's hard for me to provide a
testcase though.
I hope that helps - it took me a while to track down, so hopefully I've
saved someone else the trouble ;-)
...Ralph Bolton
http://www.coofercat.com/
From on 2004-06-13 12:16:29:
[guest - Sat Jun 12 01:27:49 2004]:
> ${*$self}{io_socket_timeout}=180;
>
> ...and found that https downloads are now considerably more reliable,
> and (I think) have stopped hanging. I never could find a specific site
> that repeatably caused a problem, so it's hard for me to provide a
> testcase though.
Sorry, I was jumping the gun somewhat... This doesn't fix the problem at
all.
I'm finding my process sitting in a blocking read() of a socket
connected to a remote server on port 443 (via strace and lsof). This
leads me to think that the socket is not non-blocking. The call to
nonblock() is in an eval{} block, so I wonder if it's somehow failing...?
From on 2006-01-06 13:41:22:
Running under mod perl 2 undef red hat and debian making connection to
remote server over ssl with timeout of 10:
The apache process exits and puts this in the error log:
[Fri Jan 06 11:59:52 2006] [notice] child pid 5140 exit signal Alarm
clock (14)
I assume this is the same bug.
Tom
From fany@cpan.org on 2006-03-08 19:09:49:
I'm experiencing the same problems using libwww-perl-5.805 with Perl
5.8.4 under Debian/Sarge and Perl 5.8.8 under SuSE Linux 10.0 when
trying to connect to a server that does not respond to HTTPS requests, e.g.:
| $ time perl -MLWP::UserAgent -e
'(my$ua=LWP::UserAgent->new)->timeout(3);print$ua->simple_request(HTTP::Request->new(GET=>"https://62.128.1.1/"))->as_string'
500 Connect failed: connect: Die Wartezeit für die Verbindung ist
abgelaufen; Die Wartezeit für die Verbindung ist abgelaufen
| Content-Type: text/plain
| Client-Date: Wed, 08 Mar 2006 19:00:52 GMT
| Client-Warning: Internal response
|
| 500 Connect failed: connect: Connection timed out; Connection timed out
|
| real 3m9.150s
| user 0m0.124s
| sys 0m0.024s
So the timeout occurs not until a few minutes later, when the OS' TCP
stack gives up, but not because I've set a timeout.
However, it works just fine for HTTP:
| $ time perl -MLWP::UserAgent -e
'(my$ua=LWP::UserAgent->new)->timeout(3);print$ua->simple_request(HTTP::Request->new(GET=>"http://62.128.1.1/"))->as_string'
| 500 Can't connect to 62.128.1.1:80 (connect: timeout)
| Content-Type: text/plain
| Client-Date: Wed, 08 Mar 2006 18:58:05 GMT
| Client-Warning: Internal response
|
| 500 Can't connect to 62.128.1.1:80 (connect: timeout)
|
| real 0m3.128s
| user 0m0.092s
| sys 0m0.020s
fany.
From fany@cpan.org on 2006-03-08 19:54:56:
The difference between HTTP and HTTPS can also be observed using strace:
For HTTP, the socket is set to O_NONBLOCK, and then select() is
used to catch the timeout:
$ strace -t perl -MLWP::UserAgent -e
'(my$ua=LWP::UserAgent->new)->timeout(3);$ua->simple_request(HTTP::Request->new(GET=>"http://62.128.1.1/"))'
[...]
20:46:34 socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
20:46:34 ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbff0b4e8) = -1 EINVAL
(Invalid argument)
20:46:34 _llseek(3, 0, 0xbff0b530, SEEK_CUR) = -1 ESPIPE (Illegal seek)
20:46:34 ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbff0b4e8) = -1 EINVAL
(Invalid argument)
20:46:34 _llseek(3, 0, 0xbff0b530, SEEK_CUR) = -1 ESPIPE (Illegal seek)
20:46:34 fcntl64(3, F_SETFD, FD_CLOEXEC) = 0
20:46:34 fcntl64(3, F_GETFL) = 0x2 (flags O_RDWR)
20:46:34 fcntl64(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
20:46:34 connect(3, {sa_family=AF_INET, sin_port=htons(80),
sin_addr=inet_addr("62.128.1.1")}, 16) = -1 EINPROGRESS (Operation now
in progress)
20:46:34
stat64("/usr/local/lib/perl5/5.8.8/i686-linux-64int-ld/IO/Select.pmc",
0xbff0b56c) = -1 ENOENT (No such file or directory)
20:46:34
stat64("/usr/local/lib/perl5/5.8.8/i686-linux-64int-ld/IO/Select.pm",
{st_mode=S_IFREG|0444, st_size=8021, ...}) = 0
20:46:34
open("/usr/local/lib/perl5/5.8.8/i686-linux-64int-ld/IO/Select.pm",
O_RDONLY|O_LARGEFILE) = 4
20:46:34 ioctl(4, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbff0b2a8) = -1 ENOTTY
(Inappropriate ioctl for device)
20:46:34 _llseek(4, 0, [0], SEEK_CUR) = 0
20:46:34 read(4, "# IO::Select.pm\n#\n# Copyright (c"..., 4096) = 4096
20:46:34 _llseek(4, 4029, [4029], SEEK_SET) = 0
20:46:34 _llseek(4, 0, [4029], SEEK_CUR) = 0
20:46:34 close(4) = 0
20:46:34 select(8, NULL, [3], NULL, {3, 0}) = 0 (Timeout)
For HTTPS, however, O_NONBLOCK does not get set,
which causes the connect() to hang:
$ strace -t perl -MLWP::UserAgent -e
'(my$ua=LWP::UserAgent->new)->timeout(3);$ua->simple_request(HTTP::Request->new(GET=>"https://62.128.1.1/"))'
[...]
20:47:45 socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
20:47:45 ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfae2c28) = -1 EINVAL
(Invalid argument)
20:47:45 _llseek(3, 0, 0xbfae2c70, SEEK_CUR) = -1 ESPIPE (Illegal seek)
20:47:45 ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfae2c28) = -1 EINVAL
(Invalid argument)
20:47:45 _llseek(3, 0, 0xbfae2c70, SEEK_CUR) = -1 ESPIPE (Illegal seek)
20:47:45 fcntl64(3, F_SETFD, FD_CLOEXEC) = 0
20:47:45 connect(3, {sa_family=AF_INET, sin_port=htons(443),
sin_addr=inet_addr("62.128.1.1")}, 16) = -1 ETIMEDOUT (Connection timed out)
20:50:54 brk(0x844e000) = 0x844e000
fany.
From fany@cpan.org on 2006-04-04 16:25:09:
I've narrowed this down further using the debugger:
The cause of this problem is the following method within Net::HTTPS
(version 1.00):
60 # The underlying SSLeay classes fails to work if the socket is
61 # placed in non-blocking mode. This override of the blocking
62 # method makes sure it stays the way it was created.
63 sub blocking { } # noop
Indeed, I get strange results when trying to circumvent this:
| fany@ray:~> perl -MNet::HTTPS -MLWP::UserAgent -le '{ package
Net::HTTPS; sub blocking { shift->SUPER::blocking(@_) } }
(my$ua=LWP::UserAgent->new)->timeout(3);print
$ua->simple_request(HTTP::Request->new(GET=>"https://www.noris.net/"))->as_string'
| 500 read failed:
| Content-Type: text/plain
| Client-Date: Tue, 04 Apr 2006 16:15:47 GMT
| Client-Warning: Internal response
|
| 500 read failed:
|
I think I'll give up now.
Here's my current workaround BTW:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
package my::LWP::UserAgent;
use base 'LWP::UserAgent';
sub simple_request {
my $self = shift;
my ($request) = @_;
return $self->SUPER::simple_request(@_)
unless $request->uri =~ /^https:/ &&
( my $timeout = $self->timeout );
my $response;
eval {
local $SIG{ALRM} = sub { die "TIMEOUT\n" };
alarm $timeout;
$response = $self->SUPER::simple_request(@_);
alarm 0;
};
$response;
}
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Regards,
fany
From on 2006-06-15 07:52:24:
<a href='http://www.yahoo.com'></a>Thanks! http://www.ringtones-dir.com/download/ <a href='http://www.ringtones-dir.com'>download ringtones</a>. <a href="http://www.ringtones-dir.com ">nokia ringtones</a>: ringtones site, Free nokia ringtones here, Download ringtones FREE. Also [url]http://www.ringtones-dir.com/free/[/url] and [link=http://www.ringtones-dir.com]ring tones[/link] From site .
From on 2006-06-15 07:52:50: