EOFError for no obvious reason?

[jgn]

Mechanize 1.0.0

Feature request to selectively turn off Content-Length checking? (???)

log of debug output below

require 'rubygems'
require 'mechanize'
require 'logger'

a = Mechanize.new
a.user_agent = "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB7.0"
a.log = Logger.new(STDOUT)
p = a.get(:url => "http://www.retailmenot.com/view/abesofmaine.com")

If I comment-out the Content-Length check in Mechanize::Chain::ResponseReader, I do get the full page (and the length is the reported Content-Length).

Maybe there should be an option to suppress the Content-Length check? Not sure if it should be at the Mechanize-level, or at the level of a particular request.

In any case, I'm sure not getting why it would be EOF'ing anyways.

Log ------

I, [2010-02-16T14:34:21.230419 #4270] INFO -- : Net::HTTP::Get: /view/abesofmaine.com
D, [2010-02-16T14:34:21.230557 #4270] DEBUG -- : request-header: accept-language => en-us,en;q=0.5
D, [2010-02-16T14:34:21.230602 #4270] DEBUG -- : request-header: connection => keep-alive
D, [2010-02-16T14:34:21.230643 #4270] DEBUG -- : request-header: accept => /
D, [2010-02-16T14:34:21.230684 #4270] DEBUG -- : request-header: accept-encoding => gzip,identity
D, [2010-02-16T14:34:21.230724 #4270] DEBUG -- : request-header: user-agent => Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 GTB7.0
D, [2010-02-16T14:34:21.230766 #4270] DEBUG -- : request-header: accept-charset => ISO-8859-1,utf-8;q=0.7,*;q=0.7
D, [2010-02-16T14:34:21.231365 #4270] DEBUG -- : request-header: host => www.retailmenot.com
D, [2010-02-16T14:34:21.231416 #4270] DEBUG -- : request-header: keep-alive => 300
D, [2010-02-16T14:34:21.353167 #4270] DEBUG -- : Read 550 bytes
D, [2010-02-16T14:34:21.353423 #4270] DEBUG -- : Read 914 bytes
D, [2010-02-16T14:34:21.353894 #4270] DEBUG -- : Read 1938 bytes
D, [2010-02-16T14:34:21.354088 #4270] DEBUG -- : Read 2302 bytes
D, [2010-02-16T14:34:21.354302 #4270] DEBUG -- : Read 3326 bytes
D, [2010-02-16T14:34:21.354510 #4270] DEBUG -- : Read 3690 bytes
D, [2010-02-16T14:34:21.460328 #4270] DEBUG -- : Read 4714 bytes
D, [2010-02-16T14:34:21.460593 #4270] DEBUG -- : Read 5078 bytes
D, [2010-02-16T14:34:21.461439 #4270] DEBUG -- : Read 6102 bytes
D, [2010-02-16T14:34:21.461647 #4270] DEBUG -- : Read 6466 bytes
D, [2010-02-16T14:34:21.462294 #4270] DEBUG -- : Read 7490 bytes
D, [2010-02-16T14:34:21.462503 #4270] DEBUG -- : Read 7854 bytes
D, [2010-02-16T14:34:21.500857 #4270] DEBUG -- : Read 8878 bytes
D, [2010-02-16T14:34:21.501224 #4270] DEBUG -- : Read 9437 bytes
E, [2010-02-16T14:34:21.501605 #4270] ERROR -- : Rescuing EOF error
D, [2010-02-16T14:34:21.729018 #4270] DEBUG -- : Read 550 bytes
D, [2010-02-16T14:34:21.729343 #4270] DEBUG -- : Read 1574 bytes
D, [2010-02-16T14:34:21.729561 #4270] DEBUG -- : Read 2598 bytes
D, [2010-02-16T14:34:21.729748 #4270] DEBUG -- : Read 3622 bytes
D, [2010-02-16T14:34:21.729930 #4270] DEBUG -- : Read 3690 bytes
D, [2010-02-16T14:34:21.831662 #4270] DEBUG -- : Read 4714 bytes
D, [2010-02-16T14:34:21.831962 #4270] DEBUG -- : Read 5738 bytes
D, [2010-02-16T14:34:21.832202 #4270] DEBUG -- : Read 6466 bytes
D, [2010-02-16T14:34:21.832404 #4270] DEBUG -- : Read 7490 bytes
D, [2010-02-16T14:34:21.832586 #4270] DEBUG -- : Read 7854 bytes
D, [2010-02-16T14:34:21.899798 #4270] DEBUG -- : Read 8878 bytes
D, [2010-02-16T14:34:21.900050 #4270] DEBUG -- : Read 9437 bytes
E, [2010-02-16T14:34:21.900409 #4270] ERROR -- : Rescuing EOF error
D, [2010-02-16T14:34:22.139545 #4270] DEBUG -- : Read 550 bytes
D, [2010-02-16T14:34:22.139962 #4270] DEBUG -- : Read 1574 bytes
D, [2010-02-16T14:34:22.140176 #4270] DEBUG -- : Read 2598 bytes
D, [2010-02-16T14:34:22.140357 #4270] DEBUG -- : Read 3622 bytes
D, [2010-02-16T14:34:22.140538 #4270] DEBUG -- : Read 3690 bytes
D, [2010-02-16T14:34:22.245101 #4270] DEBUG -- : Read 4714 bytes
D, [2010-02-16T14:34:22.245364 #4270] DEBUG -- : Read 5078 bytes
D, [2010-02-16T14:34:22.245758 #4270] DEBUG -- : Read 6102 bytes
D, [2010-02-16T14:34:22.245971 #4270] DEBUG -- : Read 6466 bytes
D, [2010-02-16T14:34:22.246691 #4270] DEBUG -- : Read 7490 bytes
D, [2010-02-16T14:34:22.246890 #4270] DEBUG -- : Read 7854 bytes
D, [2010-02-16T14:34:22.300396 #4270] DEBUG -- : Read 8878 bytes
D, [2010-02-16T14:34:22.300651 #4270] DEBUG -- : Read 9437 bytes
E, [2010-02-16T14:34:22.301006 #4270] ERROR -- : Rescuing EOF error

[runa]

That seems to be related to http://mechanize.rubyforge.org/mechanize/FAQ_rdoc.html

I got something which I think it's the same with this URL http://img5.mediafire.com/436ebd57af2a05a02e5a9b08079953108b2907fa0833f05a2b1c575915ee961f6g.jpg

but since I'm using the latest mechanize, there's no way (AFAIK) to disable keep alives!

Any hints?

[drbrain]

I cannot reproduce this error in trunk.

@runa: your error is due to a garbage header. See curl -v output.