Client doesn't handle HTTP continue #74

activeprospect · 2011-02-16T00:34:55Z

We have a server that is returning the below response. The client reads everything before the HTTP/1.1 500 Internal Server Error line as the response header. That line and everything below is contained in the response body.

  HTTP/1.1 100 Continue
  Date: Wed, 16 Feb 2011 00:26:53 GMT
  Status: 100 Continue
  Connection: keep-alive
  Set-Cookie: ARPT=IKNQPJS192.168.70.113CKOKU; path=/
  Server: Microsoft-IIS/5.0
  X-Powered-By: ASP.NET
  Content-Length: 284
  HTTP/1.1 500 Internal Server Error
  Server: Microsoft-IIS/5.0
  Date: Wed, 16 Feb 2011 00:26:53 GMT
  X-Powered-By: ASP.NET
  Connection: close
  X-AspNet-Version: 2.0.50727
  Cache-Control: private
  Content-Type: text/plain; charset=utf-8
  Content-Length: 26

  Missing parameter: Name.

The text was updated successfully, but these errors were encountered:

igrigorik · 2011-02-16T02:21:19Z

Which version of em-http? Can you try building and running this scenario through the latest in master?

activeprospect · 2011-02-16T02:54:59Z

I was using 0.3.0. When I use the master branch, I get an error:

  /Users/alex/.rvm/gems/ruby-1.9.2-p136/bundler/gems/em-http-request-1841d6904c65/lib/em-http/http_connection.rb:59:in `block in post_init': undefined method `parse_response_header' for nil:NilClass (NoMethodError)
  from /Users/alex/.rvm/gems/ruby-1.9.2-p136/bundler/gems/em-http-request-1841d6904c65/lib/em-http/http_connection.rb:78:in `call'
  from /Users/alex/.rvm/gems/ruby-1.9.2-p136/bundler/gems/em-http-request-1841d6904c65/lib/em-http/http_connection.rb:78:in `<<'
  from /Users/alex/.rvm/gems/ruby-1.9.2-p136/bundler/gems/em-http-request-1841d6904c65/lib/em-http/http_connection.rb:78:in `receive_data'
  from /Users/alex/.rvm/gems/ruby-1.9.2-p136/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in `run_machine'
  from /Users/alex/.rvm/gems/ruby-1.9.2-p136/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in `run'
  from /Users/alex/.rvm/gems/ruby-1.9.2-p136/gems/rainbows-3.0.0/lib/rainbows/event_machine.rb:75:in `worker_loop'
  from /Users/alex/.rvm/gems/ruby-1.9.2-p136/gems/unicorn-3.3.1/lib/unicorn/http_server.rb:496:in `block (2 levels) in spawn_missing_workers'
  from /Users/alex/.rvm/gems/ruby-1.9.2-p136/gems/unicorn-3.3.1/lib/unicorn/http_server.rb:493:in `fork'
  from /Users/alex/.rvm/gems/ruby-1.9.2-p136/gems/unicorn-3.3.1/lib/unicorn/http_server.rb:493:in `block in spawn_missing_workers'
  from /Users/alex/.rvm/gems/ruby-1.9.2-p136/gems/unicorn-3.3.1/lib/unicorn/http_server.rb:489:in `each'
  from /Users/alex/.rvm/gems/ruby-1.9.2-p136/gems/unicorn-3.3.1/lib/unicorn/http_server.rb:489:in `spawn_missing_workers'
  from /Users/alex/.rvm/gems/ruby-1.9.2-p136/gems/unicorn-3.3.1/lib/unicorn/http_server.rb:503:in `maintain_worker_count'
  from /Users/alex/.rvm/gems/ruby-1.9.2-p136/gems/unicorn-3.3.1/lib/unicorn/http_server.rb:161:in `start'
  from /Users/alex/.rvm/gems/ruby-1.9.2-p136/gems/rainbows-3.0.0/lib/rainbows.rb:63:in `run'
  from /Users/alex/.rvm/gems/ruby-1.9.2-p136/gems/rainbows-3.0.0/bin/rainbows:121:in `<top (required)>'
  from /Users/alex/.rvm/gems/ruby-1.9.2-p136/bin/rainbows:19:in `load'
  from /Users/alex/.rvm/gems/ruby-1.9.2-p136/bin/rainbows:19:in `<main>'

activeprospect · 2011-02-16T03:02:55Z

Using master, the headers callback is invoked with the header values that are read prior to the HTTP/1.1 500 Internal Server Error line. The chunk callback is never called.

igrigorik · 2011-02-16T03:09:06Z

Interesting. I'm trying to wrap my head around the actual exchange -- is this even a valid response pattern?

activeprospect · 2011-02-16T03:15:40Z

If I understand http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html#sec8.2.3 correctly, it appears to be valid. And apparently, there's no way to turn this behavior off when you're using HTTP 1.1:

  for compatibility with RFC 2068, a server MAY send a 100 (Continue)
  status in response to an HTTP/1.1 PUT or POST request that does
  not include an Expect request-header field with the "100-
  continue" expectation

Here is the raw full request, for reference. That's not my server, so please be kind to it. :)

  $ curl -v -X POST 'http://ws.serviceobjects.com/lv/LeadValidation.asmx/ValidateLead_V2' -d 'hi=bye'
  * About to connect() to ws.serviceobjects.com port 80 (#0)
  *   Trying 63.131.152.60... connected
  * Connected to ws.serviceobjects.com (63.131.152.60) port 80 (#0)
  > POST /lv/LeadValidation.asmx/ValidateLead_V2 HTTP/1.1
  > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
  > Host: ws.serviceobjects.com
  > Accept: */*
  > Content-Length: 6
  > Content-Type: application/x-www-form-urlencoded
  > 
  < HTTP/1.1 100 Continue
  < Set-Cookie: ARPT=IKNQPJS192.168.70.110CKOKO; path=/
  < Server: Microsoft-IIS/5.0
  < Date: Wed, 16 Feb 2011 03:08:26 GMT
  < X-Powered-By: ASP.NET
  < HTTP/1.1 500 Internal Server Error
  < Server: Microsoft-IIS/5.0
  < Date: Wed, 16 Feb 2011 03:08:26 GMT
  < X-Powered-By: ASP.NET
  < X-AspNet-Version: 2.0.50727
  < Cache-Control: private
  < Content-Type: text/plain; charset=utf-8
  < Content-Length: 26
  < 
  Missing parameter: Name.

igrigorik · 2011-02-16T04:31:07Z

The response within the last example is very different from the first one.. In the first, 100 also returned a keep-alive header, as well as a content-length (which is what's confusing me).

alexkwolfe · 2011-02-16T04:55:45Z

My apologies. I'm using em-http-request to proxy an HTTP call to ws.serviceobjects.com. The first response that I provided is the response from my proxy, not the response from ws.serviceobjects.com. Sorry for the indirection.

igrigorik · 2011-02-17T05:09:03Z

2512106

That passes on this end.. It returns a 100, which is probably not what you're looking for, but it doesn't seem to blow up.

Reading through the IIS forums, this seems to me more like a "bug" in IIS than intended behavior in the spec: http://forums.iis.net/p/1162153/1922653.aspx

The client is definitely not setting an expects header, so as per suggested workaround.. in theory doing a 1.0 signature in the request could solve this. Having said that, what a pita! Still trying to wrap my head around how to handle this.. if at all.

activeprospect · 2011-02-17T18:11:07Z

Ugh. Yeah. That's annoying.

As far as I can tell, the HTTP spec seems to indicate that IIS 5 isn't doing something "wrong" by behaving this way. Still, I'm not sure how pervasive this server behavior is, and therefore if it's worth fixing. As it is now, it seems like the client can't be used with any server that responds with an HTTP 100 because the response headers and body are not parsed.

I think the ideal scenario is to make this test pass:

  it "should handle a 100 continue" do
    EventMachine.run {
      url = 'http://ws.serviceobjects.com/lv/LeadValidation.asmx/ValidateLead_V2'
      http = EventMachine::HttpRequest.new(url).post :body => {:foo => :bar}

      http.errback { failed(http) }
      http.callback {
        http.response_header.status.should == 500
        http.response_header['X_ASPNET_VERSION'].should == '2.0.50727'
        http.response.should == 'Missing parameter: Name.'
        EventMachine.stop
      }
    }
  end

Barring that, maybe I could look at adding support for forcing a request to HTTP/1.0. Though honestly, I'm not sure what other ramifications that would have, if any.

igrigorik · 2011-02-19T16:44:46Z

The one major different between 1.0 and 1.1 is the keepalive sematics.. but something tells me that's not an issue here, since until very recently (still unreleased in master), keepalive was not an option in em-http.

Relevant bits out of the spec:

      - Upon receiving a request which includes an Expect request-header
        field with the "100-continue" expectation, an origin server MUST
        either respond with 100 (Continue) status and continue to read
        from the input stream, or respond with a final status code. The
        origin server MUST NOT wait for the request body before sending
        the 100 (Continue) response. If it responds with a final status
        code, it MAY close the transport connection or it MAY continue
        to read and discard the rest of the request.  It MUST NOT
        perform the requested method if it returns a final status code.

      - An origin server SHOULD NOT send a 100 (Continue) response if
        the request message does not include an Expect request-header
        field with the "100-continue" expectation, and MUST NOT send a
        100 (Continue) response if such a request comes from an HTTP/1.0
        (or earlier) client. There is an exception to this rule: for
        compatibility with RFC 2068, a server MAY send a 100 (Continue)
        status in response to an HTTP/1.1 PUT or POST request that does
        not include an Expect request-header field with the "100-
        continue" expectation. This exception, the purpose of which is
        to minimize any client processing delays associated with an
        undeclared wait for 100 (Continue) status, applies only to
        HTTP/1.1 requests, and not to requests with any other HTTP-
        version value.

So, assuming we're keeping the client at 1.1, you're right.. we're hitting the RFC 2068 exception. Ugh. I wish whoever wrote this stuff would actually have to implement it as well. ;-)

igrigorik · 2011-02-20T23:24:29Z

fix rfc2616 / rfc2068 conflict for 100-continue behavior, closed by e22e948

alexkwolfe · 2011-03-15T20:11:56Z

I meant to thank you sooner for looking at this and fixing it. Thanks!

…gorik#74

jormon pushed a commit to jormon/em-http-request that referenced this issue Oct 6, 2011

fix rfc2616 / rfc2068 conflict for 100-continue behavior, closes igri…

e22e948

…gorik#74

This issue was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Client doesn't handle HTTP continue #74

Client doesn't handle HTTP continue #74

activeprospect commented Feb 16, 2011

igrigorik commented Feb 16, 2011

activeprospect commented Feb 16, 2011

activeprospect commented Feb 16, 2011

igrigorik commented Feb 16, 2011

activeprospect commented Feb 16, 2011

igrigorik commented Feb 16, 2011

alexkwolfe commented Feb 16, 2011

igrigorik commented Feb 17, 2011

activeprospect commented Feb 17, 2011

igrigorik commented Feb 19, 2011

igrigorik commented Feb 20, 2011

alexkwolfe commented Mar 15, 2011

Client doesn't handle HTTP continue #74

Client doesn't handle HTTP continue #74

Comments

activeprospect commented Feb 16, 2011

igrigorik commented Feb 16, 2011

activeprospect commented Feb 16, 2011

activeprospect commented Feb 16, 2011

igrigorik commented Feb 16, 2011

activeprospect commented Feb 16, 2011

igrigorik commented Feb 16, 2011

alexkwolfe commented Feb 16, 2011

igrigorik commented Feb 17, 2011

activeprospect commented Feb 17, 2011

igrigorik commented Feb 19, 2011

igrigorik commented Feb 20, 2011

alexkwolfe commented Mar 15, 2011