Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

POST Request Body Ignored For Chunked Transfer-Encoding #7556

Closed
couchdeveloper opened this Issue · 11 comments

5 participants

@couchdeveloper

Greetings!

In an attempt to send a POST request with JSON data with HTTP header "Transfer-Encoding: Chunked" from a client to the server I encountered that the Parameters hash always evaluates to empty.

Looking where the request is handled in the middleware, the reason for this becomes quickly clear:

In file params_parser.rb:

      def parse_formatted_parameters(env)
        request = Request.new(env)

        return false if request.content_length.zero?
        ...

If the request is send with header "Transfer-Encoding: Chunked" there is no header "Content-Length". Nonetheless, there is a io stream which possibly contains data.

Is there a compelling reason for parsing the body data that requires that there is a content length?

(I've looked into the sources for the JSON parsers, and it doesn't seem so.)

On my local system to test a possible fix I changed the line

        return false if request.content_length.zero?

to

        return false if request.content_length.zero? && !(request.headers["Transfer-Encoding"].downcase.include? == "chunked")

and it worked for my limited tests sending a POST request with JSON data and header "Transfer-Encoding: Chunked" from a client.

Note: I'm not a Rails developer, and does not have a development environment yet. The workaround above has probably a better solution.

I'm using Rails primarily for testing clients, currently using WEBrick for development.

I would appreciate it if someone could comment this! :)

Thanks in advance!

Andreas

Edit:

Fixed my private suggestion:
return false if request.content_length.zero? && request.body.eof?

@tenderlove
Owner

This seems like a bug! I'll see what I can do.

@couchdeveloper
@al2o3cr

This may be a more substantial issue than it appears - looks like Rack gets grumpy when sent a chunked POST body:

rack/rack#418

@couchdeveloper
@spastorino
Owner

My PR rack/rack#432 was merged in rack so this will be fixed in the next release.
Can you test using rack from github in your app?.
I'm closing this for now but let me know if with my fix works or don't :).

@spastorino spastorino closed this
@couchdeveloper
@couchdeveloper

~~I've put more time in testing. It seems the underlaying JSON parser is not capable to parse chunked input. As an example I issued a POST request whose send buffer has size 32. ~~

Edit:

I can take this back!

It turned out, there is a bug in a JSON serializer - which incorrectly decoded an object hierarchy to JSON (which is part of a system library on the client platform). Of course, Rails and multi_json cannot accept it.

I apologize for the noise.

@spastorino
Owner

@couchdeveloper sorry but my english is not very good. Did the code work for you?

@steveklabnik
Collaborator

I read it that way.

@couchdeveloper

The "I can take back this" refers to an issue which I thought has to do with the underlaying JSON parser in Rails. However, this was not the case. Indeed this was an issue in a system library of the client OS, namely the JSON serializer in iOS, Mac OS X. I haven't thoroughly tested the various JSON parsers used in Rails, though.

Now, specifically to your code, and these two lines and I want to clarify my concern. Just let me add that I cannot test (yet) - since I don't have a Rails development environment. (Maybe I should, since the more I become involved in Rails, the more interesting it becomes ;) )

          content = @io.read(@content_length && BUFSIZE >= @content_length ? @content_length : BUFSIZE)
          raise EOFError, "bad content body"  if content.nil? || content.empty?

If I understood it correctly, @content_length is nil in case of header "Transfer-Encoding: Chunked".
so we have effectively:
content = @io.read(BUFSIZE)

If the stream's size is zero, it returns nil. This can happen, if this is the first call and the stream is empty. It can also happen if the previous call just read all remaining bytes from the stream - that is the stream is at eof.

So, I think, we need to break the loop once the stream is at eof. Alternatively, we can check if we are at eof, if we read less bytes than BUFSIZE and more than zero.

The second question is, if we should consider an empty body still as valid. If you can agree, in this case we do nothing.

Thanks! :)

@spastorino
Owner

@couchdeveloper please investigate on that

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.