New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

204 No Content responses may contain both Content-Length and Transfer-Encoding: chunked headers #1595

Closed
sjmadsen opened this Issue Sep 1, 2015 · 5 comments

Comments

Projects
None yet
5 participants
@sjmadsen

sjmadsen commented Sep 1, 2015

Let me start by saying this is a complicated problem that I've spent several hours tracking down, and it's certainly possible that the root cause isn't Passenger. I'm starting here because it's how I've worked around the problem for now.

Over the weekend, I pushed some updates to a customer's site that has been stable for many months. As part of this, I installed a handful of security and infrastructure updates. This server runs Debian Wheezy, and one of the updates was from Apache 2.2.22-13+deb7u4 to +deb7u6. Passenger was upgraded from 5.0.8 to 5.0.16.

Today I was notified that clients of an API exposed by the application were no longer functioning correctly. In particular, iOS 8.x clients were unable to parse the response for a request that returns 204 No Content. I eventually found that the server was responding with both Transfer-Encoding: chunked and Content-Length: 0 headers. According to the HTTP 1.1 spec, this is not permitted, and iOS's networking frameworks are rather picky about it.

I tried downgrading the Apache packages and it did not resolve the problem.

I found that Passenger 5.0.9 included a change that reverted the downcasing of headers from the application behind Passenger. After downgrading Passenger from 5.0.16 to 5.0.8, the application started working normally again. I re-upgraded the Apache packages and everything is still OK.

The application in question is a Rails 4.0.x app on Ruby 2.1. The changes I made over the weekend are minimal and did not touch any of the API endpoints. None of the configuration files have been touched in a long time (Apache, Passenger, etc.). This application was first deployed with Passenger 4.x, which as I understand from research also did not downcase headers.

So, I'm a little perplexed. The downcasing is (was) new, it didn't happen in 4.x when header case was untouched, it's a problem now that headers are again untouched. I don't see anything related to Transfer-Encoding after 5.0.0-rc1. On 5.0.8, Transfer-Encoding isn't in the response headers at all, but it is in 5.0.16. On 5.0.16, with Passenger's log level set to 7, I didn't see Content-Length in Passenger's responses. I suppose it's being added by Apache, but it shouldn't be doing that since Transfer-Encoding is already present.

I have a staging server available and could do some additional research if it'll help, but I could use some guidance about how to proceed.

@OnixGH

This comment has been minimized.

Show comment
Hide comment
@OnixGH

OnixGH Sep 1, 2015

Contributor

Possibly the same as: #1517

Passenger adds Transfer-Encoding if it doesn't see "Content-Length" in the application response, with that exact casing (that's the new part). So doublecheck if the app really isn't sending out something with a different case like "Content-length" or "content-length".

Contributor

OnixGH commented Sep 1, 2015

Possibly the same as: #1517

Passenger adds Transfer-Encoding if it doesn't see "Content-Length" in the application response, with that exact casing (that's the new part). So doublecheck if the app really isn't sending out something with a different case like "Content-length" or "content-length".

@Benabik

This comment has been minimized.

Show comment
Hide comment
@Benabik

Benabik Oct 2, 2015

I'm having this exact same problem with a Rails app that uses head :no_content, aka 204 No Content, for API calls that succeed without reply. Rack seems to think that sending 204 with a Content-Length header is an error. Rack::Lint checks for it, and using that at the top of my middleware stack indicates there is no Content-Length when the response leaves the app.

I tested the following:

  • WEBrick (rails server) has neither header.
  • Apache 2.4.7/Passenger 4.0.59 has Content-Length: 0.
  • Apache 2.4.7/Passenger 5.0.20 has Content-Length: 0 AND Transfer-Encoding: chunked.

Our API users succeed under 4.0.59, but fail under 5.0.20. Curl gives error 18 "transfer closed with outstanding read data remaining".

(Edited to add Apache version.)

Benabik commented Oct 2, 2015

I'm having this exact same problem with a Rails app that uses head :no_content, aka 204 No Content, for API calls that succeed without reply. Rack seems to think that sending 204 with a Content-Length header is an error. Rack::Lint checks for it, and using that at the top of my middleware stack indicates there is no Content-Length when the response leaves the app.

I tested the following:

  • WEBrick (rails server) has neither header.
  • Apache 2.4.7/Passenger 4.0.59 has Content-Length: 0.
  • Apache 2.4.7/Passenger 5.0.20 has Content-Length: 0 AND Transfer-Encoding: chunked.

Our API users succeed under 4.0.59, but fail under 5.0.20. Curl gives error 18 "transfer closed with outstanding read data remaining".

(Edited to add Apache version.)

@FooBarWidget FooBarWidget added this to the 5.0.21 milestone Oct 2, 2015

@FooBarWidget

This comment has been minimized.

Show comment
Hide comment
@FooBarWidget

FooBarWidget Oct 19, 2015

Member

It looks like this issue is not the same as #1517. Will schedule this for 5.0.22.

Member

FooBarWidget commented Oct 19, 2015

It looks like this issue is not the same as #1517. Will schedule this for 5.0.22.

@FooBarWidget FooBarWidget modified the milestones: 5.0.22, 5.0.21 Oct 19, 2015

@FooBarWidget FooBarWidget changed the title from Response may contain both Content-Length and Transfer-Encoding: chunked headers to 204 No Conten responses may contain both Content-Length and Transfer-Encoding: chunked headers Oct 19, 2015

@FooBarWidget FooBarWidget changed the title from 204 No Conten responses may contain both Content-Length and Transfer-Encoding: chunked headers to 204 No Content responses may contain both Content-Length and Transfer-Encoding: chunked headers Oct 19, 2015

@FooBarWidget

This comment has been minimized.

Show comment
Hide comment
@FooBarWidget

FooBarWidget Nov 20, 2015

Member

The problem has been fixed now and the fix will be included in 5.0.22. Passenger was erroneously adding a Transfer-Encoding: chunked response to 204 responses. Apache then proceeds with adding a Content-Length 0 to it.

Member

FooBarWidget commented Nov 20, 2015

The problem has been fixed now and the fix will be included in 5.0.22. Passenger was erroneously adding a Transfer-Encoding: chunked response to 204 responses. Apache then proceeds with adding a Content-Length 0 to it.

@FooBarWidget FooBarWidget self-assigned this Nov 20, 2015

@ekille

This comment has been minimized.

Show comment
Hide comment
@ekille

ekille Nov 20, 2015

Thanks FooBarWidget the fix! Good timing, we used the patch this morning.

ekille commented Nov 20, 2015

Thanks FooBarWidget the fix! Good timing, we used the patch this morning.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment