"EOFError: bad content body" for multipart form requests #903

Closed
rplopes opened this Issue Jun 23, 2015 · 23 comments

Projects

None yet
@rplopes
rplopes commented Jun 23, 2015

On a multipart form where users can submit profile information like a profile picture upload we're getting occasional "EOFError: bad content body" errors. They come mostly from mobile devices. These errors are being raised in 2 different parts of the code:

The errors are rare enough that they don't seem to be related to one particular setup. The fact that they happen mostly on mobile and in a form where users can upload profile pictures suggests that it might have to do with network issues, but that's as far as we can get.

This error gets reported to HoneyBadger, but it never reaches the Rails application code, so we never get to the controller and we can't rescue from it.

The closest we got to reproducing this was with a request like:

curl 'http://localhost:3000/user' -H 'Content-Type: multipart/form-data; boundary=---------------------------6230707608200005481346647437' --data 'user[short_description]=A profile'

Which in development returns an error page that includes:

  <h1 class="err">Error starting application</h1>
  <h2>Your Rack app raised an exception when Pow tried to run it.</h2>
  <section>
    <pre class="breakout small_text"><strong>EOFError: bad content body</strong>
~/.gem/ruby/2.1.6/gems/rack-1.6.4/lib/rack/multipart/parser.rb:97:in `block in fast_forward_to_first_boundary'
~/.gem/ruby/2.1.6/gems/rack-1.6.4/lib/rack/multipart/parser.rb:95:in `loop'
~/.gem/ruby/2.1.6/gems/rack-1.6.4/lib/rack/multipart/parser.rb:95:in `fast_forward_to_first_boundary'
~/.gem/ruby/2.1.6/gems/rack-1.6.4/lib/rack/multipart/parser.rb:53:in `parse'
~/.gem/ruby/2.1.6/gems/rack-1.6.4/lib/rack/multipart.rb:25:in `parse_multipart'
<a href="#" onclick="this.style.display='none',this.nextSibling.style.display='';return false">Show 44 more lines</a><div style="display: none">~/.gem/ruby/2.1.6/gems/rack-1.6.4/lib/rack/request.rb:375:in `parse_multipart'

Is there something we're overlooking? We'd expect our Rails app to at least return something like a 400 error instead of a 500.

@allxie
allxie commented Sep 2, 2015

+1 on this issue. I'm getting it using paperclip to upload photos.

@catherinetcai

+1 on this issue as well. Also getting this issue with Paperclip.

@JWesorick

+1

@yuri-zubov

+1

@ashish-agrawal-metacube

after debugging the issue i knew that in my case i was getting following error because server was not receiving file at all, to debug the issue i created a custom middleware and put it on the top of middleware stack and logged body content of POST request
I printed env["rack.input"].sting in a file and got it empty, although a 400 error might have been returned.

Rack app error: EOFError: bad content body
~/.gem/ruby/2.1.6/gems/rack-1.6.4/lib/rack/multipart/parser.rb:97:in `block in fast_forward_to_first_boundary'
~/.gem/ruby/2.1.6/gems/rack-1.6.4/lib/rack/multipart/parser.rb:95:in`loop'
~/.gem/ruby/2.1.6/gems/rack-1.6.4/lib/rack/multipart/parser.rb:95:in `fast_forward_to_first_boundary'
~/.gem/ruby/2.1.6/gems/rack-1.6.4/lib/rack/multipart/parser.rb:53:in`parse'
~/.gem/ruby/2.1.6/gems/rack-1.6.4/lib/rack/multipart.rb:25:in `parse_multipart'
Show 44 more lines
~/.gem/ruby/2.1.6/gems/rack-1.6.4/lib/rack/request.rb:375:in`parse_multipart'
@raouldevil

+1 Getting this issue when post a multipart form using an Android app the recommended HttpsUrlConnection class.

@f3ndot
f3ndot commented Nov 20, 2015

๐Ÿ‘ As well, leaking stack trace info in production-environment Rails 4.1 app

EDIT: The stack track leak appears to be due to another gem

@dmill
dmill commented Dec 2, 2015

+1 - to paperclip from Android. It is only happening intermittently for me though.

@JWesorick

Anyone have a work around?

@sergio-bobillier

This bug seems to happen when the client sends an empty "multipart form/data" POST request. I discovered It when debugging a misbehaving Angular application that was sending no data to the server:

The request is as follows:

Accept:application/json, text/plain, */*
Accept-Encoding:gzip, deflate
Accept-Language:es,es-419;q=0.8,en;q=0.6,en-US;q=0.4,ja;q=0.2
Connection:keep-alive
Content-Length:44
Content-Type:multipart/form-data; boundary=----WebKitFormBoundaryVk7UtUqqetB9e6KS
Host:devsite.local
Origin:http://devsite.local
Referer:http://devsite.local/admin
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.82 Safari/537.36
X-CSRF-Token:2V6+DXyaRwc5EB2w3Ak3krTmhzXV/GGf72fKHEjzqX1RbOrbMwO2mYOOQ1zKbNrX75I3GA3ah2l2GoCKoEuONg==

------WebKitFormBoundaryVk7UtUqqetB9e6KS--

This causes an error in Rack, then in passenger and finally Apache throws a 500 Internal Server error, here is the stack trace:

[Fri Jan 22 17:50:53.091341 2016] [core:error] [pid 29178] [client 127.0.0.1:56557] End of script output before headers: admin, referer: http://devsite.local/admin
App 29086 stderr: [ 2016-01-22 17:52:13.4371 29165/0x000000015083a8(Worker 1) utils.rb:68 ]: *** Exception EOFError in Rack application object (bad content body) (process 29165, thread 0x000000015083a8(Worker 1)):
App 29086 stderr:   from /var/lib/gems/2.0.0/gems/rack-1.6.4/lib/rack/multipart/parser.rb:97:in `block in fast_forward_to_first_boundary'
App 29086 stderr:   from /var/lib/gems/2.0.0/gems/rack-1.6.4/lib/rack/multipart/parser.rb:95:in `loop'
App 29086 stderr:   from /var/lib/gems/2.0.0/gems/rack-1.6.4/lib/rack/multipart/parser.rb:95:in `fast_forward_to_first_boundary'
App 29086 stderr:   from /var/lib/gems/2.0.0/gems/rack-1.6.4/lib/rack/multipart/parser.rb:53:in `parse'
App 29086 stderr:   from /var/lib/gems/2.0.0/gems/rack-1.6.4/lib/rack/multipart.rb:25:in `parse_multipart'
App 29086 stderr:   from /var/lib/gems/2.0.0/gems/rack-1.6.4/lib/rack/request.rb:375:in `parse_multipart'
App 29086 stderr:   from /var/lib/gems/2.0.0/gems/rack-1.6.4/lib/rack/request.rb:207:in `POST'
App 29086 stderr:   from /var/lib/gems/2.0.0/gems/rack-1.6.4/lib/rack/methodoverride.rb:39:in `method_override_param'
App 29086 stderr:   from /var/lib/gems/2.0.0/gems/rack-1.6.4/lib/rack/methodoverride.rb:27:in `method_override'
App 29086 stderr:   from /var/lib/gems/2.0.0/gems/rack-1.6.4/lib/rack/methodoverride.rb:15:in `call'
App 29086 stderr:   from /var/lib/gems/2.0.0/gems/rack-1.6.4/lib/rack/runtime.rb:18:in `call'
App 29086 stderr:   from /var/lib/gems/2.0.0/gems/activesupport-4.2.3/lib/active_support/cache/strategy/local_cache_middleware.rb:28:in `call'
App 29086 stderr:   from /var/lib/gems/2.0.0/gems/rack-1.6.4/lib/rack/lock.rb:17:in `call'
App 29086 stderr:   from /var/lib/gems/2.0.0/gems/actionpack-4.2.3/lib/action_dispatch/middleware/static.rb:116:in `call'
App 29086 stderr:   from /var/lib/gems/2.0.0/gems/rack-1.6.4/lib/rack/sendfile.rb:113:in `call'
App 29086 stderr:   from /var/lib/gems/2.0.0/gems/railties-4.2.3/lib/rails/engine.rb:518:in `call'
App 29086 stderr:   from /var/lib/gems/2.0.0/gems/railties-4.2.3/lib/rails/application.rb:165:in `call'
App 29086 stderr:   from /usr/lib/ruby/vendor_ruby/phusion_passenger/rack/thread_handler_extension.rb:77:in `process_request'
App 29086 stderr:   from /usr/lib/ruby/vendor_ruby/phusion_passenger/request_handler/thread_handler.rb:142:in `accept_and_process_next_request'
App 29086 stderr:   from /usr/lib/ruby/vendor_ruby/phusion_passenger/request_handler/thread_handler.rb:110:in `main_loop'
App 29086 stderr:   from /usr/lib/ruby/vendor_ruby/phusion_passenger/request_handler.rb:448:in `block (3 levels) in start_threads'

Sending the same empty request to a PHP backed application causes no error.

@tyler-nguyen

I made a middleware to work around this.

class BadMultipartFormDataSanitizer
  def initialize(app)
    @app = app
  end

  def call(env)
    if env['CONTENT_TYPE'] =~ /multipart\/form-data/
      begin
        Rack::Multipart.parse_multipart(env)
      rescue EOFError => ex
        # set content-type to multipart/form-data without the boundary part
        # to handle the case where empty content is submitted
        env['CONTENT_TYPE'] = 'multipart/form-data'
      end
    end

    @app.call(env)
  end
end

And added this to Rails config/application.rb

config.middleware.insert_before Rack::Runtime, 'BadMultipartFormDataSanitizer'

Anyone get a better idea?

@carloshlopez

+1 on this issue, having trouble from Android app developed in cordova. Any other clues, besides @tyler-nguyen work arround?

EDIT:
I have found that my error was because I was using PUMA as a web server, went back to unicorn and it worked

@prakashraman

+1

Had the same problem. I noticed that I had not mentioned the "name" attribute of my tag.

@isqad88
isqad88 commented Jul 14, 2016 edited

+1

Reproducing:

$ curl 'http://localhost' --data 'user=test'

#=>  env["CONTENT_TYPE"] =>"application/x-www-form-urlencoded"  - Its ok

curl 'http://localhost'  -H 'Content-Type: multipart/form-data' --data 'test=test'

#=> env["CONTENT_TYPE"] => "multipart/form-data" - Its ok

curl 'http://localhost' -H 'Content-Type: multipart/form-data; boundary=abc' --data 'test=test'

#=> env["CONTENT_TYPE"] => "multipart/form-data; boundary=abc" - Its Bad
@sahilbhatia-edmodo

+1 I am also getting this error when calling rails endpoint running behind a puma application server when, client sends an empty "multipart form/data" POST request.

@isqad88
isqad88 commented Jul 20, 2016 edited
#=> env["CONTENT_TYPE"] => "multipart/form-data; boundary=abc" - Its Bad

I tried debug for this issue, and I found that this code can not find first boundary in request body:

https://github.com/rack/rack/blob/rack-1.3/lib/rack/multipart/parser.rb#L82

UPD:
I specifically sent the body without boundary but http header with content type multipart/form-data; boundary=abc, and I got this issue

@carloshlopez

Andrew, does your code does work in unicorn?

On Wed, Jul 20, 2016, 2:35 AM Andrew N. Shalaev notifications@github.com
wrote:

#=> env["CONTENT_TYPE"] => "multipart/form-data; boundary=abc" - Its Bad

I tried debug for this issue, and I found that this code can not find
first boundary in request body:

https://github.com/rack/rack/blob/rack-1.3/lib/rack/multipart/parser.rb#L82

โ€”
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#903 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAR6Albtqh4mSXztf9XH49fda6TN2ybAks5qXc-xgaJpZM4FJ_ib
.

@isqad88
isqad88 commented Jul 21, 2016

@carloshlopez I tested in webrick with curl and debugger

@liudangyi

multipart/parser.rb is refactored in version 2.0.0. However, some problems still exist in the latest code. I encounter this problem today, with the following code.

<form action="http://localhost:4567/echo" method="post" enctype="multipart/form-data">
  <input type="submit" value="Submit">
</form>

This form will submit with an empty content, like this

Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryvrASEpt2t7qHj4v6

------WebKitFormBoundaryvrASEpt2t7qHj4v6--

Rack doesn't handle this correctly, with the following error.

Unexpected error while processing request: EOFError
    /usr/local/lib/ruby/gems/2.3.0/gems/rack-2.0.1/lib/rack/multipart/parser.rb:361:in `handle_empty_content!'
    /usr/local/lib/ruby/gems/2.3.0/gems/rack-2.0.1/lib/rack/multipart/parser.rb:185:in `on_read'
    /usr/local/lib/ruby/gems/2.3.0/gems/rack-2.0.1/lib/rack/multipart/parser.rb:72:in `block in parse'
    /usr/local/lib/ruby/gems/2.3.0/gems/rack-2.0.1/lib/rack/multipart/parser.rb:70:in `loop'
    /usr/local/lib/ruby/gems/2.3.0/gems/rack-2.0.1/lib/rack/multipart/parser.rb:70:in `parse'
    /usr/local/lib/ruby/gems/2.3.0/gems/rack-2.0.1/lib/rack/multipart.rb:52:in `extract_multipart'
    /usr/local/lib/ruby/gems/2.3.0/gems/rack-2.0.1/lib/rack/request.rb:472:in `parse_multipart'
    /usr/local/lib/ruby/gems/2.3.0/gems/rack-2.0.1/lib/rack/request.rb:335:in `POST'

...omitted

Rack detects such situation and raises an error. However, since it's a normal request sent from a browser, maybe rack should just ignore it without any complaint.

@abarrak
abarrak commented Aug 17, 2016 edited

I had the same error recently, and yes, it turns out because of an empty submission.
In my case, it was a (307) redirect after the original form post what caused the issue.

For what it worths, I was seeing it on Safari only, but not on Firefox or Chrome

@samn
samn commented Oct 20, 2016

I'm also running into this, and was able to reproduce it by sending multipart form data with a Content-Length header smaller than the length of the body.

Used this monkey patch with rack 1.6.0 to capture the error and let execution continue. The goal wasn't to fix the malformed request but rather to allow it to proceed to the application (Sinatra in this case) so it could return an appropriate error code.

module Rack
  module Multipart
    class << self
      alias_method :original_parse_multipart, :parse_multipart
      def parse_multipart(env)
        begin
          original_parse_multipart(env)
        rescue EOFError => e
          nil
        end
      end
    end
  end
end
@liukgg
liukgg commented Nov 22, 2016

+1

@carlzulauf
Contributor

Inserting a simple middleware before Rack::MethodOverride is also a work around. However, since other middleware like Rack::Sendfile write to env[RACK_ERRORS] I think the correct solution is probably to do the same in Rack::MethodOverride. I will try to submit a pull that does that.

Workaround middleware:

class IgnoreEofMiddleware
  def initialize(app)
    @app = app
  end

  def call(env)
    @app.call(env)
  rescue EOFError
    [400, {}, ["bad content"]]
  end
end

# then insert middleware before MethodOverride
# in rails that would look like
Rails.application.middleware.insert_before Rack::MethodOverride, "IgnoreEofMiddleware"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment