Fix MIME::Type.parse handling of single media with a q value

[scottwb]

An HTTP request header like this:

Accept: text/html;q=0.9

Is valid according to the HTTP spec, but is not handled well by Mime::Type.parse. It is not uncommon to receive requests like this from bots and, according to one source, the PSP's browser. The failed parsing typically manifests itself as a MissingTemplate error.

This change fixes issue #736, which is not currently open, but should be.

The fix should be easy to also apply to 3.x branches. I'd be happy to help with that if necessary.

[masterkain]

Related: #4127 #701

[josevalim]

Could you please also add a test that handles multiple types with q=? Your changes also updated this part of the code, but you haven't added any tests for it. Thanks.

[scottwb]

@josevalim Just above that line (line 66) there is a pre-existing test that handles multiple types with q=:

  test "parse with q" do
    accept = "text/xml,application/xhtml+xml,text/yaml; q=0.3,application/xml,text/html; q=0.8,image/png,text/plain; q=0.5,application/pdf,*/*; q=0.2"
    expect = [Mime::HTML, Mime::XML, Mime::PNG, Mime::PDF, Mime::TEXT, Mime::YAML, Mime::ALL]
    assert_equal expect, Mime::Type.parse(accept)
  end

This test passes before and after my change. Or am I misunderstanding your request?

[josevalim]

That was it, thanks! Merging.

[bittersweet]

Just wanted to say thanks for taking the time and doing the effort scottwb! I hope your work fixes it once and for all :)

[pheld]

Here's a freedom patch that works for this for now:

  class << Mime::Type
    Q_SEPARATOR_REGEXP = /;\s*q=/

    def parse_with_q_fix(accept_header)
      if accept_header !~ /,/
        accept_header = accept_header.split(Q_SEPARATOR_REGEXP).first
      end

      parse_without_q_fix accept_header
    end

    alias_method_chain :parse, :q_fix
  end

[jeremywadsack]

@scottwb Thanks for this fix and getting it merged. GoogleBot hits us on this every day now. @pheld Thanks for the monkey patch for the meantime.

[asanghi]

This fix is in Rails master (4.0) or Rails stable (3.2.x)? If not, can this please be merged into 3.2.x?

[scottwb]

As far as I know, it only got merged into edge (4.0). I was gonna work up some additional pull requests for the various 3.x branches one of these days. It shouldn't be too hard if you wanna take a stab at it. The changes are pretty small and should be straightforward to apply to Rails 3.

On Feb 16, 2012, at 23:47, Aditya Sanghi reply@reply.github.com wrote:

This fix is in Rails master (4.0) or Rails stable (3.2.x)? If not, can this please be merged into 3.2.x?

---

Reply to this email directly or view it on GitHub:
#4918 (comment)

[asanghi]

Fine. In the mood for taking on some stabbing.

[bokor]

what version was this merged into. I'm still on 3.0.9

[scottwb]

@designwaves I am pretty sure it was merged into 3.2.x. Not sure what 'x' is, but when I discussed it with some of the maintainers they said they did not plan any functional updates to 3.0 and 3.1 at the time so it would only be relevant to apply it to 3.2 and later.

[bokor]

Scott,

Is there a monkey patch that I can cut and paste into 3.0.9?

[steveklabnik]

@scottwb @designwaves Above, you can see this was merged into 3.2 stable.

[scottwb]

@designwaves I posted a while back about some monkey-patches I was using on 3.0.7, which should probably work for you on 3.0.9: https://gist.github.com/1754727, however I think I like the one pasted above by @pheld better.

[bokor]

sweet thank you so much for helping out with this. We are getting enough traffic now that this is becoming an issue and wanted to patch it. Awesome work man!

[bokor]

I used the Gist and it fixed all my issues. The monkey patch above only fixed one but your code fixed all. thank you for that. It really helped out.

[mladeny]

Has this been fixed for 3.2?

I saw that it was merged into master, but I'm using 3.2.13 and I'm having the same issue.

ActionView::MissingTemplate: Missing template homes/show, application/show with {:locale=>[:en], :formats=>["text/html; chars...lder, :slim, :coffee]}. Searched in: * "/app/app/views" * "/app/vendor/bundle/ruby/1.9.1/gems/kaminari-0.14.1/app/views"

HTTP_ACCEPT
text/html; charset=utf-8; q=0.1

seems to be happening with bots (as mentioned above). Here's the user agent that triggered it

HTTP_USER_AGENT
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36 HubSpot Marketing Grader

replicated with

curl -H"Accept: text/html; charset=utf-8; q=0.1" http://localhost:3000/

[rafaelfranca]

It seems to be included in 3-2-stable so maybe it is in one 3.2.x release.

[mladeny]

@rafaelfranca doesn't seem so.

I just tried 3.2.18, which is the latest version as shown in the RAILS_VERSION file under the 3.2-stable branch and I still get the same error...

[rafaelfranca]

Maybe it was reverted because caused some regression.