Skip to content

Fails with wrong headers #1370

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ghost opened this issue Jun 23, 2015 · 8 comments
Closed

Fails with wrong headers #1370

ghost opened this issue Jun 23, 2015 · 8 comments

Comments

@ghost
Copy link

ghost commented Jun 23, 2015

Originally reported by: Daniel Frischhut (Bitbucket: daniel_frischhut, GitHub: Unknown)


We got a request with the following header-fields:

  'CONNECTION': 'close'
  'CACHE-CONTROL': 'no-cache'
  'ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
  'ACCEPT-CHARSET': 'ISO-8859-1,utf-8;q=0.7,*;q=0.7)'
  'USER-AGENT': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'
  'ACCEPT-LANGUAGE': 'en-us'
  'ACCEPT-ENCODING': 'gzip,deflate'

and cherrypy faild with follwing Traceback:

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 670, in respond
    response.body = self.handler()
  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/encoding.py", line 260, in __call__
    ct.params[\'charset\'] = self.find_acceptable_charset()
  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/encoding.py", line 146, in find_acceptable_charset
    encs = request.headers.elements(\'Accept-Charset\')
  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/httputil.py", line 456, in elements
    return header_elements(key, value)
  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/httputil.py", line 237, in header_elements
    return list(reversed(sorted(result)))
  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/httputil.py", line 217, in __lt__
    if self.qvalue == other.qvalue:
  File "/usr/local/lib/python2.7/dist-packages/cherrypy/lib/httputil.py", line 207, in qvalue
    return float(val)
ValueError: invalid literal for float(): 0.7)

i edited the httputil.py in line 207 and surrounded the return float(val) with a try - except and returned the default value.


@ghost
Copy link
Author

ghost commented Jun 23, 2015

Original comment by duong nt (Bitbucket: bugmenot4, GitHub: bugmenot4):


"q=0.7)" is invalid, you should fix the application

@ghost
Copy link
Author

ghost commented Jun 23, 2015

Original comment by Daniel Frischhut (Bitbucket: daniel_frischhut, GitHub: Unknown):


I know that this is invalid, I got this header in a productive environment, so I have no influence on the browsers out there. The application returns a 500 error right now, this may is wanted in some cases - in other cases the application should continue and just ignore the wrong header element.

@ghost
Copy link
Author

ghost commented Jul 19, 2015

Original comment by Sylvain Hellegouarch (Bitbucket: Lawouach, GitHub: Lawouach):


Hi Daniel,

I don't think it's a bug per-se and the server reacts accordingly by sending out a 500 error. Yet, from a user point of view, I can appreciate you'd want the encoding header to be more forgiving.

This is more of a feature than a bug however.

@webknjaz
Copy link
Member

@jaraco I don't think there should be any forgiveness to malformed input. Do you?

@jaraco
Copy link
Member

jaraco commented Oct 4, 2016

Malformed input should return a 40x error and not a 500 error. I'm less sure about whether the error should be silently suppressed or raise a 40x.

@skieffer
Copy link

skieffer commented Jan 18, 2018

This morning my website experienced the same issue, and due to the exact same malformed header:

ACCEPT-CHARSET: ISO-8859-1,utf-8;q=0.7,*;q=0.7)

What is this client out there in the wild that adds this erroneous close-parenthesis?

For what it's worth, I came here to make the same request, i.e. that we tolerate the header. I think when the user sees a 500 error it makes me look bad, and I imagine they are probably totally unaware that they are sending this malformed header. They will never fix the problem, and my website will continue to appear broken to them.

@seils
Copy link

seils commented Apr 20, 2018

@webknjaz, @jaraco, I'll take this one.

I'm considering a couple of approaches:

  • Catch the ValueError exception and return float(0) instead, essentially making the charset with the malformed weight value unusable.
  • Catch the ValueError and remove the charset with the malformed weight value from the list.

Any preference?

@webknjaz
Copy link
Member

@seils @skieffer Malicious requests have to result in client errors http 400 (40x).
We should not tolerate this.
It this comes from client side, it means that they (1) don't use a normal browser, which normally implies that (2) it's a script/bot/exploit scanner.
So the only correct thing to do (RFC compliant) is to return 400.

@seils seils mentioned this issue Apr 20, 2018
webknjaz pushed a commit that referenced this issue May 28, 2018
Return  for requests with malformed Accept-Charset quality values.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants