New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle multiple Content-Type headers correctly #31238

Open
wants to merge 2 commits into
base: devel
from

Conversation

Projects
None yet
3 participants
@sbktrifork
Contributor

sbktrifork commented Oct 3, 2017

SUMMARY

Avoids situations where mulitple Content-Type headers including charset information can result in errors like

LookupError: unknown encoding: UTF-8, text/html

https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2 states that duplicate headers are ok.
Urllib correctly returns a comma-seperated list of Content-Type values - but cgi.parse_header isn't able to handle that, so we need to split it up.

ISSUE TYPE
  • Bugfix Pull Request
COMPONENT NAME

modules/network/basics/uri.py

ANSIBLE VERSION
ansible 2.3.2.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
  python version = 2.7.6 (default, Oct 26 2016, 20:30:19) [GCC 4.8.4]
ADDITIONAL INFORMATION
- name: Waiting for isalive urls
  become: False
  uri:
    url: http://domain/healthcheck
    status_code: 200
    return_content: true
  register: response
  until: response.content == 'isalive'
  retries: 2
  delay: 2

... can result in...

"module_stdout": "Traceback (most recent call last):\r\n  File \"/tmp/ansible_vadAlD/ansible_module_uri.py\", line 498, in <module>\r\n    main()\r\n  File \"/tmp/ansible_vadAlD/ansible_module_uri.py\", line 478, in main\r\n    u_content = to_text(content, encoding=content_encoding)\r\n  File \"/tmp/ansible_vadAlD/ansible_modlib.zip/ansible/module_utils/_text.py\", line 235, in to_text\r\nLookupError: unknown encoding: UTF-8, text/html\r\n",

... if the http response returns multiple Content-Type headers like this ...

Content-Type: text/html; charset=UTF-8
Content-Length: 500
Set-Cookie: JSESSIONID=sdfsdfsdfsdf; Path=/url; HttpOnly
Date: Tue, 03 Oct 2017 07:37:09 GMT
Content-Type: text/html

The combined content-type is currently sendt comma-seperated to cgi.parse_header, which isn't able to handle multiple values, so it returns "charset=UTF8, text/html", which then is used for decoding and results in the unknown encoding error message.

The pull request simply sends one content_header value at a time to cgi.parse_header to get the right encoding.

Handle multiple Content-Type headers correctly
Avoids situations where mulitple Content-Type headers including charset information can result in errors like 
```
LookupError: unknown encoding: UTF-8, text/html
```
@ansibot

This comment has been minimized.

Contributor

ansibot commented Oct 3, 2017

@ansibot

This comment has been minimized.

Contributor

ansibot commented Mar 19, 2018

@sbktrifork this PR contains the following merge commits:

Please rebase your branch to remove these commits.

click here for bot help

@ansibot ansibot added stale_ci and removed needs_info labels Mar 27, 2018

@ansibot ansibot added the traceback label May 30, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment