New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Headers are returned as unicode string from Response.from_file with Python 2.7.3 #99

Closed
cdman opened this Issue Mar 25, 2013 · 4 comments

Comments

Projects
None yet
3 participants
@cdman
Copy link

cdman commented Mar 25, 2013

The situation in $subject makes it impossible to return the resulting object to the WSGI stack on GAE, since it expects all headers to be str (not unicode). I think the fix is to add something like the following around line 190 in webob/response.py:

        headerlist.append((
            native_(header_name, 'utf-8'),
            native_(value, 'utf-8')
        ))
@mcdonc

This comment has been minimized.

Copy link
Member

mcdonc commented Apr 3, 2013

Definitely a bug on Python 2, thank you for the report. I'm not sure it shouldn't be native_(foo, 'latin-1') however. I dont know how this gets used, or what uses it, or what the expectations are for decoding headers with nonascii chars. Typically the HTTP spec waffles about it but makes some vague assertions that response header values should be encoded as latin-1, but I always have to go swap the entire spec into my head to remember.

@cdman

This comment has been minimized.

Copy link

cdman commented Apr 6, 2013

Couple of more updates. My usecase is to be able to return a stored response from a WSGI app. A very simple test code I came up with is the following:

import io
import webapp2
from webob.compat import bytes_

STORED_RESPONSE = io.BytesIO(bytes_(str( webapp2.Response('test') )))

class MainApp(object):
    def __call__(self, environ, start_response):
        resp = webapp2.Response.from_file(STORED_RESPONSE)
        start_response(resp.status, resp.headerlist)
        return resp.app_iter

application = MainApp()

This errors out like this (traceback edited for my privacy :-)):

Traceback (most recent call last):
  File ".../google/appengine/runtime/wsgi.py", line 223, in Handle
    result = handler(dict(self._environ), self._StartResponse)
  File "/home/cdman/udacity/mini_gae/main.py", line 10, in __call__
    start_response(resp.status, resp.headerlist)
  File ".../google/appengine/runtime/wsgi.py", line 152, in _StartResponse
    (_GetTypeName(name), name))
InvalidResponseError: header names must be str, got 'unicode' (u'Content-Type')

As far as expected encoding it looks like you remember correctly and Latin-1 is expected: http://stackoverflow.com/a/4410331/1265

Also, as far usage of Response.from_file, I couldn't find any production code (aside from my own). All that came up were test code and I think that backward compatibility wouldn't be a big issue (especially given the fact 'foo' == u'foo' under Python). My searches:

@mcdonc

This comment has been minimized.

Copy link
Member

mcdonc commented Aug 10, 2013

Marking this as "sprintable" for upcoming sprints.

@mcdonc

This comment has been minimized.

Copy link
Member

mcdonc commented Aug 10, 2013

FTR, I believe the original fix mentioned (delta swapping out 'utf-8' for 'latin-1') is correct.

deuxpi pushed a commit to deuxpi/webob that referenced this issue May 28, 2014

Philippe Gauthier
Native `str` strings from Response.from_file
Make sure that header names extracted with Response.from_file are `str`
instances otherwise most WSGI implementations will reject the response.

Should fix issue Pylons#99

deuxpi pushed a commit to deuxpi/webob that referenced this issue May 28, 2014

Philippe Gauthier
Native `str` strings from Response.from_file
Make sure that header names extracted with Response.from_file are `str`
instances otherwise most WSGI implementations will reject the response.

Should fix issue Pylons#99

deuxpi pushed a commit to deuxpi/webob that referenced this issue May 28, 2014

Philippe Gauthier
Native `str` strings from Response.from_file
Make sure that header names extracted with Response.from_file are `str`
instances otherwise most WSGI implementations will reject the response.

Should fix issue Pylons#99

@bertjwregeer bertjwregeer added this to the Version 1.5 milestone Mar 23, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment