Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed #28174 -- Fixed crash in runserver's autoreload with Python 2 on Windows with non-str environment variables. #8737

Merged
merged 1 commit into from Jul 13, 2017

Conversation

@mrogaski
Copy link

mrogaski commented Jul 12, 2017

Ensure that all environment values are str objects to avoid a TypeError.

Ticket: https://code.djangoproject.com/ticket/28174

@timgraham timgraham changed the title Fix #28174: Groom environment in autoreload.restart_with_reloader Fixed #28174 -- Fixed crash in runserver's autoreload with Python 2 on Windows with non-str environment variables. Jul 12, 2017
Copy link
Member

timgraham left a comment

Add a release note in docs/releases/1.11.4.txt. You can squash commits and use the commit message from the PR title.

@@ -285,6 +285,9 @@ def restart_with_reloader():
while True:
args = [sys.executable] + ['-W%s' % o for o in sys.warnoptions] + sys.argv
new_environ = os.environ.copy()
if sys.platform == "win32":

This comment has been minimized.

Copy link
@timgraham

timgraham Jul 12, 2017

Member

There's a _win module level constant that could be used. Also, a comment seems appropriate. I believe the patch is only needed for Python 2 so maybe also check six.PY2? (not sure if this could cause an issue on Python 3 or not)

This comment has been minimized.

Copy link
@mrogaski

mrogaski Jul 12, 2017

Author

I can't think of a scenario in Python 3 where this would occur without effort to make it happen. I think it's worth keeping the scope narrow.

def test_restart_with_reloader(self):
os.environ['SPAM'] = u'spam'
with mock.patch.object(sys, 'argv', ['-c', 'pass']):
try:

This comment has been minimized.

Copy link
@timgraham

timgraham Jul 12, 2017

Member

The try/except can be omitted. If the test fails with an error, that's fine.

@@ -0,0 +1,20 @@
import os

This comment has been minimized.

Copy link
@timgraham

timgraham Jul 12, 2017

Member

Please put this test in the existing test_autoreload.py file in the same directory.

"""

def test_restart_with_reloader(self):
os.environ['SPAM'] = u'spam'

This comment has been minimized.

Copy link
@timgraham

timgraham Jul 12, 2017

Member

We use from __future__ import unicode_literals rather than u'' prefix.

@@ -285,6 +285,9 @@ def restart_with_reloader():
while True:
args = [sys.executable] + ['-W%s' % o for o in sys.warnoptions] + sys.argv
new_environ = os.environ.copy()
if sys.platform == "win32":
for key in new_environ.keys():
new_environ[key] = str(new_environ[key])

This comment has been minimized.

Copy link
@timgraham

timgraham Jul 12, 2017

Member

I believe key should also be coerced.

This comment has been minimized.

Copy link
@mrogaski

mrogaski Jul 12, 2017

Author

Yes, it should ... and turning on unicode_literals exposes the need. Thanks.

autoreload is used.
"""

def test_restart_with_reloader(self):

This comment has been minimized.

Copy link
@timgraham

timgraham Jul 12, 2017

Member
def test_environment(self):
    """"
    With Python 2 on Windows, restart_with_reloader() coerces  environment
    variables to str to avoid "TypeError: environment can only contain
    strings" in Python's subprocess.py.
    """
from django.utils import autoreload


class TestEnvironmentGrooming(SimpleTestCase):

This comment has been minimized.

Copy link
@timgraham

timgraham Jul 12, 2017

Member

TestRestartWithReloader

@mrogaski

This comment has been minimized.

Copy link
Author

mrogaski commented Jul 12, 2017

All recommended changes have been applied.

# we coerce both to the str type.
for key in new_environ.keys():
str_key = str(key)
str_value = str(new_environ[key])

This comment has been minimized.

Copy link
@charettes

charettes Jul 12, 2017

Member

I guess this is going to crash if the value contains non-ASCII chars? Maybe we should be using force_bytes(s, encoding=SYSTEM_ENCODING) instead of a bare str() call? I'm not a Windows expert but I think we have similar handling of data the localization tools.

strings" in Python's subprocess.py.
"""
def test_environment(self):
os.environ['SPAM'] = 'spam'

This comment has been minimized.

Copy link
@charettes

charettes Jul 12, 2017

Member

I know that this is only meant to land in 1.11.x wonder if using u'' prefixes/force_text() would make what we're testing more clear.

This comment has been minimized.

Copy link
@mrogaski

mrogaski Jul 12, 2017

Author

I think it's a good example of how most people will encounter the problem if we leave it as is. Indeed, unicode_literals is how I hit it. If clarity is a concern, an extra comment should be enough.

@mrogaski

This comment has been minimized.

Copy link
Author

mrogaski commented Jul 12, 2017

Updated to use force_bytes instead of str. I opted to stick with unicode_literals for the unit test error injection, but added a comment to indicate the purpose and mechanism.

…ython 2 on Windows with non-str environment variables.
@timgraham timgraham merged commit fc6b90b into django:stable/1.11.x Jul 13, 2017
17 checks passed
17 checks passed
docs Build #17801 ended
Details
flake8 Build #17914 ended
Details
isort Build #17929 succeeded in 30 sec
Details
pull-requests-javascript Build #14295 ended
Details
pull-requests-trusty/database=mysql,label=trusty-pr,python=python3.4 Build #13563 ended
Details
pull-requests-trusty/database=mysql,label=trusty-pr,python=python3.6 Build #13563 ended
Details
pull-requests-trusty/database=mysql_gis,label=trusty-pr,python=python3.4 Build #13563 ended
Details
pull-requests-trusty/database=mysql_gis,label=trusty-pr,python=python3.6 Build #13563 ended
Details
pull-requests-trusty/database=postgis,label=trusty-pr,python=python3.4 Build #13563 ended
Details
pull-requests-trusty/database=postgis,label=trusty-pr,python=python3.6 Build #13563 ended
Details
pull-requests-trusty/database=postgres,label=trusty-pr,python=python3.4 Build #13563 ended
Details
pull-requests-trusty/database=postgres,label=trusty-pr,python=python3.6 Build #13563 ended
Details
pull-requests-trusty/database=spatialite,label=trusty-pr,python=python3.4 Build #13563 ended
Details
pull-requests-trusty/database=spatialite,label=trusty-pr,python=python3.6 Build #13563 ended
Details
pull-requests-trusty/database=sqlite3,label=trusty-pr,python=python3.4 Build #13563 ended
Details
pull-requests-trusty/database=sqlite3,label=trusty-pr,python=python3.6 Build #13563 ended
Details
pull-requests-windows/database=sqlite3,label=windows,python=Python35 Build #9873 ended
Details
@timgraham

This comment has been minimized.

Copy link
Member

timgraham commented Aug 22, 2017

Mark, did you get a Trac notification about a possible regression?

@mrogaski

This comment has been minimized.

Copy link
Author

mrogaski commented Aug 22, 2017

Yep, I'll pick that issue up. It looks like an issue with the return value of get_system_encoding() on the surface, but I'll see if the posted patch is appropriate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.