Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Add documentation about python3 compatibility #89

Closed
wants to merge 1 commit into from

4 participants

Claude Paroz Alex Gaynor Keryn Knight Yuriy Taraday
Claude Paroz
Owner
claudep commented May 23, 2012

No description provided.

Alex Gaynor alex commented on the diff May 23, 2012
django/utils/py3.py
((37 lines not shown))
  37
+    long_type = long
  38
+
  39
+    from io import BytesIO as OutputIO
  40
+
  41
+    # Glue code for syntax differences
  42
+    def exec_(code, globs=None, locs=None):
  43
+        """Execute code in a namespace."""
  44
+        if globs is None:
  45
+            frame = sys._getframe(1)
  46
+            globs = frame.f_globals
  47
+            if locs is None:
  48
+                locs = frame.f_locals
  49
+            del frame
  50
+        elif locs is None:
  51
+            locs = globs
  52
+        exec("""exec code in globs, locs""")
4
Alex Gaynor Owner
alex added a note May 23, 2012

Is exec actually used anywhere besides in this file?

Claude Paroz Owner
claudep added a note May 24, 2012

AFAIK, it's only used for the reraise definition just below. You can find the exact same code in six: https://bitbucket.org/gutworth/six/src/175a03e21623/six.py#cl-294

Alex Gaynor Owner
alex added a note May 24, 2012

If we don't actually need exec for django why don't we simplify this implementation and give it some name so it won't be public (e.g. _exec).

Claude Paroz Owner
claudep added a note May 24, 2012

Feel free to suggest any improvement/simplification. I see now that Vinay used it once in his branch in _doctest.py. I have chosen another route for updating _doctest.py and propose to copy python 3.2 doctest.py in _doctest3.py, reapply Python #1521051 on it and import conditionally _doctest.py/_doctest3.py. It's difficult for me to evaluate if this function might be useful or not in Django apps ecosystem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Alex Gaynor alex commented on the diff May 23, 2012
docs/topics/python3.txt
((20 lines not shown))
  20
+Python 2 and Python 3, you can import it from ``django.utils.py3`` where it
  21
+will be automatically converted depending on your current Python version.
  22
+
  23
+PY3
  24
+---
  25
+
  26
+If you need to know anywhere in your code if you are running Python 3 or a
  27
+previous Python 2 version, you can check the ``PY3`` boolean variable::
  28
+
  29
+    from django.utils.py3 import PY3
  30
+
  31
+    if PY3:
  32
+        # Do stuff Python 3-wise
  33
+    else:
  34
+        # Do stuff Python 2-wise
  35
+
2
Alex Gaynor Owner
alex added a note May 23, 2012

This section should probably note that using the various variables in the py3 module is preferrable to branching on if PY3

Claude Paroz Owner
claudep added a note May 24, 2012

Good suggestion. Something like: "This should be considered as a last resort solution when it is not possible to import a compatible name from django.utils.py3."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Keryn Knight kezabelle commented on the diff May 24, 2012
docs/topics/python3.txt
((93 lines not shown))
  93
+
  94
+Should be replaced by::
  95
+
  96
+    from django.utils.py3 import string_types, text_type
  97
+
  98
+    if isinstance(foo, string_types):
  99
+        print("foo is a string")
  100
+
  101
+    # I want to convert a number to a Unicode string
  102
+    bar = 45
  103
+    bar_string = text_type(bar)
  104
+
  105
+No more long type
  106
+-----------------
  107
+``long`` and ``int`` types have been unified in Python 3, meaning that  ``long``
  108
+is no more available. ``django.utils.py3`` provides both ``long_type`` and
1
Keryn Knight
kezabelle added a note May 24, 2012

"no more available" doesn't read particularly well, it'd be better to say something like "no longer available", but I'm not enthused with that either because it seems accidentally punny.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Yuriy Taraday YorikSar commented on the diff May 29, 2012
django/utils/py3.py
((100 lines not shown))
  100
+    text_type = str
  101
+    integer_types = int,
  102
+    long_type = int
  103
+
  104
+    from io import StringIO as OutputIO
  105
+
  106
+    # Glue code for syntax differences
  107
+    def reraise(tp, value, tb=None):
  108
+        if value.__traceback__ is not tb:
  109
+            raise value.with_traceback(tb)
  110
+        raise value
  111
+
  112
+    exec_ = getattr(builtins, 'exec')
  113
+    def with_metaclass(meta, base=object):
  114
+        ns = dict(base=base, meta=meta)
  115
+        exec_("""class _DjangoBase(base, metaclass=meta):
1
Yuriy Taraday
YorikSar added a note May 29, 2012

Why use exec? You can just write contents of the string in the function.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Yuriy Taraday YorikSar commented on the diff May 29, 2012
django/utils/py3.py
((40 lines not shown))
  40
+
  41
+    # Glue code for syntax differences
  42
+    def exec_(code, globs=None, locs=None):
  43
+        """Execute code in a namespace."""
  44
+        if globs is None:
  45
+            frame = sys._getframe(1)
  46
+            globs = frame.f_globals
  47
+            if locs is None:
  48
+                locs = frame.f_locals
  49
+            del frame
  50
+        elif locs is None:
  51
+            locs = globs
  52
+        exec("""exec code in globs, locs""")
  53
+    exec_("""def reraise(tp, value, tb=None):
  54
+    raise tp, value, tb
  55
+""")
1
Yuriy Taraday
YorikSar added a note May 29, 2012

Once again, exec is not needed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Claude Paroz
Owner

Pushed in 5e6ded2.
Thanks for the review. I'm not sure if I included all what you meant in the review comments, feel free to open tickets to fix unsolved issues or to add further improvements.

Claude Paroz claudep closed this June 07, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 1 unique commit by 1 author.

May 23, 2012
Claude Paroz Add documentation about python3 compatibility af830cd
This page is out of date. Refresh to see the latest.
123  django/utils/py3.py
... ...
@@ -0,0 +1,123 @@
  1
+# Compatibility layer for running Django both in 2.x and 3.x
  2
+
  3
+import sys
  4
+
  5
+if sys.version_info[0] < 3:
  6
+    PY3 = False
  7
+    # Changed module locations
  8
+    from urlparse import (urlparse, urlunparse, urljoin, urlsplit, urlunsplit,
  9
+                          urldefrag, parse_qsl)
  10
+    from urllib import (quote, unquote, quote_plus, urlopen, urlencode,
  11
+                        url2pathname, urlretrieve, unquote_plus)
  12
+    from urllib2 import (Request, OpenerDirector, UnknownHandler, HTTPHandler,
  13
+                         HTTPSHandler, HTTPDefaultErrorHandler, FTPHandler,
  14
+                         HTTPError, HTTPErrorProcessor)
  15
+    import urllib2
  16
+    import Cookie as cookies
  17
+    try:
  18
+        import cPickle as pickle
  19
+    except ImportError:
  20
+        import pickle
  21
+    try:
  22
+        import thread
  23
+    except ImportError:
  24
+        import dummy_thread as thread
  25
+    from htmlentitydefs import name2codepoint
  26
+    import HTMLParser
  27
+    from os import getcwdu
  28
+    from itertools import izip as zip
  29
+    unichr = unichr
  30
+    xrange = xrange
  31
+    maxsize = sys.maxint
  32
+
  33
+    # Type aliases
  34
+    string_types = basestring,
  35
+    text_type = unicode
  36
+    integer_types = int, long
  37
+    long_type = long
  38
+
  39
+    from io import BytesIO as OutputIO
  40
+
  41
+    # Glue code for syntax differences
  42
+    def exec_(code, globs=None, locs=None):
  43
+        """Execute code in a namespace."""
  44
+        if globs is None:
  45
+            frame = sys._getframe(1)
  46
+            globs = frame.f_globals
  47
+            if locs is None:
  48
+                locs = frame.f_locals
  49
+            del frame
  50
+        elif locs is None:
  51
+            locs = globs
  52
+        exec("""exec code in globs, locs""")
  53
+    exec_("""def reraise(tp, value, tb=None):
  54
+    raise tp, value, tb
  55
+""")
  56
+
  57
+    def with_metaclass(meta, base=object):
  58
+        class _DjangoBase(base):
  59
+            __metaclass__ = meta
  60
+        return _DjangoBase
  61
+
  62
+    iteritems = lambda o: o.iteritems()
  63
+    itervalues = lambda o: o.itervalues()
  64
+    iterkeys = lambda o: o.iterkeys()
  65
+
  66
+    # n() is useful when python3 needs a str (unicode), and python2 str (bytes)
  67
+    def n(s, encoding='utf-8'):
  68
+        return s.encode(encoding)
  69
+
  70
+else:
  71
+    PY3 = True
  72
+    import builtins
  73
+
  74
+    # Changed module locations
  75
+    from urllib.parse import (urlparse, urlunparse, urlencode, urljoin,
  76
+                              urlsplit, urlunsplit, quote, unquote,
  77
+                              quote_plus, unquote_plus, parse_qsl,
  78
+                              urldefrag)
  79
+    from urllib.request import (urlopen, url2pathname, Request, OpenerDirector,
  80
+                                UnknownHandler, HTTPHandler, HTTPSHandler,
  81
+                                HTTPDefaultErrorHandler, FTPHandler,
  82
+                                HTTPError, HTTPErrorProcessor, urlretrieve)
  83
+    import urllib.request as urllib2
  84
+    import http.cookies as cookies
  85
+    import pickle
  86
+    try:
  87
+        import _thread as thread
  88
+    except ImportError:
  89
+        import _dummy_thread as thread
  90
+    from html.entities import name2codepoint
  91
+    import html.parser as HTMLParser
  92
+    from os import getcwd as getcwdu
  93
+    zip = zip
  94
+    unichr = chr
  95
+    xrange = range
  96
+    maxsize = sys.maxsize
  97
+
  98
+    # Type aliases
  99
+    string_types = str,
  100
+    text_type = str
  101
+    integer_types = int,
  102
+    long_type = int
  103
+
  104
+    from io import StringIO as OutputIO
  105
+
  106
+    # Glue code for syntax differences
  107
+    def reraise(tp, value, tb=None):
  108
+        if value.__traceback__ is not tb:
  109
+            raise value.with_traceback(tb)
  110
+        raise value
  111
+
  112
+    exec_ = getattr(builtins, 'exec')
  113
+    def with_metaclass(meta, base=object):
  114
+        ns = dict(base=base, meta=meta)
  115
+        exec_("""class _DjangoBase(base, metaclass=meta):
  116
+    pass""", ns)
  117
+        return ns["_DjangoBase"]
  118
+
  119
+    iteritems = lambda o: o.items()
  120
+    itervalues = lambda o: o.values()
  121
+    iterkeys = lambda o: o.keys()
  122
+
  123
+    n = lambda s: s
1  docs/index.txt
@@ -183,6 +183,7 @@ Other batteries included
183 183
 * :doc:`Logging <topics/logging>`
184 184
 * :doc:`Messages <ref/contrib/messages>`
185 185
 * :doc:`Pagination <topics/pagination>`
  186
+* :doc:`Python 3 compatibility <topics/python3>`
186 187
 * :doc:`Redirects <ref/contrib/redirects>`
187 188
 * :doc:`Security <topics/security>`
188 189
 * :doc:`Serialization <topics/serialization>`
1  docs/topics/index.txt
@@ -22,6 +22,7 @@ Introductions to all the key parts of Django you'll need to know:
22 22
    i18n/index
23 23
    logging
24 24
    pagination
  25
+   python3
25 26
    security
26 27
    serialization
27 28
    settings
247  docs/topics/python3.txt
... ...
@@ -0,0 +1,247 @@
  1
+======================
  2
+Python 3 compatibility
  3
+======================
  4
+
  5
+Django 1.5 introduces a compatibility layer that allows the code to be run both
  6
+in Python 2 (2.6/2.7) and Python 3 (>= 3.2) (*work in progress*).
  7
+
  8
+This document is not meant as a complete Python 2 to Python 3 migration guide.
  9
+There are many existing resources you can read. But we describe some utilities
  10
+and guidelines that we recommend you should use when you want to ensure your
  11
+code can be run with both Python 2 and 3.
  12
+
  13
+* http://docs.python.org/py3k/howto/pyporting.html
  14
+* http://python3porting.com/
  15
+
  16
+django.utils.py3
  17
+================
  18
+
  19
+Whenever a symbol or module has different semantics or different locations on
  20
+Python 2 and Python 3, you can import it from ``django.utils.py3`` where it
  21
+will be automatically converted depending on your current Python version.
  22
+
  23
+PY3
  24
+---
  25
+
  26
+If you need to know anywhere in your code if you are running Python 3 or a
  27
+previous Python 2 version, you can check the ``PY3`` boolean variable::
  28
+
  29
+    from django.utils.py3 import PY3
  30
+
  31
+    if PY3:
  32
+        # Do stuff Python 3-wise
  33
+    else:
  34
+        # Do stuff Python 2-wise
  35
+
  36
+String handling
  37
+===============
  38
+
  39
+In Python 3, all strings are considered Unicode strings by default. Byte strings
  40
+have to be prefixed with the letter 'b'. To mimic the same behaviour in Python 2,
  41
+we recommend you import ``unicode_literals`` from the ``__future__`` library::
  42
+
  43
+    from __future__ import unicode_literals
  44
+
  45
+    my_string = "This is an unicode literal"
  46
+    my_bytestring = b"This is a bytestring"
  47
+
  48
+Be cautious if you have to slice bytestrings.
  49
+See http://docs.python.org/py3k/howto/pyporting.html#bytes-literals
  50
+
  51
+Different expected strings
  52
+--------------------------
  53
+
  54
+Some method parameters have changed the expected string type of a parameter.
  55
+For example, ``strftime`` format parameter expects a bytestring on Python 2 but
  56
+a normal (Unicode) string on Python 3. For these cases, ``django.utils.py3``
  57
+provides a ``n()`` function which encodes the string parameter only with
  58
+Python 2.
  59
+
  60
+    >>> from __future__ import unicode_literals
  61
+    >>> from datetime import datetime
  62
+
  63
+    >>> print(datetime.date(2012, 5, 21).strftime(n("%m → %Y")))
  64
+    05 → 2012
  65
+
  66
+Renamed types
  67
+=============
  68
+
  69
+Several types are named differently in Python 2 and Python 3. In order to keep
  70
+compatibility while using those types, import their corresponding aliases from
  71
+``django.utils.py3``.
  72
+
  73
+===========  =========  =====================
  74
+Python 2     Python 3   django.utils.py3
  75
+===========  =========  =====================
  76
+basestring,  str,       string_types (tuple)
  77
+unicode      str        text_type
  78
+int, long    int,       integer_types (tuple)
  79
+long         int        long_type
  80
+===========  =========  =====================
  81
+
  82
+String aliases
  83
+--------------
  84
+
  85
+Code sample::
  86
+
  87
+    if isinstance(foo, basestring):
  88
+        print("foo is a string")
  89
+
  90
+    # I want to convert a number to a Unicode string
  91
+    bar = 45
  92
+    bar_string = unicode(bar)
  93
+
  94
+Should be replaced by::
  95
+
  96
+    from django.utils.py3 import string_types, text_type
  97
+
  98
+    if isinstance(foo, string_types):
  99
+        print("foo is a string")
  100
+
  101
+    # I want to convert a number to a Unicode string
  102
+    bar = 45
  103
+    bar_string = text_type(bar)
  104
+
  105
+No more long type
  106
+-----------------
  107
+``long`` and ``int`` types have been unified in Python 3, meaning that  ``long``
  108
+is no more available. ``django.utils.py3`` provides both ``long_type`` and
  109
+``integer_types`` aliases. For example:
  110
+
  111
+.. code-block:: python
  112
+
  113
+    # Old Python 2 code
  114
+    my_var = long(333463247234623)
  115
+    if isinstance(my_var, (int, long)):
  116
+        # ...
  117
+
  118
+Should be replaced by:
  119
+
  120
+.. code-block:: python
  121
+
  122
+    from django.utils.py3 import long_type, integer_types
  123
+
  124
+    my_var = long_type(333463247234623)
  125
+    if isinstance(my_var, integer_types):
  126
+        # ...
  127
+
  128
+
  129
+Changed module locations
  130
+========================
  131
+
  132
+The following modules have changed their location in Python 3. Therefore, it is
  133
+recommended to import them from the ``django.utils.py3`` compatibility layer:
  134
+
  135
+=============================== ======================================  ======================
  136
+Python 2                        Python3                                 django.utils.py3
  137
+=============================== ======================================  ======================
  138
+Cookie                          http.cookies                            cookies
  139
+
  140
+urlparse.urlparse               urllib.parse.urlparse                   urlparse
  141
+urlparse.urlunparse             urllib.parse.urlunparse                 urlunparse
  142
+urlparse.urljoin                urllib.parse.urljoin                    urljoin
  143
+urlparse.urlsplit               urllib.parse.urlsplit                   urlsplit
  144
+urlparse.urlunsplit             urllib.parse.urlunsplit                 urlunsplit
  145
+urlparse.urldefrag              urllib.parse.urldefrag                  urldefrag
  146
+urlparse.parse_qsl              urllib.parse.parse_qsl                  parse_qsl
  147
+urllib.quote                    urllib.parse.quote                      quote
  148
+urllib.unquote                  urllib.parse.unquote                    unquote
  149
+urllib.quote_plus               urllib.parse.quote_plus                 quote_plus
  150
+urllib.unquote_plus             urllib.parse.unquote_plus               unquote_plus
  151
+urllib.urlencode                urllib.parse.urlencode                  urlencode
  152
+urllib.urlopen                  urllib.request.urlopen                  urlopen
  153
+urllib.url2pathname             urllib.request.url2pathname             url2pathname
  154
+urllib.urlretrieve              urllib.request.urlretrieve              urlretrieve
  155
+urllib2                         urllib.request                          urllib2
  156
+urllib2.Request                 urllib.request.Request                  Request
  157
+urllib2.OpenerDirector          urllib.request.OpenerDirector           OpenerDirector
  158
+urllib2.UnknownHandler          urllib.request.UnknownHandler           UnknownHandler
  159
+urllib2.HTTPHandler             urllib.request.HTTPHandler              HTTPHandler
  160
+urllib2.HTTPSHandler            urllib.request.HTTPSHandler             HTTPSHandler
  161
+urllib2.HTTPDefaultErrorHandler urllib.request.HTTPDefaultErrorHandler  HTTPDefaultErrorHandler
  162
+urllib2.FTPHandler              urllib.request.FTPHandler               FTPHandler
  163
+urllib2.HTTPError               urllib.request.HTTPError                HTTPError
  164
+urllib2.HTTPErrorProcessor      urllib.request.HTTPErrorProcessor       HTTPErrorProcessor
  165
+
  166
+htmlentitydefs.name2codepoint   html.entities.name2codepoint            name2codepoint
  167
+HTMLParser                      html.parser                             HTMLParser
  168
+cPickle/pickle                  pickle                                  pickle
  169
+thread/dummy_thread             _thread/_dummy_thread                   thread
  170
+
  171
+os.getcwdu                      os.getcwd                               getcwdu
  172
+itertools.izip                  zip                                     zip
  173
+sys.maxint                      sys.maxsize                             maxsize
  174
+unichr                          chr                                     unichr
  175
+xrange                          range                                   xrange
  176
+=============================== ======================================  ======================
  177
+
  178
+
  179
+Ouptut encoding now Unicode
  180
+===========================
  181
+
  182
+If you want to catch stdout/stderr output, the output content is UTF-8 encoded
  183
+in Python 2, while it is Unicode strings in Python 3. You can use the OutputIO
  184
+stream to capture this output::
  185
+
  186
+    from django.utils.py3 import OutputIO
  187
+
  188
+    try:
  189
+        old_stdout = sys.stdout
  190
+        out = OutputIO()
  191
+        sys.stdout = out
  192
+        # Do stuff which produces standard output
  193
+        result = out.getvalue()
  194
+    finally:
  195
+        sys.stdout = old_stdout
  196
+
  197
+Dict iteritems/itervalues/iterkeys
  198
+==================================
  199
+
  200
+The iteritems(), itervalues() and iterkeys() methods of dictionaries do not
  201
+exist any more in Python 3, simply because they represent the default items()
  202
+values() and keys() behavior in Python 3. Therefore, to keep compatibility,
  203
+use similar functions from ``django.utils.py3``::
  204
+
  205
+    from django.utils.py3 import iteritems, itervalues, iterkeys
  206
+
  207
+    my_dict = {'a': 21, 'b': 42}
  208
+    for key, value in iteritems(my_dict):
  209
+        # ...
  210
+    for value in itervalues(my_dict):
  211
+        # ...
  212
+    for key in iterkeys(my_dict):
  213
+        # ...
  214
+
  215
+Note that in Python 3, dict.keys(), dict.items() and dict.values() return
  216
+"views" instead of lists. Wrap them into list() if you really need their return
  217
+values to be in a list.
  218
+
  219
+http://docs.python.org/release/3.0.1/whatsnew/3.0.html#views-and-iterators-instead-of-lists
  220
+
  221
+Metaclass
  222
+=========
  223
+
  224
+The syntax for declaring metaclasses has changed in Python 3.
  225
+``django.utils.py3`` offers a compatible way to declare metaclasses::
  226
+
  227
+    from django.utils.py3 import with_metaclass
  228
+
  229
+    class MyClass(with_metaclass(SubClass1, SubClass2,...)):
  230
+        # ...
  231
+
  232
+Re-raising exceptions
  233
+=====================
  234
+
  235
+One of the syntaxes to raise exceptions (raise E, V, T) is gone in Python 3.
  236
+This is especially used in very specific cases where you want to re-raise a
  237
+different exception that the initial one, while keeping the original traceback.
  238
+So, instead of::
  239
+
  240
+    raise Exception, Exception(msg), traceback
  241
+
  242
+Use::
  243
+
  244
+    from django.utils.py3 import reraise
  245
+
  246
+    reraise(Exception, Exception(msg), traceback)
  247
+
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.