Add encoding utility

[benjaminp]

Originally reported by: Claude Paroz (Bitbucket: claudep, GitHub: claudep)

---

Hi,

We're planning to embed six in Django as the blessed Python 3 compatibility layer. One thing we're missing is a function that returns a byte string on Python 2 and Unicode in Python 3. This is typically useful when providing arguments for strftime() or type(). Our current implementation is:

• Python 2: n = lambda n: n.encode('utf-8')
• Python 3 (noop): n = lambda n: n

We could of course provide it in some of our own modules, but it would be nice to have it in six, unless you suggest a better replacement.

---

• Bitbucket: https://bitbucket.org/gutworth/six/issue/12

[benjaminp]

That seems reasonable. What were you planning on calling it?

[benjaminp]

Original comment by Claude Paroz (Bitbucket: claudep, GitHub: claudep):

---

Currently it's n(). It may be the choice of Vinay Sajip (should ask him). Maybe to echo the b() and u() functions? Frankly, I've no strong opinion on the name.

[benjaminp]

Does this need to support all of unicode? It doesn't seem that blindly encoding something in utf-8 is what you'd always want. Would encoding with just ascii be sufficient? Also, it does extra work in Python 2 if you pass a str.

[benjaminp]

Original comment by Claude Paroz (Bitbucket: claudep, GitHub: claudep):

---

I think that the use case for Python 2 is clearly to take a Unicode string and transform it into a utf-8 byte string. Using the 'ascii' encoding would lead to UnicodeEncodeError in many cases. Surely, testing if it is already a byte string could be a nice addition (Python 2, again).

[benjaminp]

Original comment by Claude Paroz (Bitbucket: claudep, GitHub: claudep):

---

Attached is a patch proposal for this new n() function.

[benjaminp]

I'm wondering if using utf-8 is always the correct thing to do. For example, with type(), non-ascii identifiers should not be used in Python 2, since it doesn't support them. time.strftime will always return a byte-string thus leaking the utf-8 into the return value.

[benjaminp]

Original comment by Aymeric Augustin (Bitbucket: aaugustin, GitHub: aaugustin):

---

We revisited significant parts of Vinay Sajip's experimental port since opening this issue, and we probably aren't going to need this function.

When Python 2 requires a byte string and Python 3 a unicode string, Django uses:

• str when the contents is guaranteed to be an ASCII unicode string eg. when it's a literal
• smart_str (from django.utils.encoding) otherwise

[benjaminp]

Original comment by Aymeric Augustin (Bitbucket: aaugustin, GitHub: aaugustin):

---

We revisited significant parts of Vinay Sajip's experimental port since opening this issue, and we probably aren't going to need this function.

When Python 2 requires a byte string and Python 3 a unicode string, Django uses:

• str when the contents is guaranteed to be an ASCII unicode string eg. when it's a literal
• smart_str (from django.utils.encoding) otherwise

[benjaminp]

Original comment by Aymeric Augustin (Bitbucket: aaugustin, GitHub: aaugustin):

---

This issue was opened on behalf of the Django core team.

The discussion above didn't mention any other use case.

Since Django finally isn't going to need this addition, I'm closing this ticket.