ElementTree.write() raises TypeError when xml_declaration = True and encoding is a unicode string

[DavidBuxton]

BPO	15811
Nosy	@amauryfa, @florentx
Files	xml_declaration_test.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2012-10-15.12:55:57.132>
created_at = <Date 2012-08-29.13:18:39.556>
labels = ['expert-XML', 'type-bug', 'library']
title = 'ElementTree.write() raises TypeError when xml_declaration = True and encoding is a unicode string'
updated_at = <Date 2012-10-15.12:55:57.130>
user = 'https://bugs.python.org/DavidBuxton'

bugs.python.org fields:

activity = <Date 2012-10-15.12:55:57.130>
actor = 'eli.bendersky'
assignee = 'none'
closed = True
closed_date = <Date 2012-10-15.12:55:57.132>
closer = 'eli.bendersky'
components = ['Library (Lib)', 'XML']
creation = <Date 2012-08-29.13:18:39.556>
creator = 'David.Buxton'
dependencies = []
files = ['27046']
hgrepos = []
issue_num = 15811
keywords = ['patch']
message_count = 6.0
messages = ['169373', '169391', '169392', '169395', '169396', '172966']
nosy_count = 4.0
nosy_names = ['amaury.forgeotdarc', 'eli.bendersky', 'flox', 'David.Buxton']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue15811'
versions = ['Python 2.7']

[DavidBuxton]

The problem is an inconsistency between the ElementTree.write() method on Python 2 and 3 when xml_declaration is True. For Python 2.7 the encoding argument MUST NOT be a unicode string. For Python 3.2 the encoding argument MUST be a unicode string.

On Python 2.7.3 (ElementTree 1.3.0) you can only use byte strings as the encoding argument when including the xml declaration. If you use a unicode object you get TypeError thrown:

    >>> from xml.etree import ElementTree as ET
    >>> from io import BytesIO
    >>> 
    >>> tree = ET.ElementTree(ET.Element(u'example'))
    >>> tree.write(BytesIO(), xml_declaration=True, encoding='utf-8')
    >>> tree.write(BytesIO(), xml_declaration=True, encoding=u'utf-8')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 813, in write
        write("<?xml version='1.0' encoding='%s'?>\n" % encoding)
    TypeError: 'unicode' does not have the buffer interface

So the encoding argument must be a byte string.

However on Python 3.2.3 (ElementTree 1.3.0) the same argument must be a unicode string. If you pass a byte string in it raises TypeError.

This only happens when you pass in an encoding and xml_declaration=True. This is a (small) problem when writing Py 2/3 compatible code since the version of ElementTree is supposed to be the same.

[DavidBuxton]

A patch against the current default branch to add tests for the xml_declaration keyword argument. This passes when applied to the 2.7 branch too.

This does NOT test whether one can use both bytes/unicode for the encoding argument. It just uses the native string type for each interpreter. I don't know whether you would actually consider that part of things worth testing yet.

[elibendersky]

Thanks for the patch, David.

Alas, due to personal reasons I will not be able to work on core Python in the next 2-3 months. I may throw in a review here and there, but that's not a promise ;-)

[amauryfa]

Why is it a problem? Encoding must be a text string:
encoding='utf-8'
works with both Python 2 and 3.

Or if you used "from __future__ import unicode_literals",
str('utf-8')
works as well.

[DavidBuxton]

Only a problem because I am using unicode_literals and it didn't occur to me to use str('utf-8') to get a native string on both 2+3. Much the best solution, thank you.

But that is still a little smelly - I think what I want ideally is for ElementTree to accept str or unicode on 2.7. Either way I appreciate this is very low priority and indeed debatable as to whether it is "wrong".

[elibendersky]

Finally found time to look at this, sorry for the delay.

I agree with Amaury, and don't think any change is necessary in 2.7, the behavior there is quite consistent with what was usually done in 2.x. For porting or keeping the code 2/3 compatible, Amaury's solution is appropriate.