Fix DTML upload for Python 3.

[icemac]

Without this change uploading a file to a DTMLMethod via ZMI breaks because self.munge(file) tries to use a string regex on a bytes object (the file variable).

What do you think is this patch the way to go?

Open tasks:

• Discuss solution
• Write test (method is currently untested)
• Port fix to DTMLDocument (it duplicates the code)
• Write test for DTMLDocument

See zopefoundation/Products.SiteErrorLog#11 which uses this method in tests and fails with AttributeError: 'str' object has no attribute 'read'. in OFS/DTMLMethod.py:272

[davisagli]

basestring is not defined in Python 3

[icemac]

You are right in general, but it is defined inside the module:

Zope/src/OFS/DTMLMethod.py

Lines 45 to 46 in bc180d5

	if sys.version_info >= (3, ):
	basestring = str

[rbu]

That if sys.version_info could use the PY3 symbol imported above

[icemac]

Although it is not completely the same it is close enough, as we will drop Python 2 support before Python 4 comes out. :)

[rbu]

Shouldn't the conversion binary_type->text_type happen for Python 2 as well? I would assume the file type is always binary_type (in which case the method's default should also be changed to file=b'').

Plus, if the desired output type is text_type, then you might also want to decode inputs that are non-file-like (such as file=b'fnord').

[icemac]

@rbu I think you are generally right. But there are databases out in the world which have stored the file contents as bytes (aka Python 2 str). Can we provide a way to convert their file contents to text so we do not have to live forever with file contents wich might be bytes?

[rbu]

Ok I see what you're saying. In the old code, Python 2 would always have str byte strings and byte file objects. Those were read, resulting in bytes being saved to ZODB. As I understand it, per the WSGI spec, file will usually be a bytes file object, so read()ing from it needs to be decoded. So the goal of this code is to always save byte strings in PY2 and always save text strings in PY3.

To make that consistent, though, the default method parameter needs to be decoded as well. What do you think of this:

        if not isinstance(file, basestring):
            if REQUEST and not file:
                raise ValueError('No file specified')
            file = file.read()

        if PY3 and isinstance(file, binary_type):
                file = file.decode('utf-8')

[icemac]

You are right if manage_upload is called with a file parameter which is bytes it should be decoded, too.

[rbu]

FYI: I've rebased this on top of master

[icemac]

I changed the behaviour of manage_upload to store always a native str. (This seems to be the same what manage_edit does.) Does this sound reasonable? If yes I'll implement the same for DTMLDocument.

[leorochael]

@icemac it sounds reasonable that both manage_upload and manage_edit store the same thing.

[rbu]

In the current state, you don't need the basestring symbol anymore.

[icemac]

Good catch.

[rbu]

Overall, this looks good to me