New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'str' does not support the buffer interface #216

Closed
jonfen opened this Issue Nov 14, 2018 · 2 comments

Comments

2 participants
@jonfen
Copy link

jonfen commented Nov 14, 2018

The uploads portion of the following example code fails with the error above for both create() and save(), removing the uploads assignment makes the call successful, but not valuable:

https://python-redmine.com/resources/issue.html?highlight=stringio

from io import StringIO
issue = redmine.issue.create(
     project_id='vacation',
     subject='Vacation',
     tracker_id=8,
     description='foo',
     status_id=3,
     priority_id=7,
     assigned_to_id=123,
     watcher_user_ids=[123],
     parent_issue_id=345,
     start_date=datetime.date(2014, 1, 1),
     due_date=datetime.date(2014, 2, 1),
     estimated_hours=4,
     done_ratio=40,
     custom_fields=[{'id': 1, 'value': 'foo'}, {'id': 2, 'value': 'bar'}],
     uploads=[{'path': '/absolute/path/to/file'}, {'path': StringIO('I am content of file 2')}]
)

produces error: 'str' does not support the buffer interface

Redmine Server: 3.2.0
Python: 3.3
OS: Windows 10 Pro

@maxtepkeev maxtepkeev self-assigned this Nov 24, 2018

@maxtepkeev maxtepkeev added the bug label Nov 24, 2018

@jonfen

This comment has been minimized.

Copy link

jonfen commented Jan 2, 2019

file_contents = 'I am content of file 2'
uploads=[{'path': '/absolute/path/to/file'}, {'path': BytesIO(file_contents.encode('utf-8'))}]

After upgrading to Python 3.7 StringIO supports ByteIO, but it did not in 3.3. The code above would be more accurate.

@jonfen jonfen closed this Jan 2, 2019

maxtepkeev added a commit that referenced this issue Jan 2, 2019

@maxtepkeev

This comment has been minimized.

Copy link
Owner

maxtepkeev commented Jan 2, 2019

@jonfen You're absolutely right that one should use BytesIO under Python 3. I was also investigating this issue for the last two days, since I finally found some time to do this and here are my findings:

  • This error comes from the standard library httplib (which is mocked in tests, that's why we didn't find it earlier) when it tries to send the data over the socket as it expects to send bytes, but receives a unicode object, httplib is used internally by urllib3, which is used internally by requests which is what python-redmine uses to do all the network calls
  • Using StringIO or any other file-like object under Python 2 works most of the time unless you specifically say that you want to put a unicode string inside (using a u'' prefix, unicode function or whatever)
  • Under Python 3 things get more complicated since all strings are unicode by default
  • Starting from Python 3.6 things will start working magically (exactly what you've experienced after upgrading to 3.7), because they're trying to do the conversion under the hood, but it works only if you have only ascii chars in your strings, add something else, say a char from Russian language and everything will fail again

So basically what we need to do is to convert our unicode strings to bytes before feeding a file-like object to Python-Redmine which is exactly what I'm doing now in c8b49bd. Since it's a clients application failure and it requires Python-Redmine to do some unnecessary things, I'm emitting a PerformanceWarning which explains the issue and asks a client application to fix it, I've also changed the docs to use BytesIO instead of StringIO.

Thank you very much for rasing this issue as it helped to find this nasty bug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment