bulk2 error inserting csv file utf-8 coded

[dkitt]

UTF-8 Coding Problem

Example file from Developers Guide: Walkthrough for Bulk Insert

When following example using requests manually all works okay. But when we try to load the file using simple-salesforce library (its bulk2 module) we got an error.

Inserting the file using library:

from simple_salesforce import Salesforce
sf = Salesforce(instance_url='****', ...)
sf.bulk2.Account.insert("./bulkinsert.csv")

You get an error 405:
'HTTP Method \'"PATCH\' not allowed. Allowed are HEAD,GET,PUT,PATCH,POST,DELETE'
The system complains about a verb "PATCH which is not PATCH.

Source of error

The file mentioned there has a word Bodø in it. The character ø is coded in 2 bytes. Library reads in the CSV file as a string. It then uploads the string using PUT request. Then it tries to conclude the upload by PATCH request but the server does not get PATCH but "PATCH as an http verb.

Root cause of the error

The simple-salesforce library is using post() of requests library but wrongly assigns string as data parameter as in:

text = 'Bodø'
s = requests.Session()
r = s.post('******', data=text)

It must not assign a string but an array of bytes to data= parameter!
The requests library later assigns "Content-Length" header based on len() function.

len(request.body) == 405
len(request.body.encode()) == 406

Consequently it sends more bytes than announced in "Content-Length" header and receiver (server) gets corrupted http word.