Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ConnectionResetError: [WinError 10054] upon uploading large file #42

Closed
diman82 opened this issue Jul 28, 2016 · 18 comments
Closed

ConnectionResetError: [WinError 10054] upon uploading large file #42

diman82 opened this issue Jul 28, 2016 · 18 comments

Comments

@diman82
Copy link

diman82 commented Jul 28, 2016

I'm trying to upload a 800mb file, and I get the following error message:

c:\Python36\Scripts>python onedrive.py
https://login.live.com/oauth20_authorize.srf?response_type=code&scope=wl.signin+wl.offline_access+onedrive.readwrite&client_id=0000000044194611&redirect_uri=http%3A%2F%2Flocalhost%2F
127.0.0.1 - - [29/Jul/2016 00:59:10] "GET /?code=<my_code>
 HTTP/1.1" 200 -
<class 'str'>
Uploading item/s..
Traceback (most recent call last):
  File "c:\python36\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 578, in urlopen
    chunked=chunked)
  File "c:\python36\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 362, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "c:\python36\lib\http\client.py", line 1155, in request
    self._send_request(method, url, body, headers)
  File "c:\python36\lib\http\client.py", line 1200, in _send_request
    self.endheaders(body)
  File "c:\python36\lib\http\client.py", line 1151, in endheaders
    self._send_output(message_body)
  File "c:\python36\lib\http\client.py", line 985, in _send_output
    self.send(message_body)
  File "c:\python36\lib\http\client.py", line 954, in send
    self.sock.sendall(datablock)
  File "c:\python36\lib\ssl.py", line 887, in sendall
    v = self.send(data[count:])
  File "c:\python36\lib\ssl.py", line 857, in send
    return self._sslobj.write(data)
  File "c:\python36\lib\ssl.py", line 582, in write
    return self._sslobj.write(data)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\python36\lib\site-packages\requests\adapters.py", line 403, in send
    timeout=timeout
  File "c:\python36\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 623, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "c:\python36\lib\site-packages\requests\packages\urllib3\util\retry.py", line 255, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "c:\python36\lib\site-packages\requests\packages\urllib3\packages\six.py", line 309, in reraise
    raise value.with_traceback(tb)
  File "c:\python36\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 578, in urlopen
    chunked=chunked)
  File "c:\python36\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 362, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "c:\python36\lib\http\client.py", line 1155, in request
    self._send_request(method, url, body, headers)
  File "c:\python36\lib\http\client.py", line 1200, in _send_request
    self.endheaders(body)
  File "c:\python36\lib\http\client.py", line 1151, in endheaders
    self._send_output(message_body)
  File "c:\python36\lib\http\client.py", line 985, in _send_output
    self.send(message_body)
  File "c:\python36\lib\http\client.py", line 954, in send
    self.sock.sendall(datablock)
  File "c:\python36\lib\ssl.py", line 887, in sendall
    v = self.send(data[count:])
  File "c:\python36\lib\ssl.py", line 857, in send
    return self._sslobj.write(data)
  File "c:\python36\lib\ssl.py", line 582, in write
    return self._sslobj.write(data)
requests.packages.urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "onedrive.py", line 23, in <module>
    client.item(drive="me", id="8930240577EB1134%2166557").children["Agron_reshumon.rar"].upload("//dd-wrt/usb/Upload/Agron_reshumon.rar")
  File "c:\python36\lib\site-packages\onedrivesdk\request\item_request_builder.py", line 134, in upload
    return self.content.request().upload(local_path)
  File "c:\python36\lib\site-packages\onedrivesdk\request\item_content_request.py", line 59, in upload
    entity_response = self.send(path=content_local_path)
  File "c:\python36\lib\site-packages\onedrivesdk\request_base.py", line 145, in send
    path=path)
  File "c:\python36\lib\site-packages\onedrivesdk\http_provider.py", line 61, in send
    response = session.send(prepped)
  File "c:\python36\lib\site-packages\requests\sessions.py", line 585, in send
    r = adapter.send(request, **kwargs)
  File "c:\python36\lib\site-packages\requests\adapters.py", line 453, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

I guess this error is probably due to a limit of the uploaded file size (100mb)?
If so, it would be nice if the SDK checks the file size prior to uploading, and breaks it to chunks, or notifies, or this feature is already implemented?

@diman82
Copy link
Author

diman82 commented Jul 28, 2016

#14 Just found this, though a practical usage example would be welcomed.

@diman82 diman82 closed this as completed Jul 28, 2016
@cdmayer
Copy link
Contributor

cdmayer commented Jul 28, 2016

Use upload_aysnc and your problems will be solved.

@diman82
Copy link
Author

diman82 commented Jul 28, 2016

I get 'invalidRequest - API not found' error:

Uploading item/s..
Traceback (most recent call last):
  File "onedrive.py", line 25, in <module>
    client.item(drive="me", id="8930240577EB1134%2166557").children["RoyalTSInstaller.msi"].upload_async("./RoyalTSInstaller.msi")
  File "c:\python36\lib\site-packages\onedrivesdk\version_bridge\fragment_upload.py", line 157, in fragment_upload_async
    session = self.create_session(item).post()
  File "c:\python36\lib\site-packages\onedrivesdk\request\item_create_session.py", line 106, in post
    return self.request().post()
  File "c:\python36\lib\site-packages\onedrivesdk\request\item_create_session.py", line 60, in post
    entity = UploadSession(json.loads(self.send(self.body_options).content))
  File "c:\python36\lib\site-packages\onedrivesdk\request_base.py", line 163, in send
    content=content_dict)
  File "c:\python36\lib\site-packages\onedrivesdk\http_provider.py", line 71, in send
    custom_response = HttpResponse(response.status_code, response.headers, response.text)
  File "c:\python36\lib\site-packages\onedrivesdk\http_response.py", line 49, in __init__
    raise OneDriveError(message["error"], self.status)
onedrivesdk.error.OneDriveError: invalidRequest - API not found

My code (after successful auth):

print('Uploading item/s..')
#Upload an item
#client.item(drive="me", id="8930240577EB1134%2166557").children["RoyalTSInstaller.msi"].upload("./RoyalTSInstaller.msi")
client.item(drive="me", id="8930240577EB1134%2166557").children["RoyalTSInstaller.msi"].upload_async("./RoyalTSInstaller.msi")

@diman82 diman82 reopened this Jul 28, 2016
@cdmayer
Copy link
Contributor

cdmayer commented Jul 28, 2016

Are you sure that the item ID you inserted is a folder? The error you're getting is usually from that.

@diman82
Copy link
Author

diman82 commented Jul 28, 2016

Absolutely:
image

Besides, if I've only changed from upload (which works fine) to upload_async, all the other details remained the same.

@cdmayer
Copy link
Contributor

cdmayer commented Jul 29, 2016

Do you have version 1.0.7 installed? The previous version of the SDK did not support files >100mb.

@diman82
Copy link
Author

diman82 commented Jul 29, 2016

Yes, I've just rechecked this. Reinstalled in any case:

Proceed (y/n)? y
  Successfully uninstalled onedrivesdk-1.0.7

c:\Python36\Scripts>
c:\Python36\Scripts>
c:\Python36\Scripts>pip install onedrivesdk
Collecting onedrivesdk
  Using cached onedrivesdk-1.0.7.zip
Requirement already satisfied (use --upgrade to upgrade): requests>=2.6.1 in c:\python36\lib\site-packages (from onedrivesdk)
Installing collected packages: onedrivesdk
  Running setup.py install for onedrivesdk ... done
Successfully installed onedrivesdk-1.0.7

Also tried running the script on another device, same error. Clueless.

@cdmayer
Copy link
Contributor

cdmayer commented Jul 29, 2016

I guess we need to debug. Can you add this line to HttpProvider.send() right before the return statement?

print(request.url, '\n'.join(['{}: {}'.format(k, v) for k, v in request.headers.items()]), '\n', request.method)

Run your request and send me back the output. You may want to remove the 'Authorization' header from your comment so no one sees your access token.

@diman82
Copy link
Author

diman82 commented Jul 30, 2016

I've added also the print of 'custom_response' object.

        custom_response = HttpResponse(response.status_code, response.headers, response.text)
        #debug!!
        print(request.url, '\n'.join(['{}: {}'.format(k, v) for k, v in request.headers.items()]), '\n', request.method)
        print(custom_response)
        return custom_response

NON-WORKING 'upload_async' (small file of 1MB):

c:\Python36\Scripts>python onedrive.py
https://login.live.com/oauth20_token.srf Content-Type: application/x-www-form-urlencoded
 POST
Content: {"token_type":"bearer","expires_in":3600,"scope":"wl.signin wl.offline_access onedrive.readwrite","access_token":"<MY_ACCESS_TOKEN>","refresh_token":"<MY_REFRESH_TOKEN>","user_id":"b4db8907776da42438d7d1900531a04a"}
Status: 200
Headers: {'Content-Length': '1593', 'Server': 'Microsoft-IIS/8.5', 'X-Content-Type-Options': 'nosniff', 'X-XSS-Protection': '1; mode=block', 'Date': 'Sat, 30 Jul 2016 11:31:52 GMT', 'Connection': 'close', 'Strict-Transport-Security': 'max-age=31536000', 'Cache-Control': 'no-store', 'Pragma': 'no-cache', 'Content-Type': 'application/json'}

Uploading item/s..

WORKING 'upload' (small file of 1MB):

c:\Python36\Scripts>python onedrive.py
https://login.live.com/oauth20_token.srf Content-Type: application/x-www-form-urlencoded
 POST
Headers: {'X-XSS-Protection': '1; mode=block', 'Strict-Transport-Security': 'max-age=31536000', 'Connection': 'close', 'Server': 'Microsoft-IIS/8.5', 'Pragma': 'no-cache', 'Content-Length': '1593', 'Cache-Control': 'no-store', 'Date': 'Sat, 30 Jul 2016 11:32:30 GMT', 'X-Content-Type-Options': 'nosniff', 'Content-Type': 'application/json'}
Status: 200
Content: {"token_type":"bearer","expires_in":3600,"scope":"wl.signin wl.offline_access onedrive.readwrite","access_token":"<MY_ACCESS_TOKEN>","refresh_token":"<MY_REFRESH_TOKEN>","user_id":"b4db8907776da42438d7d1900531a04a"}

Uploading item/s..
https://api.onedrive.com/v1.0/drives/me/items/8930240577EB1134%2166557/children/test3.mp3/content Authorization: bearer <MY_ACCESS_TOKEN>
X-RequestStats: SDK-Version=python-v1.0.7
 PUT
Headers: {'Location': "https://api.onedrive.com/v1.0/drives('me')/items('8930240577EB1134!66557')/children('8930240577EB1134!71279')", 'OData-Version': '4.0', 'Content-Location': 'https://idtpna.by3302.livefilestore.com/y3p18gYn9_LbCMakEVQ1_8xBqAShxxE1n_cbLV4GjRqPd9lm0U5gPDvttksKhFzNHpQwbfUbSM9_Mk0EEjgrNyf9PIcYih8zt4C3u0745JNqyWcnh27vzCnTE86UAwiC6Ko5xGN9I3rz1trjU4I7bCXx_8MSX19RR-L27OQCVttQXIyPa2htrFs9GBeY8WwX-yn', 'X-MSNSERVER': 'BY3302____PAP215', 'X-ItemVersion': '0', 'Content-Type': 'application/json; odata.metadata=minimal; odata.streaming=true', 'P3P': 'CP="BUS CUR CONo FIN IVDo ONL OUR PHY SAMo TELo"', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'Server': 'Microsoft-IIS/8.5', 'X-Resource-Id': '8930240577EB1134!71279', 'X-AsmVersion': 'UNKNOWN; 19.46.0.0', 'Date': 'Sat, 30 Jul 2016 11:32:36 GMT', 'X-Last-Modified-ISO8601': '2016-07-30T11:32:37.697Z', 'Content-Length': '1336', 'X-MSEdge-Ref': 'Ref A: A7C3F1D35DC54D83AE77D23BB0C5AA2C Ref B: D0D00CBFD779194CC54EFC434E8188F1 Ref C: Sat Jul 30 04:32:36 2016 PST', 'X-Cid': '-8561303285725982412', 'ETag': 'aODkzMDI0MDU3N0VCMTEzNCE3MTI3OS4w'}
Status: 201
Content: {"@odata.context":"https://api.onedrive.com/v1.0/$metadata#drives('me')/items('8930240577EB1134%2166557')/children/$entity","@content.downloadUrl":"https://idtpna.by3302.livefilestore.com/y3pYAUbbGd8tSuH5NoEAZe3tJA7Gluf3W9nRhhrGRA3dDVxxdw1BL3zIhWRVrrsIm2NALwP2N94_QXZRruef5Oyf6cxasHO-sjgLiRF0S7ajNorJtadxKxHSXLgObE8mB_DuWHWAG4_wMpqqNPSA8Eljm_Ypb_sWVSReKa6cSYUv-40trBZoB3AWspyv7O-F3n0kNPzr_BK5JOxbSOFOnsdtg","createdBy":{"application":{"displayName":"onedrive-sdk-python","id":"44194611"},"user":{"displayName":"Dmitri M","id":"8930240577eb1134"}},"createdDateTime":"2016-07-30T11:32:37.697Z","cTag":"aYzo4OTMwMjQwNTc3RUIxMTM0ITcxMjc5LjI1Nw","eTag":"aODkzMDI0MDU3N0VCMTEzNCE3MTI3OS4w","id":"8930240577EB1134!71279","lastModifiedBy":{"application":{"displayName":"onedrive-sdk-python","id":"44194611"},"user":{"displayName":"Dmitri M","id":"8930240577eb1134"}},"lastModifiedDateTime":"2016-07-30T11:32:37.697Z","name":"test3.mp3","parentReference":{"driveId":"8930240577eb1134","id":"8930240577EB1134!66557","path":"/drive/root:/Uploaded"},"size":1444238,"webUrl":"https://1drv.ms/u/s!ADQR63cFJDCJhKxv","file":{"hashes":{"crc32Hash":"0171BEB4","sha1Hash":"A736385202B739CA21F327B297ECAD6F0172BC5A"},"mimeType":"audio/mpeg"},"fileSystemInfo":{"createdDateTime":"2016-07-30T11:32:37.697Z","lastModifiedDateTime":"2016-07-30T11:32:37.697Z"}}

NON-WORKING 'upload_async' (big file of 130MB):

c:\Python36\Scripts>python onedrive.py
https://login.live.com/oauth20_token.srf Content-Type: application/x-www-form-urlencoded
 POST
Headers: {'Strict-Transport-Security': 'max-age=31536000', 'Connection': 'close', 'Pragma': 'no-cache', 'X-Content-Type-Options': 'nosniff', 'Content-Type': 'application/json', 'Content-Length': '1593', 'Date': 'Sat, 30 Jul 2016 11:34:48 GMT', 'X-XSS-Protection': '1; mode=block', 'Server': 'Microsoft-IIS/8.5', 'Cache-Control': 'no-store'}
Content: {"token_type":"bearer","expires_in":3600,"scope":"wl.signin wl.offline_access onedrive.readwrite","access_token":"<MY_ACCESS_TOKEN>","refresh_token":"<MY_REFRESH_TOKEN>","user_id":"b4db8907776da42438d7d1900531a04a"}
Status: 200

Uploading item/s..
Traceback (most recent call last):
  File "onedrive.py", line 34, in <module>
    client.item(drive="me", id="8930240577EB1134%2166557").children["RoyalTSInstaller.msi"].upload_async("./RoyalTSInstaller.msi")
  File "c:\python36\lib\site-packages\onedrivesdk\version_bridge\fragment_upload.py", line 157, in fragment_upload_async
    session = self.create_session(item).post()
  File "c:\python36\lib\site-packages\onedrivesdk\request\item_create_session.py", line 106, in post
    return self.request().post()
  File "c:\python36\lib\site-packages\onedrivesdk\request\item_create_session.py", line 60, in post
    entity = UploadSession(json.loads(self.send(self.body_options).content))
  File "c:\python36\lib\site-packages\onedrivesdk\request_base.py", line 163, in send
    content=content_dict)
  File "c:\python36\lib\site-packages\onedrivesdk\http_provider.py", line 71, in send
    custom_response = HttpResponse(response.status_code, response.headers, response.text)
  File "c:\python36\lib\site-packages\onedrivesdk\http_response.py", line 49, in __init__
    raise OneDriveError(message["error"], self.status)
onedrivesdk.error.OneDriveError: invalidRequest - API not found

NON-WORKING 'upload' (big file of 130MB):

c:\Python36\Scripts>python onedrive.py
https://login.live.com/oauth20_token.srf Content-Type: application/x-www-form-urlencoded
 POST
Content: {"token_type":"bearer","expires_in":3600,"scope":"wl.signin wl.offline_access onedrive.readwrite","access_token":"<MY_ACCESS_TOKEN>","refresh_token":"<MY_REFRESH_TOKEN>","user_id":"b4db8907776da42438d7d1900531a04a"}
Headers: {'Server': 'Microsoft-IIS/8.5', 'Date': 'Sat, 30 Jul 2016 11:40:14 GMT', 'Connection': 'close', 'Content-Type': 'application/json', 'Pragma': 'no-cache', 'X-Content-Type-Options': 'nosniff', 'Strict-Transport-Security': 'max-age=31536000', 'Cache-Control': 'no-store', 'X-XSS-Protection': '1; mode=block', 'Content-Length': '1593'}
Status: 200

Uploading item/s..
Traceback (most recent call last):
  File "c:\python36\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 578, in urlopen
    chunked=chunked)
  File "c:\python36\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 362, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "c:\python36\lib\http\client.py", line 1155, in request
    self._send_request(method, url, body, headers)
  File "c:\python36\lib\http\client.py", line 1200, in _send_request
    self.endheaders(body)
  File "c:\python36\lib\http\client.py", line 1151, in endheaders
    self._send_output(message_body)
  File "c:\python36\lib\http\client.py", line 985, in _send_output
    self.send(message_body)
  File "c:\python36\lib\http\client.py", line 954, in send
    self.sock.sendall(datablock)
  File "c:\python36\lib\ssl.py", line 887, in sendall
    v = self.send(data[count:])
  File "c:\python36\lib\ssl.py", line 857, in send
    return self._sslobj.write(data)
  File "c:\python36\lib\ssl.py", line 582, in write
    return self._sslobj.write(data)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\python36\lib\site-packages\requests\adapters.py", line 403, in send
    timeout=timeout
  File "c:\python36\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 623, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "c:\python36\lib\site-packages\requests\packages\urllib3\util\retry.py", line 255, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "c:\python36\lib\site-packages\requests\packages\urllib3\packages\six.py", line 309, in reraise
    raise value.with_traceback(tb)
  File "c:\python36\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 578, in urlopen
    chunked=chunked)
  File "c:\python36\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 362, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "c:\python36\lib\http\client.py", line 1155, in request
    self._send_request(method, url, body, headers)
  File "c:\python36\lib\http\client.py", line 1200, in _send_request
    self.endheaders(body)
  File "c:\python36\lib\http\client.py", line 1151, in endheaders
    self._send_output(message_body)
  File "c:\python36\lib\http\client.py", line 985, in _send_output
    self.send(message_body)
  File "c:\python36\lib\http\client.py", line 954, in send
    self.sock.sendall(datablock)
  File "c:\python36\lib\ssl.py", line 887, in sendall
    v = self.send(data[count:])
  File "c:\python36\lib\ssl.py", line 857, in send
    return self._sslobj.write(data)
  File "c:\python36\lib\ssl.py", line 582, in write
    return self._sslobj.write(data)
requests.packages.urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "onedrive.py", line 33, in <module>
    client.item(drive="me", id="8930240577EB1134%2166557").children["RoyalTSInstaller.msi"].upload("./RoyalTSInstaller.msi")
  File "c:\python36\lib\site-packages\onedrivesdk\request\item_request_builder.py", line 134, in upload
    return self.content.request().upload(local_path)
  File "c:\python36\lib\site-packages\onedrivesdk\request\item_content_request.py", line 59, in upload
    entity_response = self.send(path=content_local_path)
  File "c:\python36\lib\site-packages\onedrivesdk\request_base.py", line 145, in send
    path=path)
  File "c:\python36\lib\site-packages\onedrivesdk\http_provider.py", line 61, in send
    response = session.send(prepped)
  File "c:\python36\lib\site-packages\requests\sessions.py", line 585, in send
    r = adapter.send(request, **kwargs)
  File "c:\python36\lib\site-packages\requests\adapters.py", line 453, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

@cdmayer
Copy link
Contributor

cdmayer commented Aug 1, 2016

OK a few things:

  • The small-file upload_async bug has been fixed in 1.0.8. You should be able to use pip install onedrivesdk --upgrade to fix that.
  • Using upload on a file larger than 100MB will never work.
  • When you send the request before printing, the exception gets thrown before you have a chance to print the request information. Please move the debugging print statement to be ABOVE the line that sends the request. Should be something like this:
    print(request.url, '\n'.join(['{}: {}'.format(k, v) for k, v in request.headers.items()]), '\n', request.method)
    custom_response = HttpResponse(response.status_code, response.headers, response.text)
    print(custom_response)
    return custom_response

Thanks for sticking with me through all this debugging. We'll get it!

@diman82
Copy link
Author

diman82 commented Aug 1, 2016

OK, I've upgraded to 1.0.8, now I get a different error

c:\Python36\Scripts>..\python.exe onedrive.py
https://login.live.com/oauth20_token.srf Content-Type: application/x-www-form-urlencoded
 POST
Content: {"token_type":"bearer","expires_in":3600,"scope":"wl.signin wl.offline_access onedrive.readwrite","access_token":"<ACCESS_TOKEN>","refresh_token":"<REFRESH_TOKEN>","user_id":"b4db8907776da42438d7d1900531a04a"}
Headers: {'Server': 'Microsoft-IIS/8.5', 'X-XSS-Protection': '1; mode=block', 'Connection': 'close', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'Pragma': 'no-cache', 'Content-Length': '1593', 'Cache-Control': 'no-store', 'Strict-Transport-Security': 'max-age=31536000', 'Date': 'Mon, 01 Aug 2016 20:56:45 GMT'}
Status: 200

Uploading item/s..
Traceback (most recent call last):
  File "onedrive.py", line 34, in <module>
    client.item(drive="me", id="8930240577EB1134%2166557").children["RoyalTSInstaller.msi"].upload_async("./RoyalTSInstaller.msi")
  File "c:\Python36\lib\site-packages\onedrivesdk\version_bridge\fragment_upload.py", line 157, in fragment_upload_async
    session = self.create_session(item).post()
  File "c:\Python36\lib\site-packages\onedrivesdk\request\item_create_session.py", line 106, in post
    return self.request().post()
  File "c:\Python36\lib\site-packages\onedrivesdk\request\item_create_session.py", line 60, in post
    entity = UploadSession(json.loads(self.send(self.body_options).content))
  File "c:\Python36\lib\site-packages\onedrivesdk\request_base.py", line 163, in send
    content=content_dict)
  File "c:\Python36\lib\site-packages\onedrivesdk\http_provider.py", line 69, in send
    response = session.send(prepped)
  File "c:\Python36\lib\site-packages\requests\sessions.py", line 585, in send
    r = adapter.send(request, **kwargs)
  File "c:\Python36\lib\site-packages\requests\adapters.py", line 403, in send
    timeout=timeout
  File "c:\Python36\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 578, in urlopen
    chunked=chunked)
  File "c:\Python36\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 362, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "c:\Python36\lib\http\client.py", line 1155, in request
    self._send_request(method, url, body, headers)
  File "c:\Python36\lib\http\client.py", line 1195, in _send_request
    self.putheader(hdr, value)
  File "c:\Python36\lib\http\client.py", line 1132, in putheader
    raise ValueError('Invalid header value %r' % (values[i],))
**ValueError: Invalid header value b'SDK-Version=python-v1.0.8\r\n**'

@cdmayer
Copy link
Contributor

cdmayer commented Aug 1, 2016

Did you change the location of the debug statement? I still can't see the request information.

Although it's unlikely to be the problem, Python 3.6 is not supported at this time since it's in alpha. Could you please switch to the latest 3.5 release? This is tested to be working in 3.5 so at this point the issue is probably not with the SDK.

@diman82
Copy link
Author

diman82 commented Aug 1, 2016

OK, so I installed Python 3.5 and still get the same error, also I get the very same on my embedded device (ARM7 Netgear R7000):

root@DD-WRT:/opt/bin# python3.5 onedrivesdk3.5.py
Uploading item/s..
Traceback (most recent call last):
  File "onedrivesdk3.5.py", line 39, in <module>
    client.item(drive="me", id="8930240577EB1134%2166557").children["Agron_reshumon.rar"].upload_async("/tmp/mnt/sda1/Upload/Agron_reshumon.rar")
  File "/opt/lib/python3.5/site-packages/onedrivesdk/version_bridge/fragment_upload.py", line 157, in fragment_upload_async
    session = self.create_session(item).post()
  File "/opt/lib/python3.5/site-packages/onedrivesdk/request/item_create_session.py", line 106, in post
    return self.request().post()
  File "/opt/lib/python3.5/site-packages/onedrivesdk/request/item_create_session.py", line 60, in post
    entity = UploadSession(json.loads(self.send(self.body_options).content))
  File "/opt/lib/python3.5/site-packages/onedrivesdk/request_base.py", line 163, in send
    content=content_dict)
  File "/opt/lib/python3.5/site-packages/onedrivesdk/http_provider.py", line 69, in send
    response = session.send(prepped)
  File "/opt/lib/python3.5/site-packages/requests/sessions.py", line 585, in send
    r = adapter.send(request, **kwargs)
  File "/opt/lib/python3.5/site-packages/requests/adapters.py", line 403, in send
    timeout=timeout
  File "/opt/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 578, in urlopen
    chunked=chunked)
  File "/opt/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 362, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/opt/lib/python3.5/http/client.py", line 1083, in request
    self._send_request(method, url, body, headers)
  File "/opt/lib/python3.5/http/client.py", line 1123, in _send_request
    self.putheader(hdr, value)
  File "/opt/lib/python3.5/http/client.py", line 1060, in putheader
    raise ValueError('Invalid header value %r' % (values[i],))
ValueError: Invalid header value b'SDK-Version=python-v1.0.8\r\n'

Yes, I've put the debug statement where you requested, but I think you won't see it, because the script fails beforehand, in the if-else statement, so I've added the debug statement a little before, like this:

        if path:
            with open(path, mode='rb') as f:
                request = requests.Request(method,
                                           url,
                                           headers=headers,
                                           data=f)
                prepped = request.prepare()
                print(request.url, '\n'.join(['{}: {}'.format(k, v) for k, v in request.headers.items()]), '\n', request.method)
                response = session.send(prepped)
        else:
            request = requests.Request(method,
                                       url,
                                       headers=headers,
                                       data=data,
                                       json=content)
            prepped = request.prepare()
            print(request.url, '\n'.join(['{}: {}'.format(k, v) for k, v in request.headers.items()]), '\n', request.method)
            response = session.send(prepped)
        #print(request.url, '\n'.join(['{}: {}'.format(k, v) for k, v in request.headers.items()]), '\n', request.method)
        custom_response = HttpResponse(response.status_code, response.headers, response.text)
        print(custom_response)
        return custom_response

This is the output:

c:\Python36\Scripts>..\python.exe onedrive.py
https://login.live.com/oauth20_token.srf Content-Type: application/x-www-form-urlencoded
 POST
Headers: {'Date': 'Mon, 01 Aug 2016 21:48:28 GMT', 'Connection': 'close', 'X-XSS-Protection': '1; mode=block', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'Cache-Control': 'no-store', 'Pragma': 'no-cache', 'Content-Length': '1593', 'Server': 'Microsoft-IIS/8.5', 'Strict-Transport-Security': 'max-age=31536000'}
Status: 200
Content: {"token_type":"bearer","expires_in":3600,"scope":"wl.signin wl.offline_access onedrive.readwrite","access_token":"<ACCESS_TOKEN>","refresh_token":"<REFRESH_TOKEN>","user_id":"b4db8907776da42438d7d1900531a04a"}

Uploading item/s..
https://api.onedrive.com/v1.0/drives/me/items/8930240577EB1134%2166557/children/RoyalTSInstaller.msi/upload.createSession X-RequestStats: SDK-Version=python-v1.0.8

Content-Type: application/json
Authorization: bearer
<AUTH LONG CODE>
 POST
Traceback (most recent call last):
  File "onedrive.py", line 34, in <module>
    client.item(drive="me", id="8930240577EB1134%2166557").children["RoyalTSInstaller.msi"].upload_async("./RoyalTSInstaller.msi")
  File "c:\Python36\lib\site-packages\onedrivesdk\version_bridge\fragment_upload.py", line 157, in fragment_upload_async
    session = self.create_session(item).post()
  File "c:\Python36\lib\site-packages\onedrivesdk\request\item_create_session.py", line 106, in post
    return self.request().post()
  File "c:\Python36\lib\site-packages\onedrivesdk\request\item_create_session.py", line 60, in post
    entity = UploadSession(json.loads(self.send(self.body_options).content))
  File "c:\Python36\lib\site-packages\onedrivesdk\request_base.py", line 163, in send
    content=content_dict)
  File "c:\Python36\lib\site-packages\onedrivesdk\http_provider.py", line 71, in send
    response = session.send(prepped)
  File "c:\Python36\lib\site-packages\requests\sessions.py", line 585, in send
    r = adapter.send(request, **kwargs)
  File "c:\Python36\lib\site-packages\requests\adapters.py", line 403, in send
    timeout=timeout
  File "c:\Python36\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 578, in urlopen
    chunked=chunked)
  File "c:\Python36\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 362, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "c:\Python36\lib\http\client.py", line 1155, in request
    self._send_request(method, url, body, headers)
  File "c:\Python36\lib\http\client.py", line 1195, in _send_request
    self.putheader(hdr, value)
  File "c:\Python36\lib\http\client.py", line 1132, in putheader
    raise ValueError('Invalid header value %r' % (values[i],))
ValueError: Invalid header value b'SDK-Version=python-v1.0.8\r\n'

I think Onedrive API complains about the header SDK-Version=python-v1.0.8\r\n.
Is this correct, and necessary?

@diman82
Copy link
Author

diman82 commented Aug 1, 2016

BTW, I get this ValueError: Invalid header value exception also for an 'upload' method or getting top three elements of root (actually for everything), so maybe it'll be proper to close this issue and open a new one?

@cdmayer
Copy link
Contributor

cdmayer commented Aug 1, 2016

Looks like GH added a line break even though there wasn't one in the editor. That's what I get for committing a change using the Github online editor.

Uploaded version 1.0.9 to PyPi. Please give that a shot. Thanks for your patience.

@diman82
Copy link
Author

diman82 commented Aug 1, 2016

No problem, I'm glad to help investigating this issue.
Sadly, back to the primal error:

c:\Python36\Scripts>..\python.exe onedrive.py
https://login.live.com/oauth20_token.srf Content-Type: application/x-www-form-urlencoded
 POST
Uploading item/s..
https://api.onedrive.com/v1.0/drives/me/items/8930240577EB1134%2166557/children/RoyalTSInstaller.msi/upload.createSession X-RequestStats: SDK-Version=python-v1.0.9
Authorization: bearer 
<AUTH>
Content-Type: application/json
 POST
Traceback (most recent call last):
  File "onedrive.py", line 34, in <module>
    client.item(drive="me", id="8930240577EB1134%2166557").children["RoyalTSInstaller.msi"].upload_async("./RoyalTSInstaller.msi")
  File "c:\Python36\lib\site-packages\onedrivesdk\version_bridge\fragment_upload.py", line 157, in fragment_upload_async
    session = self.create_session(item).post()
  File "c:\Python36\lib\site-packages\onedrivesdk\request\item_create_session.py", line 106, in post
    return self.request().post()
  File "c:\Python36\lib\site-packages\onedrivesdk\request\item_create_session.py", line 60, in post
    entity = UploadSession(json.loads(self.send(self.body_options).content))
  File "c:\Python36\lib\site-packages\onedrivesdk\request_base.py", line 163, in send
    content=content_dict)
  File "c:\Python36\lib\site-packages\onedrivesdk\http_provider.py", line 73, in send
    custom_response = HttpResponse(response.status_code, response.headers, response.text)
  File "c:\Python36\lib\site-packages\onedrivesdk\http_response.py", line 49, in __init__
    raise OneDriveError(message["error"], self.status)
**onedrivesdk.error.OneDriveError: invalidRequest - API not found**

@cdmayer
Copy link
Contributor

cdmayer commented Aug 2, 2016

I believe I see the issue. The underlying problem is that you are mixing path- and resource-based access together. You can read more here: https://dev.onedrive.com/items/upload_post.htm

To fix, I would recommend that you use the path to your target file, something like this:

client.item(drive="me", path="foo/RoyalTSInstaller.msi").upload_async("./RoyalTSInstaller.msi")

Let me know if that works for you.

@diman82
Copy link
Author

diman82 commented Aug 2, 2016

I didn't understand much from dev.onedrive.com documentation, but you've identified the problem properly!

So, this variations work:
client.item(drive="me", path="/RoyalTSInstaller.msi").upload_async("./RoyalTSInstaller.msi")
client.item(drive="me", path="Uploaded/test1.mp3").upload_async("./test1.mp3")

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants