Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multipart Exception at Amazon S3 #7919

Closed
ibr123 opened this issue Jan 17, 2018 · 8 comments

Comments

@ibr123
Copy link

commented Jan 17, 2018

Hi,

i have NextCloud (version 12.0.4.3) installed on a Linux and configured to use Amazon S3 bucket as the primary storage, and its connected to the bucket, i already checked the bucket and it contains files as (urn:oid: ) but sometimes (i noticed its occur more often when creating a file inside Sync folder not moving from outside to inside Sync folder) an exception occur and the NextCloud client can't sync, the exception is:
An exception occurred while completing a multipart upload: Error executing \\\"CompleteMultipartUpload\\\" on \\\"https:\\\/\\\/zoolznextcloud.s3.us-east-1.amazonaws.com

Thanks

@ibr123

This comment has been minimized.

Copy link
Author

commented Jan 21, 2018

UPDATE
according to this issue the multipart exception was solved in later versions, so I installed the latest version of next cloud from this website the file latest-master.zip | 2018-01-20 22:04 | 57M |

yet the exception is still showing, in the nextcloud.log the version of server is "version":"14.0.0.0"
below is the exception that I have got:

{"reqId":"FjnET3lLnQ0HCOuTPYhA","level":3,"time":"2018-01-21T07:50:35+00:00","remoteAddr":"xxxxxxxxxxxx","user":"testUserLatest","app":"no app in context","method":"PUT","url":"\/nextcloud\/remote.php\/dav\/files\/testUserLatest\/q.txt","message":"Exception: {\"Exception\":\"Aws\\\\S3\\\\Exception\\\\S3MultipartUploadException\",\"Message\":\"An exception occurred while completing a multipart upload: Error executing \\\"CompleteMultipartUpload\\\" on \\\"https:\\\/\\\/zoolznextcloud.s3.us-east-1.amazonaws.com\\\/urn%3Aoid%3A146?uploadId=eKAEfL3_1rbr_6LgBOS1S9SRYVsYwP8YsEndByAVmlVTYR3nUafw89WXanVZY7n4DfdQ83bLHlGqITyAssi2AB5tjU9c5kKQSK4BJTVLJ0bSmc0IMBa._YmrQOfa2hre\\\"; AWS HTTP error: Client error response [url] https:\\\/\\\/zoolznextcloud.s3.us-east-1.amazonaws.com\\\/urn%3Aoid%3A146?uploadId=eKAEfL3_1rbr_6LgBOS1S9SRYVsYwP8YsEndByAVmlVTYR3nUafw89WXanVZY7n4DfdQ83bLHlGqITyAssi2AB5tjU9c5kKQSK4BJTVLJ0bSmc0IMBa._YmrQOfa2hre [status code] 400 [reason phrase] Bad Request MalformedXML (client): The XML you provided was not well-formed or did not validate against our published schema - <Error><Code>MalformedXML<\\\/Code><Message>The XML you provided was not well-formed or did not validate against our published schema<\\\/Message><RequestId>167E9B85554B2338<\\\/RequestId><HostId>lojeyIgN19jVKTOKqrcW6dUJjyxqfjyFOYpXr2ZUWE6CphtAb9E9Br9K+nAE+DN1LstWREeK9rE=<\\\/HostId><\\\/Error>\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/guzzlehttp\\\/promises\\\/src\\\/Promise.php(203): Aws\\\\Multipart\\\\AbstractUploadManager->Aws\\\\Multipart\\\\{closure}(*** sensitive parameters replaced ***)\\n#1 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/guzzlehttp\\\/promises\\\/src\\\/Promise.php(156): GuzzleHttp\\\\Promise\\\\Promise::callHandler(2, Object(Aws\\\\S3\\\\Exception\\\\S3Exception), Array)\\n#2 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/guzzlehttp\\\/promises\\\/src\\\/TaskQueue.php(47): GuzzleHttp\\\\Promise\\\\Promise::GuzzleHttp\\\\Promise\\\\{closure}(*** sensitive parameters replaced ***)\\n#3 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/guzzlehttp\\\/promises\\\/src\\\/Promise.php(234): GuzzleHttp\\\\Promise\\\\TaskQueue->run()\\n#4 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/guzzlehttp\\\/promises\\\/src\\\/Promise.php(267): GuzzleHttp\\\\Promise\\\\Promise->waitIfPending()\\n#5 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/guzzlehttp\\\/promises\\\/src\\\/Promise.php(225): GuzzleHttp\\\\Promise\\\\Promise->invokeWaitList()\\n#6 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/guzzlehttp\\\/promises\\\/src\\\/Promise.php(267): GuzzleHttp\\\\Promise\\\\Promise->waitIfPending()\\n#7 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/guzzlehttp\\\/promises\\\/src\\\/Promise.php(225): GuzzleHttp\\\\Promise\\\\Promise->invokeWaitList()\\n#8 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/guzzlehttp\\\/promises\\\/src\\\/Promise.php(62): GuzzleHttp\\\\Promise\\\\Promise->waitIfPending()\\n#9 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/guzzlehttp\\\/promises\\\/src\\\/Coroutine.php(65): GuzzleHttp\\\\Promise\\\\Promise->wait()\\n#10 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/guzzlehttp\\\/promises\\\/src\\\/Promise.php(246): GuzzleHttp\\\\Promise\\\\Coroutine->GuzzleHttp\\\\Promise\\\\{closure}(*** sensitive parameters replaced ***)\\n#11 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/guzzlehttp\\\/promises\\\/src\\\/Promise.php(223): GuzzleHttp\\\\Promise\\\\Promise->invokeWaitFn()\\n#12 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/guzzlehttp\\\/promises\\\/src\\\/Promise.php(267): GuzzleHttp\\\\Promise\\\\Promise->waitIfPending()\\n#13 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/guzzlehttp\\\/promises\\\/src\\\/Promise.php(225): GuzzleHttp\\\\Promise\\\\Promise->invokeWaitList()\\n#14 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/guzzlehttp\\\/promises\\\/src\\\/Promise.php(62): GuzzleHttp\\\\Promise\\\\Promise->waitIfPending()\\n#15 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/aws\\\/aws-sdk-php\\\/src\\\/Multipart\\\/AbstractUploadManager.php(83): GuzzleHttp\\\\Promise\\\\Promise->wait()\\n#16 \\\/var\\\/www\\\/nextcloud\\\/lib\\\/private\\\/Files\\\/ObjectStore\\\/S3ObjectTrait.php(107): Aws\\\\Multipart\\\\AbstractUploadManager->upload()\\n#17 \\\/var\\\/www\\\/nextcloud\\\/lib\\\/private\\\/Files\\\/ObjectStore\\\/S3ObjectTrait.php(83): OC\\\\Files\\\\ObjectStore\\\\S3->multiPartUpload('urn:oid:146', Resource id #72)\\n#18 \\\/var\\\/www\\\/nextcloud\\\/lib\\\/private\\\/Files\\\/ObjectStore\\\/ObjectStoreStorage.php(406): OC\\\\Files\\\\ObjectStore\\\\S3->writeObject('urn:oid:146', Resource id #72)\\n#19 \\\/var\\\/www\\\/nextcloud\\\/lib\\\/private\\\/Files\\\/ObjectStore\\\/ObjectStoreStorage.php(306): OC\\\\Files\\\\ObjectStore\\\\ObjectStoreStorage->writeBack('\\\/tmp\\\/oc_tmp_e4D...', 'files\\\/q.txt.ocT...')\\n#20 [internal function]: OC\\\\Files\\\\ObjectStore\\\\ObjectStoreStorage->OC\\\\Files\\\\ObjectStore\\\\{closure}(*** sensitive parameters replaced ***)\\n#21 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/icewind\\\/streams\\\/src\\\/CallbackWrapper.php(109): call_user_func(Object(Closure))\\n#22 [internal function]: Icewind\\\\Streams\\\\CallbackWrapper->stream_close()\\n#23 \\\/var\\\/www\\\/nextcloud\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/File.php(162): fclose(Resource id #70)\\n#24 \\\/var\\\/www\\\/nextcloud\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/Directory.php(151): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\File->put(Resource id #66)\\n#25 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(1096): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\Directory->createFile('q.txt', Resource id #66)\\n#26 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(525): Sabre\\\\DAV\\\\Server->createFile('files\\\/testUserL...', Resource id #66, NULL)\\n#27 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpPut(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#28 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#29 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(479): Sabre\\\\Event\\\\EventEmitter->emit('method:PUT', Array)\\n#30 \\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(254): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#31 \\\/var\\\/www\\\/nextcloud\\\/apps\\\/dav\\\/lib\\\/Server.php(283): Sabre\\\\DAV\\\\Server->exec()\\n#32 \\\/var\\\/www\\\/nextcloud\\\/apps\\\/dav\\\/appinfo\\\/v2\\\/remote.php(35): OCA\\\\DAV\\\\Server->exec()\\n#33 \\\/var\\\/www\\\/nextcloud\\\/remote.php(164): require_once('\\\/var\\\/www\\\/nextcl...')\\n#34 {main}\",\"File\":\"\\\/var\\\/www\\\/nextcloud\\\/3rdparty\\\/aws\\\/aws-sdk-php\\\/src\\\/Multipart\\\/AbstractUploadManager.php\",\"Line\":136}","userAgent":"Mozilla\/5.0 (Windows) mirall\/2.3.3 (build 1) (Nextcloud)","version":"14.0.0.0"}

@ibr123

This comment has been minimized.

Copy link
Author

commented Jan 22, 2018

UPDATE
at the file S3ObjectTrait.php I changed the line if ($stat['size'] && $stat['size'] < S3_UPLOAD_PART_SIZE) { to if ($stat['size'] < S3_UPLOAD_PART_SIZE) { and the I was able to create files in the Sync folder, which what was generating the exception I just added, meaning the Exception didn't appear

After that I tied to do that on another server that I have installed using snap but I was unable to make any change to the file "S3ObjectTrait.php" at snap directory
How to do that?

@zebulongriggs

This comment has been minimized.

Copy link

commented Jan 23, 2018

I'm having exactly the same issue and also installed using snap. Any thoughts on how to make the change in the snap directory?

@ArnY

This comment has been minimized.

Copy link

commented Jan 24, 2018

We are also having the same problem with Ceph RGW with a slight difference in the error message (but with the same meaning), also with Nextcloud 12.0.4, installed from sources:

ownCloud[311]: {webdav} Exception: {"Exception":"Sabre\\DAV\\Exception","Message":"An exception occurred while completing a multipart upload: Error executing \"CompleteMultipartUpload\" on \"http:\/\/cloudrgw.ha.univ-nantes.prive\/cloudprod\/urn%3Aoid%3A20093192?uploadId=2~T2in108rAus-_wJp8CcRK2pAXBOC7Gk\"; AWS HTTP error: Client error response [url] http:\/\/cloudrgw.ha.univ-nantes.prive\/cloudprod\/urn%3Aoid%3A20093192?uploadId=2~T2in108rAus-_wJp8CcRK2pAXBOC7Gk [status code] 400 [reason phrase] Bad Request MalformedXML (client): - <?xml version=\"1.0\" encoding=\"UTF-8\"?><Error><Code>MalformedXML<\/Code><BucketName>cloudprod<\/BucketName><RequestId>tx00000000000000004488c-005a68802b-98d27-default<\/RequestId><HostId>98d27-default-default<\/HostId><\/Error>...

@ShaunMaher

This comment has been minimized.

Copy link

commented Feb 5, 2018

@ibr123's solution above, modifying S3ObjectTrait.php, solved this issue for me.

@ibr123

This comment has been minimized.

Copy link
Author

commented Feb 5, 2018

@ShaunMaher glad I helped , if you installed NextCloud manually, and keep it that way it works, but if you try to do the same from "snap" installation you wont be able to make these modification, since will give you an error that you cant change system's files

@ProteanHub

This comment has been minimized.

Copy link

commented Feb 13, 2018

Making this change allowed the file to get added, but was zero bytes in length.

@ibr123

This comment has been minimized.

Copy link
Author

commented Feb 13, 2018

@ProteanHub when creating a new file inside the synced folder its going to be zero because it doesn't contain any data, which is what caused this exception in the first place, the problem was uploading a zero sized file.
its clear when you copy a file with size from outside to the synced folder, in this case it actually two actions, creating a file inside synced folder, then writing the contents to the newly created file, so in this case, there is creating action, yet the exception doesn't appear since the file has size
NextCLoud devs say there is releases solved this issue, I didn't try it, but you can, its at the end of this issue

@ibr123 ibr123 closed this Feb 13, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.