Snap install doesn't allow Transfer-Encoding: chunked on PUT.

[ncw]

Snap install doesn't allow Transfer-Encoding: chunked on PUT.

This does work with an owncloud I have installed manually though, leading me to think that this is a problem with the snap packaging, in particular the webserver config.

For example

Without chunked transfer encoding

$ curl -X PUT --user user:pass --data @file.txt "https://nextcloud.example.com/remote.php/webdav/file.txt"
$ curl -X PROPFIND --user user:pass "https://nextcloud.example.com/remote.php/webdav/file.txt"
[snip]
    <d:getlastmodified>Fri, 06 Oct 2017 14:43:22 GMT</d:getlastmodified>
    <d:getcontentlength>5</d:getcontentlength>  <-------------- length 5
[snip]

And with chunked transfer encoding the upload is truncated to 0 bytes with no warning.

$ curl -X PUT --user user:pass --data @file.txt --header "Transfer-Encoding: chunked" "https://nextcloud.example.com/remote.php/webdav/file.txt"
$ curl -X PROPFIND --user user:pass "https://nextcloud.example.com/remote.php/webdav/file.txt"
[snip]
    <d:getlastmodified>Fri, 06 Oct 2017 14:43:39 GMT</d:getlastmodified>
    <d:getcontentlength>0</d:getcontentlength>  <-------------- length 0!
[snip]

A possibly related problem is discussed in the SabreDav wiki which says

Finder uses Transfer-Encoding: Chunked in PUT request bodies. This is a little used HTTP feature, and therefore not implemented in a bunch of webservers. The only server I've seen so far that handles this reasonably well is Apache + mod_php. Nginx and Lighttpd respond with 411 Length Required, which is completely ignored by Finder. This was seen on nginx 0.7.63. It was recently reported that a development release (1.3.8) no longer had this issue.

When using this with Apache + FastCGI PHP completely drops the request body, so it will seem as if the PUT request was succesful, but the file will end up empty.

Diving down the rat-hole further I find this php bug which points the finger at apache's fast cgi implemention with this bug.

The symptoms of the above seem to be that of using Apache with FastCGI but I'm not sure exactly what is used in the snap. Is it possible to swap to using apache + mod_php?

[kyrofa]

We were initially using mod_php, but switched to php-fpm a while back for more fine-grained control over performance. I suspect we'll have to wait for those bugs to be fixed ☹️ .

[ncw]

That sounds reasonable. It isn't a big deal for me it just means rclone can't stream uploads to nextcloud for the moment.

[gbrault]

Hello there! Do we now if this is fixed now?

[r4co0n]

The Apache bug blocking this issue is reported against mod_fcgid, not mod_proxy_fcgi which this snap is using and we will probably wait long for the fix landing, which won't change a thing.

@ncw Can you still reproduce this issue with the current version? How about the edge release channel?

Ha, I think I found the correct bug, it is still open.

[devurandom]

Does anyone know what the current status for php-fpm + nginx / lighttpd is? Are they affected? If not, which version / configuration is necessary for this to work?

[e-alfred]

Does this only apply to Apache/Nginx + PHP-FPM? Or does it affect mod_php too? If not, the rclone documentation is misleading because this would only affect the Snap package, but not Nextcloud in general (if manually installed for example).

[r4co0n]

This is an Apache httpd-2 bug.

[e-alfred]

The bug referred in the original post was fixed in Apache by this patch: https://svn.apache.org/viewvc?view=revision&revision=1848298

Is this problem still relevant then?

[kyrofa]

I'm going to go ahead and close this issue. We can't do anything about it other than wait for upstream components to fix it, which we update continually, so there isn't much value in continuing to track it here. Sounds like it probably has been fixed by now, anyway.

[ylluminate]

Has this been resolved?

[kyrofa]

It sounds like it, but I haven't confirmed.

[gnoutchd]

I just got bitten by this bug hard, as in my phone's backups (uploaded automatically via Davx5) were wrecked for weeks and it was only by luck that I noticed.

According to comments in nextcloud/server#7995, nextcloud/vm and a few other deployments have workarounds in place (something involving a SetEnv proxy-sendcl somewhere in the apache config). Could the snap do the same thing?