Use X-Accel-Redirect or X-Sendfile when possible for reduced resource usage

[rkkoszewski]

From what I could observe, Nextcloud uses PHP to send the stored files to the client, which can easily time out for huge files, use a lot of RAM and is generally slower especially on low powered or older servers.

It would be great to have an option to use X-Accel-Redirect (Nginx) or X-Sendfile (Apache) when possible to take advantage of the web server's optimised native capabilities.

It might not be possible to use it for encrypted storages or remote storages, but it would greatly reduce the stress on the server for local files.

[MorrisJobke]

It would be great to have an option to use X-Accel-Redirect (Nginx) or X-Sendfile (Apache) when possible to take advantage of the web server's optimised native capabilities.

It might not be possible to use it for encrypted storages or remote storages, but it would greatly reduce the stress on the server for local files.

We discussed this quite intensively in the past and the biggest problem was on how to determine where are file is located and how it interferes with other storage and the web server. In the end it turned out that it was a major amount of work to get all the possible data storages covered and not break them completely.

It still this sounds like a nice feature, but the requests for this are quite low. Currently there are no plans to implement such a feature. Thus I will close this ticket for now. This does not mean we don't want this feature at all, but it is simply not on our roadmap for the near future. If somebody wants to implement this feature nevertheless we are happy to assist, discuss and help out.

If you wish to have this feature implemented by the Nextcloud GmbH there is the option for consulting work on top of your Nextcloud Enterprise subscription to get your features implemented.

[manwegit]

I had X-Sendfile working around the Owncloud Nextcloud fork.
Ubuntu system, apache + php-fpm. Had to modify some core files to fix filesending problems. And apache config was tightly coupled with nextcloud setup.

If I have the time, I'll try to see if it's still feasible with limited support after the filesystem things revamp.

[mark99i]

I will still bring up this topic, since when distributing files from NextCloud, even at low speeds, there is a very high disk I/o (up to 1600 iops), and sometimes the cpu load. Load Average on a server with 2x E5-2630 v4 reaches 40. most resources are used by php-fpm processes. at the same time, I use nginx + php-fpm with NextCloud 19.0.0. the built-in checks are all passed. I attached network and disk io graphs for two days. disk loading does not match network loading. I don't think fastcgi cache will help me

[gstrauss]

@MorrisJobke wrote

We discussed this quite intensively in the past and the biggest problem was on how to determine where are file is located and how it interferes with other storage and the web server. In the end it turned out that it was a major amount of work to get all the possible data storages covered and not break them completely.

Would you please share a little bit more detail about some of the reasons why this was deemed so difficult?

It still this sounds like a nice feature, but the requests for this are quite low. Currently there are no plans to implement such a feature. Thus I will close this ticket for now. This does not mean we don't want this feature at all, but it is simply not on our roadmap for the near future. If somebody wants to implement this feature nevertheless we are happy to assist, discuss and help out.

What about an improvement where X-Sendfile is used when safe to do so, or else continue with existing behavior? Why bother trying to get all the possible data storages covered instead of disabling the new feature by default, and enabling the new feature where it is known to be safe to do so?

How about a new NextCloud config option which, if present, indicates the name of the X-Sendfile header (X-Sendfile or X-Accel-Redirect) to be used on eligible responses.

If this new X-Sendfile option were enabled in the NextCloud config, then FileDisplayResponse() could use X-Sendfile if the target file is not a temporary file, i.e. !$file->isEncrypted() && $file->getStorage()->isLocal() . FileDisplayResponse() might also optionally check size and use X-Sendfile only if the file exceeds a certain minimum size, e.g. $file->getSize() > 32KB . A future improvement could be to provide a mechanism to determine if the temporary file is actually ephemeral and about to be removed, or if it has a non-zero lifetime (in a cache) which is long enough for the webserver to reasonably read the header response from NextCloud and to open the file indicated by the X-Sendfile response header. FileDisplayResponse() might need an extra bit of information to indicate if the object is eligible for X-Sendfile. That could be an optional attribute set on the FileDisplayResponse object by caller, or could be an optional argument to the constructor, or could even be a new method on $file->isXSendfileEligible() if arbitrary logic is needed to determine eligibility. [Edit: similar to $file->getContent(), there could be a $file->getXSendfile() which returns a string to a file path iff the response is eligible for X-Sendfile]

As an aside, some use of DataDisplayResponse() appears at first glance to be able to use FileDisplayResponse(), unless the temporary file is to be cleaned up imminently (and therefore must be read into memory).

@manwegit wrote

I had X-Sendfile working around the Owncloud Nextcloud fork.
Ubuntu system, apache + php-fpm. Had to modify some core files to fix filesending problems. And apache config was tightly coupled with nextcloud setup.

If I have the time, I'll try to see if it's still feasible with limited support after the filesystem things revamp.

@manwegit might you have anything to add from your experience getting this working?

[gstrauss]

FYI: In the past, ownCloud supported X-Sendfile since ownCloud 5.
https://doc.owncloud.com/server/8.1/admin_manual/configuration_files/serving_static_files_configuration.html

However, it looks like that support was removed in 2015 in owncloud/core@8479702 with commit message:

Remove XSendFile support
Required to ensure proper locking

locking is, of course, an issue that must be acknowledged. Still, I think there must be some low-hanging fruit -- such as large images, audio, video, and other media -- for which the admin would like to enable X-Sendfile.

[gstrauss]

@MorrisJobke @violoncelloCH: ping. Would you please tag others in nextcloud who might be able to contribute some guidance?

I am willing to put together some patches (as demonstrated above).

File locking matters when the file might be modified in a non-atomic manner, e.g. via partial PUT with range, implemented as direct file modification rather than copy temp, modify temp, and atomic rename into place. The lighttpd web server mod_webdav implements atomic file updates, and provides an option (disabled by default) to enable unsafe direct file modification for partial PUT, should the admin desire. I think that there should be a similar option for admins to enable X-Sendfile in Nextcloud, should the admin attest that the partial PUT scenario is not one that affects the admin's system, and the admin instead wants the drastic performance increase available by using X-Sendfile.

[MorrisJobke]

I’m not with the project anymore as a maintainer. Maybe @skjnldsv can help..

[gstrauss]

At the moment, Nextcloud has over 1.4k open issues and 251 pull requests. I am guessing that a closed issue like this one is going to get lost in the noise, so I plan to open a new issue in Nov if there is no other guidance shared before then.

[gstrauss]

I opened new issue #29467 Use X-Sendfile when possible for reduced resource usage