Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apache images built after ~mid June 2022 introduce 1GB upload limit #1796

Closed
henryk opened this issue Jul 28, 2022 · 21 comments
Closed

Apache images built after ~mid June 2022 introduce 1GB upload limit #1796

henryk opened this issue Jul 28, 2022 · 21 comments

Comments

@henryk
Copy link

henryk commented Jul 28, 2022

After my last docker image update I suddenly saw upload failures for very large files, even though I hadn't changed any configuration, and the nextcloud docker build also has no related configuration changes. I finally tracked the problem down to a change in Apache default configuration.

Steps to reproduce:

  1. Start with nextcloud apache docker image built before June 9, 2022
  2. Check included Apache version (is 2.4.53)
  3. Create large testfile (dd if=/dev/zero of=testfile bs=1M count=1025)
  4. Create upload share in nextcloud
  5. curl -X PUT -H "X-Requested-With: XMLHttpRequest" -T testfile http://<shareid>:<sharepass>@<host>/public.php/webdav/testfile
  6. Update docker image to newest version (Apache is now 2.4.54)
  7. curl -X PUT -H "X-Requested-With: XMLHttpRequest" -T testfile http://<shareid>:<sharepass>@<host>/public.php/webdav/testfile2

Actual result:
Step 5 works, Step 7 doesn't work. A smaller testfile (dd if=/dev/zero of=testfile bs=1M count=1024) will still work with the new image.

Root cause: https://httpd.apache.org/docs/current/en/mod/core.html#limitrequestbody has changed default value in Apache version 2.4.53 from 0 (unlimited) to 1073741824. Note: This also means the the various documentation entries about increasing PHP_UPLOAD_LIMIT etc. don't apply.

IMHO this is a regression, and the apache nextcloud docker image should change the builtin Apache configuration back to the old value of 0 (unlimited).

@henryk
Copy link
Author

henryk commented Jul 28, 2022

The following workaround can be used (e.g. with docker-compose). Create a file nextcloud-apache.conf with the following contents:

LimitRequestBody 0

and bind mount into the container. E.g. in docker-compose:

....
  volumes:
    ....
    - nextcloud-apache.conf:/etc/apache2/conf-enabled/nextcloud-apache.conf:ro
....

But IMHO this really should be done as part of the container build, since it's an unexpected breaking behaviour change.

@niziak
Copy link

niziak commented Aug 9, 2022

Same issue affected me.

I cannot find any information about this change in Apache changelog. But documentation says:

LimitRequestBody Directive

Default: LimitRequestBody 1073741824
compatibility: In Apache HTTP Server 2.4.53 and earlier, the default value was 0 (unlimited)

Thank you @henryk for fast workaround.

leviia referenced this issue in leviia/server Aug 9, 2022
@Wouter0100
Copy link

Effected me as well. During debugging I "fixed" it quickly by adding the following to my .htaccess:

LimitRequestBody 0

But the solution provided in this thread is better IMO and I deployed it instead. Thanks for the (proper-ish) workaround.

@Practicalbutterfly5
Copy link

This issue is still around on apache docker image of nextcloud. Thanks for the workaround. I was wondering why my rclone sync tasks were failing after upgrade.

@mathesst
Copy link

Hi,

I'm also having this issue. Personally I would think that the apache config value should be configurable as the corresponding php config. Or that the "LimitRequestBody" value should be set to the same php value, if it is set.

BG Mathias

@mathesst
Copy link

Just saw another Issue with the same problem #1830

@kangaroo72
Copy link

I can confirm that bug. Sadly it's not fixed in the image, yet.
Thanks for that fix.

kangaroo72 added a commit to kangaroo72/homelab that referenced this issue Jan 12, 2023
@t-schuster
Copy link

I think the following patch should fix this.

diff --git a/update.sh b/update.sh
index 608a9bb..962b72e 100755
--- a/update.sh
+++ b/update.sh
@@ -20,7 +20,7 @@ declare -A base=(
 )
 
 declare -A extras=(
-	[apache]='\nRUN a2enmod headers rewrite remoteip ;\\\n    {\\\n     echo RemoteIPHeader X-Real-IP ;\\\n     echo RemoteIPTrustedProxy 10.0.0.0/8 ;\\\n     echo RemoteIPTrustedProxy 172.16.0.0/12 ;\\\n     echo RemoteIPTrustedProxy 192.168.0.0/16 ;\\\n    } > /etc/apache2/conf-available/remoteip.conf;\\\n    a2enconf remoteip'
+	[apache]='\nRUN a2enmod headers rewrite remoteip ;\\\n    {\\\n     echo RemoteIPHeader X-Real-IP ;\\\n     echo RemoteIPTrustedProxy 10.0.0.0/8 ;\\\n     echo RemoteIPTrustedProxy 172.16.0.0/12 ;\\\n     echo RemoteIPTrustedProxy 192.168.0.0/16 ;\\\n    } > /etc/apache2/conf-available/remoteip.conf;\\\n    a2enconf remoteip\\\n    { echo LimitRequestBody 0 ; } > /etc/apache2/conf-available/limitrequestbody.conf;\\\n    a2enconf limitrequestbody'
 	[fpm]=''
 	[fpm-alpine]=''
 )

@kangaroo72
Copy link

My workaround is a nextcloud-apache.conf with content LimitRequestBody 0 as a host-mount in my yml-file

@Trufax
Copy link

Trufax commented Jul 2, 2023

Thanks for the workaround, searched for ages. Hope this get fixed soon.

@Markus87
Copy link

Still an issue with Nextcloud 27.

@J0WI
Copy link
Contributor

J0WI commented Aug 10, 2023

There are multiple parameters and conditions that affect the upload limit: https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/big_file_upload_configuration.html
The optimal configuration is highly dependent on your use case.

@t-schuster
Copy link

Optimal configuration depends on the use case, correct, but the patch I wrote would atleast work, even if not optimally. In the container setup, the only limit I'm aware off is the Apache Limit right now, making this arbitrary 1GB limitation.

@svigerske
Copy link

svigerske commented Aug 11, 2023

It's kind of embarrassing that such an issue is still not fixed after over a year. Some update quietly introduced a previously not existing upload limit. A year ago, it took me a day to figure out why this is suddenly happening. I am not aware of documentation that announced this change. Also the current README here still does not mention that there is this limit one needs to be aware of.
After all, this is a software for storing files on a server. Something like a 1GB upload restriction is something to be aware of.

And instead of restoring previous behavior with a simple patch, one gets some excuse like "there are additional places to set an upload limit".

joshtrichards added a commit to joshtrichards/nc-documentation that referenced this issue Aug 11, 2023
Apache's default for this changed from 0 (unlimited) to 1 GiB. This impacts non-chunking client transactions.

Resolves: #35778 and #37695 and documents needed adjustments applicable to various others like nextcloud/docker#1796

Signed-off-by: Josh Richards <josh.t.richards@gmail.com>
@adripo
Copy link
Contributor

adripo commented Sep 13, 2023

Hello everyone,@J0WI,@joshtrichards

I wanted to inform you that I have created a pull request to address this issue.
This pull request introduces an ENV variable APACHE_BODY_LIMIT that allows users to set the LimitRequestBody value. The default value is aligned with the new Apache default.

Thank you for your attention. Your feedback and contributions are highly appreciated.

@henryk
Copy link
Author

henryk commented Sep 14, 2023

Note: I still think the docker build should just revert the Apache change generally (well, at least the footgun is documented now via nextcloud/documentation#10989). No one has mentioned yet the reason for the Apache change (and Apache doesn't make it easy to find): It's CVE-2022-29404. If you have a) mod_lua and b) are running a lua script that reads unbounded input, you get a DoS if you allow unlimited LimitRequestBody. As far as I can tell neither condition a and certainly not condition b are present in the nextcloud docker image, so the change doesn't really make sense.

@J0WI
Copy link
Contributor

J0WI commented Sep 14, 2023

Thanks for the pointer. I was looking for the reason of the change.

Setting it to unlimited for everyone like in #2043 might be an unexpected (and potentially dangerous) default. I think #2065 is a good compromise for this issue.

backportbot-nextcloud bot pushed a commit to nextcloud/documentation that referenced this issue Oct 3, 2023
Apache's default for this changed from 0 (unlimited) to 1 GiB. This impacts non-chunking client transactions.

Resolves: #35778 and #37695 and documents needed adjustments applicable to various others like nextcloud/docker#1796

Signed-off-by: Josh Richards <josh.t.richards@gmail.com>
@joshtrichards
Copy link
Member

Fixed in #2065

@svigerske
Copy link

So to use the fix, I would have to rebuild the docker image for apache with APACHE_BODY_LIMIT set to 0. Thus, the image from https://hub.docker.com/_/nextcloud is still unusable, unless one builds on top of it to overwrite /etc/apache2/conf-available/apache-limits.conf.

If NextCloud thinks that limiting uploads to 1GB is a good default, then I would have expected that I could change this default by setting the apache body limit when starting the container. But for that, /etc/apache2/conf-available/apache-limits.conf would need to be written by the startup script of the container and not in the build of the image.

@joshtrichards
Copy link
Member

joshtrichards commented Oct 28, 2023

@svigerske No, APACHE_BODY_LIMIT can be configured at run time just like all the other auto configuration environment variables.

If you're looking at the PR and trying to figure out how in the heck it works, here is a quick overview:

ENV values, defined in a Dockerfile, can be overridden when starting a container. While the RUN command that references that variable does execute at build time, the value of APACHE_BODY_LIMIT isn't what is written to apache-limits.conf in that RUN command. Instead only a reference to the variable is written (I.e. the variable name technically). Apache then handles picking up the value of APACHE_BODY_LIMIT from the container's environment when it loads that config file at runtime.

That value will either be the default (1G, as set in the Dockerfile) or the value specified at container run time (e.g. via environment in your Compose).

echo 'LimitRequestBody ${APACHE_BODY_LIMIT}'; \

https://httpd.apache.org/docs/2.4/configuring.html#syntax

The same approach is used for PHP_MEMORY_LIMIT since PHP also has built in support for substituting values in its php.ini with the contents of environment variables.

@svigerske
Copy link

Oh ok, you're right, single quotes. Sorry. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.