Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

File more than 50Mb size gets failed to upload #1759

Closed
ChaitanyaPTank opened this issue Apr 6, 2024 · 28 comments
Closed

File more than 50Mb size gets failed to upload #1759

ChaitanyaPTank opened this issue Apr 6, 2024 · 28 comments

Comments

@ChaitanyaPTank
Copy link
Contributor

Describe the bug
Upload fails for file size more than 50MB.

To reproduce
Steps to reproduce the behavior:

  1. Go to 'Upload tab/menu in side bar'
  2. Try to upload file larger than 50 MB
  3. See error

Expected behavior
Media should get uploaded successfully.

Screenshots
image
image

Environment

  • OS: Running in Docker

Additional context
Logs...

127.0.0.1 - - [06/Apr/2024:18:29:33 +0000] "GET / HTTP/1.1" 200 2224 "-" "curl/7.64.0"
192.168.0.110 - - [06/Apr/2024:18:33:06 +0000] "GET /build/assets/AboutKoelModal.44cf1604.css HTTP/1.1" 200 778 "http://192.168.0.102:8111/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36"
192.168.0.110 - - [06/Apr/2024:18:33:06 +0000] "GET /build/assets/logo.cf71f526.js HTTP/1.1" 200 354 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36"
192.168.0.110 - - [06/Apr/2024:18:33:06 +0000] "GET /build/assets/AboutKoelModal.342f3cc6.js HTTP/1.1" 200 2065 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36"
192.168.0.110 - - [06/Apr/2024:18:33:06 +0000] "GET /build/assets/logo.c97b706c.svg HTTP/1.1" 200 2472 "http://192.168.0.102:8111/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36"
192.168.0.110 - - [06/Apr/2024:18:33:06 +0000] "GET /build/assets/keycdn.4eda7c5b.svg HTTP/1.1" 200 2525 "http://192.168.0.102:8111/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36"
192.168.0.110 - - [06/Apr/2024:18:33:06 +0000] "GET /build/assets/render.fa026c32.svg HTTP/1.1" 200 1919 "http://192.168.0.102:8111/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36"
192.168.0.110 - - [06/Apr/2024:18:33:06 +0000] "GET /build/assets/what-the-diff.017b5dab.svg HTTP/1.1" 200 1683 "http://192.168.0.102:8111/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36"
127.0.0.1 - - [06/Apr/2024:18:34:34 +0000] "GET / HTTP/1.1" 200 2224 "-" "curl/7.64.0"
[Sat Apr 06 18:38:31.063692 2024] [php:warn] [pid 18] [client 192.168.0.110:50695] PHP Warning:  POST Content-Length of 57213847 bytes exceeds the limit of 52428800 bytes in Unknown on line 0, referer: http://192.168.0.102:8111/
192.168.0.110 - - [06/Apr/2024:18:38:31 +0000] "POST /api/upload HTTP/1.1" 413 2449 "http://192.168.0.102:8111/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36"```
@ChaitanyaPTank
Copy link
Contributor Author

I searched for the issue and most says this is about php.ini file or Apache server config but running php -ini | grep upload_max indicates that max allowed size is 800MB

@phanan
Copy link
Member

phanan commented Apr 11, 2024

This is 99.99% somehing your config, which I can't help you debug, unfortunately. With PHP it's both upload_max_filesize and post_max_size, for example. Or you might be using a different config.

@ChaitanyaPTank
Copy link
Contributor Author

ChaitanyaPTank commented Apr 11, 2024 •
edited
Loading

Thank you for your response.

I am running koel using docker-compose, can you please point to the config file I should change ? I am not familiar with PHP, however I tried changing php.ini, php.ini-development and php.ini-production inside docker container and restarted with docker-compose restart however it is not working.

Edit: running php --ini in container gives below output...

Loaded Configuration File:         /usr/local/etc/php/php.ini
Scan for additional .ini files in: /usr/local/etc/php/conf.d
Additional .ini files parsed:      /usr/local/etc/php/conf.d/docker-php-ext-bcmath.ini,
/usr/local/etc/php/conf.d/docker-php-ext-exif.ini,
/usr/local/etc/php/conf.d/docker-php-ext-gd.ini,
/usr/local/etc/php/conf.d/docker-php-ext-pdo_mysql.ini,
/usr/local/etc/php/conf.d/docker-php-ext-pdo_pgsql.ini,
/usr/local/etc/php/conf.d/docker-php-ext-pgsql.ini,
/usr/local/etc/php/conf.d/docker-php-ext-sodium.ini,
/usr/local/etc/php/conf.d/docker-php-ext-zip.ini

@phanan
Copy link
Member

phanan commented Apr 11, 2024 via email

@ChaitanyaPTank
Copy link
Contributor Author

I did and it shows below...
image
image

@ChaitanyaPTank
Copy link
Contributor Author

So, it appears that .htaccess was overriding these properties hence Local Value is 50M.

Currently I solved it by changing value in .htaccess file and restarted using docker-compose restart. Which means once container is destroyed, those changes will be gone.

Do we have any better way for this ? Can we handle it using an environment variable ?

I will be glad to work on it and raise PR once approach is decided.

Thanks.

@phanan
Copy link
Member

phanan commented Apr 12, 2024

Good catch! Yeah, I think making it configurable with a sensible default would be the best solution. Something like:

// .env.example

MAX_UPLOAD_SIZE =  # Max upload size, for example '50M'

// config/koel.php
[
    ...
    'max_upload_size' => env('MAX_UPLOAD_SIZE', '50M'),
    ...
],


// AppServiceProvider.php::boot()

init_set('upload_max_filesize', config('koel.max_upload_size'));
init_set('post_max_size', config('koel.max_upload_size'));

Would love to receive a PR, of course!

@ChaitanyaPTank
Copy link
Contributor Author

ChaitanyaPTank commented Apr 14, 2024 •
edited
Loading

Thank you for the code snippets, I just tried to clone the master branch and run the server but unable to get it running.

I have looked through the Developer Documentation, but it just says to run npm run dev. Can you please give me list of the things I will need to install ? Here is what I have done up-to now...

  1. Installed PHP
  2. Installed Composer
  3. Run php composer.phar install
  4. Run npm run dev (it gave error about some missing extension which I installed with apt)
  5. Got server running but when accessing http://localhost:8000/api/ping it gives below error
SQLSTATE[HY000] [2002] Connection refused (SQL: select * from `personal_access_tokens` where `token` = e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 limit 1)

@phanan
Copy link
Member

phanan commented Apr 14, 2024 via email

@ChaitanyaPTank
Copy link
Contributor Author

ChaitanyaPTank commented Apr 14, 2024 via email

@phanan
Copy link
Member

phanan commented Apr 14, 2024 via email

@ChaitanyaPTank
Copy link
Contributor Author

I have never worked with PHP and have familiarity with JavaScript, hence I have not idea how things work here. I will try to get server running.

@ChaitanyaPTank
Copy link
Contributor Author

ChaitanyaPTank commented Apr 14, 2024 •
edited
Loading

If I use DB_CONNECTION=sqlite then it gives below error,

SQLSTATE[HY000]: General error: 1 no such table: personal_access_tokens (SQL: select * from "personal_access_tokens" where "token" = e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 limit 1)

Edit: I have also not worked with SQL much.

@ChaitanyaPTank
Copy link
Contributor Author

Finally got server running after using sqlite-persistant as a db and running php artisan migrate. However what will be the initial user to login ?

@ChaitanyaPTank
Copy link
Contributor Author

I tried to do ini_set but the changes were not shown in phpinfo() I think it can not be setup from php code as per this answer.

If that's true, can we do it in .htaccess somehow by retrieving the environment variable and set value ?

@phanan
Copy link
Member

phanan commented Apr 14, 2024 via email

@phanan
Copy link
Member

phanan commented Apr 15, 2024 via email

@ChaitanyaPTank
Copy link
Contributor Author

I am not sure how it's done or to say where it's configured to use .env.example file during build/installation. Here is what I have understood...

  1. Create .htaccess.example file and write upload_max_size and post_size_max rules having value of {ENV:MAX_UPLOAD_SIZE}
  2. Add it to git.
  3. Replace it during installation (don't know how)

One thought I am having is that if people write invalid rules in .htaccess.example then will it show error in the same terminal ? Because without familiarity with PHP or apache, it will be little difficult to find what's wrong in the config. In summary I think people can easily shoot their foot in this approach (please correct me if I have misunderstood).

@phanan
Copy link
Member

phanan commented Apr 15, 2024 via email

@ChaitanyaPTank
Copy link
Contributor Author

Makes sense,

So How to implement it ? as I said above ?

@phanan
Copy link
Member

phanan commented Apr 15, 2024 via email

@ChaitanyaPTank
Copy link
Contributor Author

ChaitanyaPTank commented Apr 23, 2024 •
edited
Loading

I have

  • added .htaccess.example besides .env.example and put
  • "@php -r \"copy('.htaccess.example', './public/.htaccess');\"" command in composer.json

Anything else todo ?

@phanan
Copy link
Member

phanan commented Apr 23, 2024 via email

@ChaitanyaPTank
Copy link
Contributor Author

No I have not raised PR because I have not tested it locally. What I am wondering is, will this approach work for docker deployment ?

@phanan
Copy link
Member

phanan commented Apr 23, 2024 via email

@ChaitanyaPTank
Copy link
Contributor Author

Yes, I have tested and it updates file.

@ChaitanyaPTank ChaitanyaPTank mentioned this issue Apr 23, 2024
Merged
@ChaitanyaPTank
Copy link
Contributor Author

Hi @phanan I have raised PR, please have a look.

@ChaitanyaPTank
Copy link
Contributor Author

Solved by #1762

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@phanan @ChaitanyaPTank