BUG: Can't upload folder with 300K files.

[andresmrm]

Describe the bug
Trying to upload a local folder with ~300K files just hangs forever. No file is uploaded. Progress bar stays at 0.

To Reproduce
Steps to reproduce the behavior:

1. Go to folder in Aleph.
2. Click on 'Upload'
3. Click on 'click here' to upload a local folder.
4. Click accept.
5. Wait for hours, no progress bar increase.

Expected behavior
Files uploaded.

Aleph version
3.15.1

Screenshots

After more than 12 hours.

Additional context
Hi! Uploading a folder with 8K files worked, but not another with 300K files. They are all PDFs, about 300KB each. The queue seems to remain empty (no file being processed).

I installed Aleph on a VPS, following the production deploy guide from the docs.
Changed the docker compose a little to add a Caddy as reverse proxy. Is that expected? I found nothing about it in the docs, but the compose seems to be missing a reverse proxy.

Deployment: docker-compose up --scale ingest-file=10 --scale convert-document=5 --scale worker=3 -d
VPS: 10 cores, 60GB memory, 800GB nvme
Browsers tested: Chrome 117 and Firefox 118.

I work for an NGO of investigative journalists, mostly about environment preservation and human rights. If you could invite me to your Slack so we could discuss this and other issues we would be very grateful.

Thanks for the attention!

[stchris]

Thanks for reporting this issue @andresmrm ! We think our community Slack is better suited to help you fix this issue. I have also filed #3384 to improve our documentation and will close this issue for now.