Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timeout when uploading large files to cloud storage #81

Closed
reinhart1010 opened this issue Sep 9, 2022 · 12 comments
Closed

Timeout when uploading large files to cloud storage #81

reinhart1010 opened this issue Sep 9, 2022 · 12 comments
Labels
bug Something isn't working

Comments

@reinhart1010
Copy link

Hi, I decided to configure Gokapi to use Cloudflare R2, with auto set as the S3 region. When the file is successfully uploaded to the Gokapi server, the server returns an error (504 Gateway Timeout as I used Nginx).

Looking at the logs, I see the following warning, in which I have configured my server with the correct access key and secret access key. Requesting a new one from Cloudflare doesn't fix the issue as well:

caused by: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.

@reinhart1010
Copy link
Author

This issue also persists by using us-east-1 instead of auto.

@Forceu
Copy link
Owner

Forceu commented Sep 11, 2022

I just tried setting up Gokapi with R2 and could not reproduce the problem. Are you using the correct endpoint (https://<ACCOUNT_ID>.r2.cloudflarestorage.com)?

@reinhart1010
Copy link
Author

Apparently I inputted the bucket URL as <ACCOUNT_ID>.r2.cloudflarestorage.com) (without the https://).

@reinhart1010
Copy link
Author

reinhart1010 commented Sep 13, 2022

Edit: The issue still persists even when using "https://", and apparently only occur on large files (e.g. a ~80MB APK file). Small files such as PNG images can still be uploaded successfully.

@reinhart1010 reinhart1010 reopened this Sep 13, 2022
@Forceu Forceu changed the title Cloudflare R2 and request signing issues Timeout when uploading large files to cloud storage Sep 14, 2022
@Forceu Forceu added the bug Something isn't working label Sep 14, 2022
@Forceu
Copy link
Owner

Forceu commented Sep 14, 2022

Thanks for the feedback! I assume the webserver times out after trying to upload a large file to the cloudserver, which is a blocking call. I might have some time to fix it tomorrow, otherwise it might take a couple of days.

Forceu added a commit that referenced this issue Sep 14, 2022
@Forceu
Copy link
Owner

Forceu commented Sep 14, 2022

This issue has been fixed in 510ee9d. If you are using docker, the image latest-dev contains the fix.

I will probably implement websockets at some point, so that a high timeout is not needed anymore.

@Forceu Forceu pinned this issue Sep 14, 2022
@reinhart1010
Copy link
Author

reinhart1010 commented Oct 3, 2022

I'm currently running the latest stable version for production, but might check out the latest Docker dev version soon

@Forceu Forceu unpinned this issue Feb 25, 2023
@hra42
Copy link

hra42 commented Nov 17, 2023

I run the current latest-dev with Cloudflare R2 storage. When I upload small files <1mb, the upload is fine and everything works. If the uploaded file is >1mb the upload seems to be stuck. After a few minutes the upload is complete and I can share the URL. I tried with a go windows binary as upload and an image. The image seems to be stuck far longer (if it ever completes)

running on arm64

@Forceu
Copy link
Owner

Forceu commented Nov 17, 2023

Thanks for the feedback. Are there any error messages? And I assume you are using the latest version?

@hra42
Copy link

hra42 commented Nov 17, 2023

Thanks for the feedback. Are there any error messages? And I assume you are using the latest version?

When I close the session I get EOF, but thats to be expected, otherwise no erros.
Yes I'm using latest-dev from dockerhub on Raspberry Pi 5 on the latest docker version. Arm64 image.

@hra42
Copy link

hra42 commented Nov 18, 2023

now its even stranger, the issue seems to be gone for files <2mb with the latest-dev version. Just repulled the image and the issue is gone now.

This is might be caused by slow upload speeds from ISP, I would guess.

@Forceu
Copy link
Owner

Forceu commented Feb 7, 2024

This bug appears to be fixed. If you still experience problems with the latest version, please post here and I will reopen the issue.

@Forceu Forceu closed this as completed Feb 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants