You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I planned to use this project as a backup endpoint on my Raspberry Pi but noticed that OOM happens about halfway through an upload of 4.7 GB archive (I've build an image for Arm64v8).
The memory usage peak of this container was 3.8 GB before being killed due to OOM. My Rpi has 4GB RAM.
Due to my lack of knowledge of Go (I'm learning), I cannot say whether is this This seems to be by design (File stored in memory before flushing out to the disk) or there might be a memory leak, but the RAM usage pattern doesn't support that theory well.
The text was updated successfully, but these errors were encountered:
xZero707
changed the title
Upload of large files are out of question, unless machine has enough RAM to accommodate whole file into the memory
Upload of large files is out of question, unless machine has enough RAM to accommodate whole file into the memory
Jul 29, 2021
xZero707
changed the title
Upload of large files is out of question, unless machine has enough RAM to accommodate whole file into the memory
Upload of large files via POST is out of question, unless machine has enough RAM to accommodate whole file into the memory
Aug 1, 2021
@xZero707 thanks for your report.
As you described, the old implementation uses io.ReadAll to read the uploaded file into memory. So this is "expected" behavior.
I've released v2 last year. In v2, I'm using io.Copy instead of io.ReadAll.
If you are still using this server, please consider using v2.
However, I didn't test such situation so this issue can be still there. If so, please mention me.
I planned to use this project as a backup endpoint on my Raspberry Pi but noticed that OOM happens about halfway through an upload of 4.7 GB archive (I've build an image for Arm64v8).
The memory usage peak of this container was 3.8 GB before being killed due to OOM. My Rpi has 4GB RAM.
Due to my lack of knowledge of Go (I'm learning), I cannot say whether is thisThis seems to be by design (File stored in memory before flushing out to the disk)or there might be a memory leak, but the RAM usage pattern doesn't support that theory well.EDIT: Found the issue (or at least one part of it): https://github.com/mayth/go-simple-upload-server/blob/master/server.go#L83
This function seems to load the whole file into the memory.
The text was updated successfully, but these errors were encountered: