Stream tarball downloads and file uploads to avoid memory buffering#73
Merged
Stream tarball downloads and file uploads to avoid memory buffering#73
Conversation
Member
ericmj
commented
Mar 9, 2026
ericmj
commented
- Stream tarball downloads to disk via Store.get_to_file instead of loading full binary into memory
- Stream file uploads via put_file! to avoid File.read! on every doc
- Add TmpDir GenServer for process-based temp file cleanup
- Add put_file! to Store behaviour and backends (GS, S3, Local)
- Fix flaky debouncer test grace time
- Fix unsafe paths test to assert on error logs
- Stream tarball downloads to disk via Store.get_to_file instead of loading full binary into memory - Stream file uploads via put_file! to avoid File.read! on every doc - Add TmpDir GenServer for process-based temp file cleanup - Add put_file! to Store behaviour and backends (GS, S3, Local) - Fix flaky debouncer test grace time - Fix unsafe paths test to assert on error logs
Use GenServer.call instead of cast in track/1 so the monitor is set up synchronously before the caller can exit. Replace Process.sleep with deterministic waiting: monitor the task, wait for :DOWN, then sync with the GenServer via :sys.get_state to ensure cleanup has been processed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.