Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

At the moment this part synchronously reads every asset file from disk to memory, potentially blocking the process for a long time. I think we should consider preparing for a few optimizations (some of these we've already been doing in expo publish): #167

Closed
jkhales opened this issue Dec 16, 2020 · 0 comments
Assignees

Comments

@jkhales
Copy link
Contributor

jkhales commented Dec 16, 2020

At the moment this part synchronously reads every asset file from disk to memory, potentially blocking the process for a long time. I think we should consider preparing for a few optimizations (some of these we've already been doing in expo publish):

  • Parallelize hash calculation
  • Parallelize uploading
  • Pipe the file read stream to upload request (i.e. uploadWithPresignedPostAsync(fs.createReadStream(filename)), ...)
  • Possibly show a progress bar while reading/hashing/uploading files in the future

Perhaps we should already make RawAsset have a filename field instead of buffer. This way we don't keep every asset of the app (which could easily be hundreds of megabytes) in memory for the duration of the command, but instead they're only read as needed and can be garbage collected as soon as that particular asset is done.

Originally posted by @fson in #147 (comment)

@jkhales jkhales self-assigned this Dec 16, 2020
@jkhales jkhales closed this as completed Dec 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant