Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

280 remove file io #281

Merged
merged 11 commits into from Nov 3, 2019
Merged

280 remove file io #281

merged 11 commits into from Nov 3, 2019

Conversation

@ethanbsmith
Copy link
Contributor

@ethanbsmith ethanbsmith commented Oct 17, 2019

removed disk i/o from critical path of functions that fetch data form the internet. these functions now operate entirely in memory

main drivers for this:

  1. the original process created significant contention with other disk i/o operations. this was particularly apparent on large batch downloads with older, slower disks and database servers
  2. the implementation pattern was inconsistent. some functions used an entirely in-memory path, while others used disk i/o with temporary files. all functions now consistently use the in-memory pattern

Fixes #280

ethanbsmith added 6 commits Sep 23, 2018
bring fork up to date
This reverts commit 9cf41d9, reversing
changes made to b0ab966.
repalced all for download.file and curl_fetch_disk. left  calls in loadSymbol.r alone
@ethanbsmith ethanbsmith marked this pull request as ready for review Oct 17, 2019
@ethanbsmith ethanbsmith reopened this Oct 17, 2019
ethanbsmith added 2 commits Oct 17, 2019
@joshuaulrich
Copy link
Owner

@joshuaulrich joshuaulrich commented Oct 26, 2019

Hi @ethanbsmith, thanks for working on this PR! Is it ready for review?

@ethanbsmith
Copy link
Contributor Author

@ethanbsmith ethanbsmith commented Oct 26, 2019

@joshuaulrich joshuaulrich merged commit 1739987 into joshuaulrich:master Nov 3, 2019
1 check passed
1 check passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@joshuaulrich joshuaulrich added this to the Release 0.4-16 milestone Feb 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

2 participants
You can’t perform that action at this time.