Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deleting large non-binary file takes extremely long time #1202

Closed
bdukes opened this issue Mar 25, 2020 · 3 comments · Fixed by #1210
Closed

Deleting large non-binary file takes extremely long time #1202

bdukes opened this issue Mar 25, 2020 · 3 comments · Fixed by #1210
Labels
needs triage Awaiting triage

Comments

@bdukes
Copy link

bdukes commented Mar 25, 2020

We have a generator which includes a local copy of Yarn (the result of yarn set version ^1), which is a 5MB JavaScript file. After updating this file to the latest version, the delete of the file is taking a very long time (close to an hour). After debugging the process, it looks like the conflict detection is looping through the file to calculate a diff.

Is it possible to short-circuit that conflict check for deletions, or configuration I could do to skip it for specific files?

@bdukes bdukes added the needs triage Awaiting triage label Mar 25, 2020
@mshima
Copy link
Member

mshima commented Mar 25, 2020

Can't you directly copy to the real path at end step?

@bdukes
Copy link
Author

bdukes commented Mar 25, 2020

So instead of using this.fs.delete in writing(), just use fs.unlink directly in end()? That does seem like it would be a workable workaround.

@mshima
Copy link
Member

mshima commented Mar 25, 2020

Exactly.
The api to ignore conflicts can be done, and should be kind of easy, it would use transformStream.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs triage Awaiting triage
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants