Efficiency enhancement + Progress.js failed for streams that not support 'clearLine' method. #7
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Enhance efficiency for large storage by filter files by modified time. Now trash directory will not be removed every time. Add profiler to evaluate script efficiency. Add logs to be more informative.
…hod like 'clearLine'.
mehwww
added a commit
that referenced
this pull request
Jul 25, 2015
Efficiency enhancement + Progress.js failed for streams that not support 'clearLine' method.
|
This will make those delete objects undetectable which against the 'sync'. What do you mean 3.4G file which is one big file or a mass of small file? Honestly I didn't consider complex situation at first, maybe we can specified the program behavior by adding some command line flag. Thanks for the pr though! |
|
I see, 3.4G is the total size of a mass of small file. Sorry for the bug introduced by my pr. In my case, deletion is not a problem for the 'sync', just like 'rsync' service without the '--delete' option. I think a command line argument for such application is a good idea. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For larger storage syncronization (my case containes 3.4G file), delete "trash" and recreate md5 signature for all file consumes too much time. I invested into the ncp code and opened options "clobber" and "modified" to utilize the ability to filter the files by comparing the modification time of original file and "trash" file, which reduce "trash" time to 30s in my case.
PS. I add a profiler to show actural cost for every steps.
PS. Important! When I redirect stdout to file, which not support "clearLine" clearly, osync crashes. This request solved this bug.