[RFC] copy recursively

[schnittstabil]

At least copying recursively makes it necessary to use a two step approach:

1. gather all files and folders to process
2. do the actual copy

Substantial advantages:

• determine errors due to mutually recursively linked directories in advance (see test below)
• remove duplicated tasks (same source and destination)
• determine copy tasks with same destination to throw errors in advance
• calculate the total size during the first step, and emitting the progress during the second step

The current approach of the first step:

1. use globby to gather all files and directories
2. for each directory path: re-glob with path + '/**/*'

@ALL Comments on all aspects are highly welcome.

[jamestalmage]

When globbing the contents of a directory with /somedir/**/*, you want to avoid adding additional glob tasks for /somedir/subdir/**/*. See how I solved that here.

Also, since you are going to be calling glob/globby many times, you want store and pass in these caching options.

[YurySolovyov]

calculate the total size during the first step, and emitting the progress during the second step

cp-file now has that.

determine copy tasks with same destination to throw errors in advance

Can be racy.

[sindresorhus]

When globbing the contents of a directory with /somedir/**/*, you want to avoid adding additional glob tasks for /somedir/subdir/**/*. See how I solved that here.

@jamestalmage Is that something that could maybe be added to globby? Sounds generally useful.

[sindresorhus]

I really like the idea of a two step process. My only concern is that it might cause some race issues. What if something in the filesystem hierarchy changes between step 1 and 2? For example, we glob foo/bar.jpg, but by the time we start copying in step 2 that file no longer exist or has moved. How should that be handled? It might be a non-issue as the globbing itself is already racy (node-glob states that in its docs). Just thought I'd bring it up.

[sindresorhus]

for each directory path: re-glob with path + '/*/'

Would be better to do this in globby: sindresorhus/globby#33

[sindresorhus]

@schnittstabil What are your thoughts on this now?

[sindresorhus]

Closing in favor of #61