EmptyFiles creates a lot of files

[skwasjer]

Is the feature request related to a problem

Is there a way to exclude (specific) empty files?

EmptyFiles creates a lot of files on disk, even if there's no specific need to the project/solution that is transitively consuming this (via eg. DiffEngine/Verify). For one of my (multi-targeted) solutions, I currently count 49 files per test project, per target framework, which equates to over 1400 files. This creates a lot of I/O, even if the files are small.

My project uses none of the file extension specific verifications, it just does plain *.cs or *.json diffs, but still we have to pay the penalty of including all these miscellaneous files.

It was also confusing to find empty.nupkg files after filtering for *.(s)nukpg after dotnet pack-ing my project, hence why I investigated this (but otherwise, this is a separate issue and can be resolved by the consumer).

Describe the solution

Possibly via a custom msbuild property:

• Simplest way would be a complete feature toggle: -p:CopyEmptyFiles=false (where true or not specified means copy all)
• Alternatively, opt-in to certain file extensions based on the project its needs, f.ex.:
  -p:CopyEmptyFilesToOutput=bmp,jpg?

The default behavior would still be to copy all, as currently is the case.

Describe alternatives considered

Is there really even a need to put all these files on disk, all the time, during each build? Could it not be streamed from an embedded resource on demand just before starting a diff tool, for example to a temp file? I am not sure of all the use cases so hard for me to judge.

[SimonCropp]

those files are marked with CopyToOutputDirectory="PreserveNewest" so msbuild should only be copying them once.

is that not the case?

[skwasjer]

Accessing timestamps is still required. I checked with Process monitor, over 30k I/O related events with EmptyFiles in the path on a single build (not rebuild).

But also, do consider CI builds where it is unlikely these intermediates/artifacts are cached at all.

[SimonCropp]

yeah. thats a problem. let me see if i can reduce that

[SimonCropp]

re nupkg and dotnet pack-ing my project, i guess i have not experienced that since i have this in my directory props <PackageOutputPath>$(SolutionDir)nugets</PackageOutputPath>

then i exclude the nugets dir from source control.

this way i always have a single directory that contains only the resulting nugets. and i can use nugets/* to glob them

[skwasjer]

Yea, that's just a side-effect which was easily solved (seen in the referenced PRs/commits) but it is what actually led me to this repository. Initially, I had no idea where these files came from, I just never noticed them before. But like I said in OP, it is a separate issue which does not bother me. At most it may need some documentation.

[skwasjer]

Hey Simon, thanks for addressing the excessive file access issue 👍. It does however only partially resolve my original request of not wanting certain (if any) of these files. In my specific case, I still prefer not to have 1400 files created and sitting in my output folders if I don't need them.

So in case anyone is wondering, I went with including this package explicitly excluding the transitive build assets to get rid of all of them:

<PackageReference Include="Verify.Xunit" Version="30.18.0" />
<PackageReference Include="EmptyFiles" Version="8.13.0" ExcludeAssets="buildTransitive" />

Of course, you cannot use any of the verify/diff API's that need these files.

[SimonCropp]

In my specific case, I still prefer not to have 1400 files created and sitting in my output folders if I don't need them.

yeah. i need to think more on that. i will probably go with the resource suggestion you made. but it will be a breaking change, so i need to spend a little more time ensuring that is the correct approach

[skwasjer]

Sounds good man, I'll keep an eye out. Thanks again and keep up the good work 👍

And let me know if I can be of help.