New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Files and folders generated in Target with 'BeforeTarget=PrepareForPublish' aren't copied from wwwroot to output #114

Closed
damienpontifex opened this Issue Feb 3, 2017 · 15 comments

Comments

Projects
None yet
@damienpontifex
Copy link

damienpontifex commented Feb 3, 2017

See initial issue from dotnet/cli#5498 where files/folders generated in target with a dependency of BeforeTargets="PrepareForPublish" where the folder doesn't exist at the start of publish aren't copied to publish output.

There is a current workaround published by @dsplaisted dotnet/cli#5498 (comment) on the original issue

@vijayrkn suggested this should be able to be calculated in the target without this workaround and so raising the issue here.

Original Issue description

Steps to reproduce

See sample project for reproduction here https://github.com/damienpontifex/NotPublishingGeneratedFiles

<Target Name="PrepublishScript" BeforeTargets="PrepareForPublish">
    <Exec Command="npm install" />
    <Exec Command="npm run build" />
</Target>

npm build script

"scripts": {
    "build": "tsc"
}

tsconfig.json has the property outFile = wwwroot/ts/app.js

PrepareForPublish compiles typescript into wwwroot/ts folder.

This folder is also removed on dotnet clean and ignored by git

Steps to reproduce:

  1. dotnet restore
  2. dotnet publish -c Release -o <your output dir>

If you re-run the publish step, the files are appropriately copied as they existed at the time the publish was started.

If you run dotnet clean to clear away generated files and re-run publish step, they are again missed when copying to output as per original restore and publish.

I have also tried changing the PrepublishScript target value of BeforeTargets to ComputeFilesToPublish from looking at targets here https://github.com/dotnet/sdk/blob/master/src/Tasks/Microsoft.NET.Build.Tasks/build/Microsoft.NET.Publish.targets but with no success

n.b. If I generate the files into a folder that does exist and has content (e.g. wwwroot/js has site.js in it) then the copy works fine. It's only if I generate both a new folder and the files inside it in this step

Expected behavior

Files generated into wwwroot to be in output directory under wwwroot

Actual behavior

Files are generated in source wwwroot folder, but not copied into publish output wwwroot folder

Environment data

dotnet --info output:
.NET Command Line Tools (1.0.0-rc3-004530)

Product Information:
Version: 1.0.0-rc3-004530
Commit SHA-1 hash: 0de3338607

Runtime Environment:
OS Name: Mac OS X
OS Version: 10.12
OS Platform: Darwin
RID: osx.10.12-x64
Base Path: /usr/local/share/dotnet/sdk/1.0.0-rc3-004530

@martincostello

This comment has been minimized.

Copy link
Contributor

martincostello commented Feb 14, 2017

I'm having similar problems, but not just with generated files. I'm also finding that some files that are on-disk in the repository are also not being published; in my case they are files from the .well-known folder.

@dsplaisted

This comment has been minimized.

Copy link
Contributor

dsplaisted commented Feb 14, 2017

@martincostello Folders starting with a . are intentionally not included, in order to avoid publishing folders like .git and .vs. You can include them yourself like this:

<ItemGroup>
  <Content Include=".well-known/**" CopyToPublishDirectory="PreserveNewest" />
</ItemGroup>
@dabide

This comment has been minimized.

Copy link

dabide commented Feb 15, 2017

The workaround by @nil4 in dotnet/cli#5498 (comment) works. I modified it a bit with help from @dasMulli on Slack. Add the following to the global <PropertyGroup>:

  <GeneratedItemPatterns>wwwroot/dist/**;wwwroot/lib/**;wwwroot/jspm_packages/**</GeneratedItemPatterns>
  <DefaultItemExcludes>$(DefaultItemExcludes);$(GeneratedItemPatterns)</DefaultItemExcludes>

Then add the pre-publish target:

  <Target Name="PrepublishScript" BeforeTargets="PrepareForPublish">
    <Exec Command="bower install" />
    <Exec Command="npm install" />
    <Exec Command="gulp build" />
    <ItemGroup>
      <_GeneratedFiles Include="$(GeneratedItemPatterns)" />
      <ContentWithTargetPath Include="@(_GeneratedFiles->'%(FullPath)')" RelativePath="%(_GeneratedFiles.Identity)" TargetPath="%(_GeneratedFiles.Identity)" CopyToPublishDirectory="PreserveNewest" />
    </ItemGroup>
  </Target>
@tuespetre

This comment has been minimized.

Copy link
Contributor

tuespetre commented Feb 28, 2017

Everyone, I've found through trying to work around dotnet/cli#5779 that I am not having this issue using dotnet CLI version 1.0.0-rc4-004911. I'm wondering if this was really the issue at all or if the issue was really the Exec issue all along.

@dasMulli

This comment has been minimized.

Copy link
Contributor

dasMulli commented Mar 5, 2017

@tuespetre I see no change using 1.0.0-rc4-004915.. maybe while playing around the files have already been generated in between multiple dotnet publish calls?
The issue only affects files that did no exist prior to the target being run, which likely happens on clean builds on a build server..

@tuespetre

This comment has been minimized.

Copy link
Contributor

tuespetre commented Mar 5, 2017

I was deleting the publish folder and watching it get recreated. I dunno, stuff's just weird in general.

@svallis

This comment has been minimized.

Copy link

svallis commented Mar 16, 2017

I'm currently working around this issue by moving the npm install and gulp commands out into my Dockerfile where they run before dotnet publish. I somewhat prefer these processes being strongly tied to publishing rather than the Docker configuration, but it's a decent solution that results in no wasted execution for the time being.

@ctolkien

This comment has been minimized.

Copy link

ctolkien commented Mar 20, 2017

We're hitting the same issue here, where by:

<Target Name="PrepublishScript" BeforeTargets="PrepareForPublish">
    <Exec Command="npm install" />
    <Exec Command="npm run webpack" />
  </Target>

npm run webpack creates a bundle.js file in wwwroot/js which is not included in the publish output.

@MaximBalaganskiy

This comment has been minimized.

Copy link

MaximBalaganskiy commented Mar 20, 2017

@ctolkien #114 (comment) this works
one issue though, you might want to modify it if you're using revisioned bundles. This is what I've come up with

	<Target Name="PrepublishScript" AfterTargets="ComputeFilesToPublish">
		<!-- Exclude old script and font files from publish output-->
		<ItemGroup>
			<PrevScriptFiles Include="wwwroot\dist\**" />
			<ResolvedFileToPublish Remove="@(PrevScriptFiles->'%(FullPath)')" />
		</ItemGroup>

		<Exec Command="npm install" />
		<Exec Command="npm run-script jspm -- install -y" />
		<Exec Command="npm run-script gulp -- prodBundle" />

		<!-- Include the newly-built files in the publish output -->
		<ItemGroup>
			<DistFiles Include="wwwroot\dist\**" />
			<ResolvedFileToPublish Include="@(DistFiles->'%(FullPath)')" Exclude="@(ResolvedFileToPublish)">
				<RelativePath>%(DistFiles.Identity)</RelativePath>
				<CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory>
			</ResolvedFileToPublish>
		</ItemGroup>
	</Target>

tuespetre added a commit to tuespetre/sdk that referenced this issue Mar 20, 2017

tuespetre added a commit to tuespetre/websdk that referenced this issue Mar 21, 2017

Update Microsoft.NET.Sdk.Web.ProjectSystem.targets
Add a web-specific target to ensure that generated content is copied to output.

Fixes aspnet#114

tuespetre added a commit to tuespetre/websdk that referenced this issue Mar 23, 2017

Update Microsoft.NET.Sdk.Web.ProjectSystem.targets
Add a web-specific target to ensure that generated content is copied to output.

Fixes aspnet#114

tuespetre added a commit to tuespetre/websdk that referenced this issue Mar 23, 2017

Update Microsoft.NET.Sdk.Web.ProjectSystem.targets
Add a web-specific target to ensure that generated content is copied to output.

Fixes aspnet#114

tuespetre added a commit to tuespetre/websdk that referenced this issue Mar 23, 2017

tuespetre added a commit to tuespetre/websdk that referenced this issue Mar 23, 2017

@ctolkien

This comment has been minimized.

Copy link

ctolkien commented Apr 20, 2017

Is there some reference for when we can expect this fix to drop in .. whatever product it's released in?

@vijayrkn

This comment has been minimized.

Copy link
Contributor

vijayrkn commented Apr 21, 2017

This change is already available in the latest CLI (and should be available in an upcoming VS release).

@ctolkien

This comment has been minimized.

Copy link

ctolkien commented Jun 22, 2017

Sooo I just tried this again with the 1.0.4 SDK and the problem still seems to exist. Any clarity as to wether it has landed or not?

@vijayrkn

This comment has been minimized.

Copy link
Contributor

vijayrkn commented Jun 22, 2017

the changes are available in the 2.0 cli. You can install the 2.0 cli on the machine and it will work from both VS and CLI.
Once 2.0 cli is installed, VS will start using the 2.0 cli so you will get the same behavior from commandline & VS.

@ctolkien

This comment has been minimized.

Copy link

ctolkien commented Jun 22, 2017

But I can't install it on my CI machine...

Will sit tight for now.

@t0xicDen

This comment has been minimized.

Copy link

t0xicDen commented Sep 13, 2017

Who still has similar issue try to read my comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment