Building solution with /t:Package does not include all files for services

[aL3891]

Hello, i'm trying to build a solution containing multiple sf apps and various services using msbuild. first of I build all the services like so

msbuild /m /p:"Configuration=AppsDebug;Platform=x64;OutputPath=bin\x64\Debug\;IntermediateOutputPath=obj\x64\Debug\" 

Then I run the package target on only the sf apps:

msbuild /m /p:"Configuration=FabricDebug;Platform=x64;OutputPath=bin\x64\Debug\;IntermediateOutputPath=obj\x64\Debug\" /t:Package 

When this is done there are no errors and the pkg folders are created correctly, however a lot of dlls are missing from the package folder, all the service fabric dlls are included but any other dlls that the service references are not.

if I run the package target directly on the sfproj file, not the solution, everything works, but i'd like to run it on the solution because its a lot faster. is that not supported or am I missing something else?

[aL3891]

I've also tried running it as one step but that doesn't work since the apps projects don't have "Package" targets, but maybe there's a way around that.. it feels like this should work though

[benaadams]

This the packaging issue? NuGet/Home#4853

[aL3891]

hmm interesting, I remember seeing this a while back actually I just didn't connect the dots :) i'll try and see if I can apply the workaround

[aL3891]

I've been reading though that other issue and I wonder if their solution is applicable to this problem because afaik, the service fabric package target is not using nuget to package(right?), also, we're not using the new .csproj here just the old one, although we're still using msbuild 15(.1.1012.6693)

Also, my problem is that the project refernces are not included, the framework (and service fabric) assemblies are included

[aL3891]

if anyone is interested i worked around my issue (building multiple apps with common project references in parallel) by building each sfproj file but also specifying the output and intermediate paths and setting different ones for each app. that avoids the read/write conflicts that would happen otherwise.

its still kind of sub optimal because all projects are always built, and the project references are built once for every app, so you need a fast harddrive :)

It seems like it should work with msbuild too though, would still love to get a comment from the team.

[aL3891]

Any thoughts on this one?

[mikkelhegn]

@ravipal

[ravipal]

@aL3891, Can you please provide the cmd that you use to build multiple *sfproj? Are you saying the common projects are built once for each *sfproj, instead of skipping the build.

[aL3891]

we just do msbuild <sfproj> /t:package on each project. this causes all dependencies to be rebuilt each time.

This could be avoided if we could do msbuild <sln file> instead, but that does not work as i've detailed above

[ravipal]

As you figured out, the package target exist only on sfproj, so msbuild on solution file is not possible.

[aL3891]

Well, solutions doenst have targets, its perfectly possible to invoke the package target on a solution, as long as the solution configuration only specifies sfprojects. however as i've detailed in this issue, doing that causes the package target to skip a number of files when copying the service build output over.

That is why i've raised this issue, to enable packaging the whole solution at once.

[ravipal]

Got it. We will look into enabling package from solution.

[mikkelhegn]

@dbreshears

[dbreshears]

We currently have this on our tooling backlog, but I don't have a timeframe for investigating at the moment as it appears nothing is blocked. Will admit that this seems broken that references are not packaged correctly when packaged from the solution level

[aL3891]

I've actually I've taken things into my own hand over at https://github.com/aL3891/ServiceFabricSdkContrib with my own packaging targets that run during build :) feel free to check them out!

[murrays]

I'm seeing a similar issue when packaging for sf deploy. We have mixed framework standard projects. I have converted problem .sfproj to by reference projects and it appears to build ok. I am seeing framework overrides when I use standard packages, notably System.Net.Http
We are on framework 4.6.1 (we tried 4.7.1 but thats not currently Azure supported, go figure).
So for framework, assembly version for System.Net.Http is 4.0.0.0
The package assembly version in System.Net.Http.4.3.3 is 4.1.1.2 i.e. higher than 4.0.0.0
However the compiler chooses System.Net.Http from its extension dir (i.e. C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\Microsoft\Microsoft.NET.Build.Extensions\net461\lib\System.Net.Http).
The extension System.Net.Http is at version 4.2.0.0 which is then choosen during build. Looking at the bin directory, sure enough I see System.Net.Http at 4.2.0.0 and the automatic binding redirects point to 4.2.0.0. So far so good!

Now I package to SF deployment using MSBUILD /t:Package. This copies most assemblies from the the bin directory EXCEPT framework assemblies. What it does is copy the System.Net.Http at 4.0.0.0 from the framework directory! So now I have a Code directory with a System.Net.Http at version 4.0.0.0 but a binding redirect telling it to load 4.2.0.0. Since 4.2.0.0 doesn't exist, I get a loading exception.

Does anyone else see this? Is there any way to make MSBUILD just copy the bin dir rather than using framework assemblies?

The package operation also fails to copy netstandard.dll so I guess this is a general issue rather than a framework specific issue.

Any clues?

I'm using VisualStudio 15.5.7

[murrays]

Ok I got some more observations and a potential solution tho its not nice.
Looking at the verbose output I can now see that MSBUILD is copying from the framework directory for things like System.Net.Http, even though during build it has selected the Build Extensions one and created a binding redirect to 4.2.0.0

I have created a post package target that copies all binaries into the pkg\Code directory basically overriding what MSBUILD /t:Package has done. This has worked for me locally deploying to SF dev node and I'll be trying shortly on an actual azure instance. For those of you interested the targets file is attached below. One warning though, it does increase SF package size by quite a bit and consequently impacts deploy times.

ServiceFabricPackageHelper.zip

To add to your sfproj here you just do an import like this:

[ravipal]

Can you please try it by setting the <PackageOutputFolder>true</PackageOutputFolder> to *.sfproj to change the package behavior to include all the files from output folder? This new feature is available in the latest 1.6.5 nuget package.

[murrays]

Hi Ravipal.

Much appreciated that flag seems to work a treat. Wish I had known about this yesterday!! Personally as we all move towards a 'standard/core' world I would have thought this should be the default.

Many thanks though I will be updating our builds to use this flag.

[murrays]

Were still having issues with build time and runtime resolution of the build extension assemblies. Examples of this are where a dependent build is up referenced to 4.2.0.0 of System.Net.Http but appears to pass a forward reference of 4.0.0.0 to dependent assemblies which then error as that conflicts with their 4.2.0.0! Some of our IIS services are failing to demand load 4.0.0.0 even though we have a binding redirect downgrade in the Web.config.

The only reliable way we have found to work around this is to copy the build extension assemblies to a 'known' location and directly reference them. Using this approach all builds include the assembly in the build output and resulting packages. All builds reference the extension assembly hence there is no conflict at build or runtime. Since the assembly is no longer tagged as 'framework' it appears in the bin directory.

This essentially bypasses the build time resolution which seems to be essentially broken in this respect.

Any comments on the validity of this approach?

[bledbet]

Is there a current comprehensive list of steps, versions, whatever else is required to get this to work?

I have this working when I package and publish from my local - without adding a nuget reference to NetStandard.Library, I just added the <Reference Include="netstandard" /> entry to all of my .csproj files, set them all to "Copy Local"=true, and added the <PackageOutputFolder>true</PackageOutputFolder> to my sfproj. Whether I package with VS, or with msbuild commandline, I get the netstandard.dll in my Service Fabric code packages. However, when I commit this to Azure Devops repo and use a build agent (I've tried our on-prem agent and our Hosted agent with VS 2017), the netstandard.dll does not end up in the code package, and my services fail to load with

System.IO.FileNotFoundException: Could not load file or assembly 'netstandard, Version=2.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The system cannot find the file specified.