VS2017 Docker-Compose Project breaks build on command line

[billpratt]

Steps to reproduce

In Visual Studio 2017: File->New Project->ASP.NET Core Web Application
Choose either Web API or Web Application
Check "Enable Docker Support"

Go to command line and run

dotnet restore ./[NAME].sln
dotnet build ./[NAME].sln

Expected behavior

Restore and build work when a Docker-Compose project (.dcproj) is in the solution.
Should work across all platforms that dotnet is supported

Actual behavior

error MSB4019: The imported project "C:\Program Files\dotnet\sdk\1.0.0\Sdks\Microsoft.Docker.Sdk\Sdk\Sdk.props" was not found. Confirm that the path in the declaration is correct, and that the file exists on disk.

PS C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10> dotnet --version
1.0.0

PS C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10> dotnet restore .\WebApplication10.sln
C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10\docker-compose.dcproj : error MSB4019: The imported project "C:\Program Files\dotnet\sdk\1.0.0\Sdks\Microsoft.Docker.Sdk\Sdk\Sdk.props" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.
  Restoring packages for C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10\WebApplication10\WebApplication10.csproj...
  Restoring packages for C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10\WebApplication10\WebApplication10.csproj...
  Restore completed in 785.82 ms for C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10\WebApplication10\WebApplication10.csproj.
  Lock file has not changed. Skipping lock file write. Path: C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10\WebApplication10\obj\project.assets.json
  Restore completed in 1.08 sec for C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10\WebApplication10\WebApplication10.csproj.

  NuGet Config files used:
      C:\Users\foo\AppData\Roaming\NuGet\NuGet.Config
      C:\Program Files (x86)\NuGet\Config\Microsoft.VisualStudio.Offline.config

  Feeds used:
      https://api.nuget.org/v3/index.json
      C:\Program Files (x86)\Microsoft SDKs\NuGetPackages\
      
PS C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10> dotnet build .\WebApplication10.sln
Microsoft (R) Build Engine version 15.1.548.43366
Copyright (C) Microsoft Corporation. All rights reserved.

C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10\docker-compose.dcproj : error MSB4019: The imported project "C:\Program Files\dotnet\sdk\1.0.0\Sdks\Microsoft.Docker.Sdk\Sdk\Sdk.props" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.
  WebApplication10 -> C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10\WebApplication10\bin\Debug\netcoreapp1.1\WebApplication10.dll

Build FAILED.

C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10\docker-compose.dcproj : error MSB4019: The imported project "C:\Program Files\dotnet\sdk\1.0.0\Sdks\Microsoft.Docker.Sdk\Sdk\Sdk.props" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.
    0 Warning(s)
    1 Error(s)

Time Elapsed 00:00:04.20
PS C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10>

Environment data

dotnet --info output:

.NET Command Line Tools (1.0.1)

Product Information:
Version: 1.0.1
Commit SHA-1 hash: 005db40

Runtime Environment:
OS Name: Windows
OS Version: 10.0.14393
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\1.0.1

[livarcocc]

cc @MichaelSimons

This is an SDK that ships only with VS. It is not supported in the CLI.

[billpratt]

Right, completely understand that. By adding the docker-compose project feature to a solution completely breaks running dotnet restore and dotnet build on a solution file. Since the dcproj is a VS2017 feature, I dont expect dotnet to build it, but perhaps ignore it.

The microsoft/aspnetcore-build:1.0-1.1 docker image has "fixed" this by adding the SDK in the dotnet path. Howerver, I hope this is a temporary workaround.

root@8ca61219426a:/usr/share/dotnet# find / -xdev 2>/dev/null -name "Microsoft.Docker.Sdk"
/usr/share/dotnet/sdk/1.0.1/Sdks/Microsoft.Docker.Sdk

[livarcocc]

There is a PR out right now add this SDK to the CLI, which should address this issue.

dotnet/cli#6180

[billpratt]

Great! Thanks for the quick turn around

[jgreene]

It appears this issue was closed since dotnet/cli#6180 is supposed to resolve it. Then dotnet/cli#6180 was closed because it requires the Microsoft.Docker.SDK to be open sourced before it can be merged. So how are users supposed to resolve this issue going forward?

[philcontrolf1]

@jgreene In the short term, you just need to copy the appropriate files from the VS 2017 installation (C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\Sdks\Microsoft.Docker.Sdk) to where dotnet expects to find them (C:\Program Files\dotnet\sdk\1.0.1\Sdks\Microsoft.Docker.Sdk on Windows, /opt/dotnet/sdk/1.0.1/Sdks/Microsoft.Docker.Sdk on *nix). Longer term hopefully the SDK will be open sourced and these files can be distributed...

[billpratt]

@livarcocc I propose that this issue be re-opened since there is no fix currently in place except for manually copying SDK files.

[jgreene]

@philcontrolf1 thank you for the solution, this worked for me

[AlexandreOuellet]

Manually copying the files also worked for me, but I also suggest to reopen this issue, as it is indeed an issue until the files are distributed with the CLI.

[philcontrolf1]

With vague apologies for the self-promotion, having put together a whole pipeline of .NET Core 1.1, Visual Studio 2017 and Docker on TeamCity, I wrote a blog post for work on it - hopefully it will help anyone else trying to do this.

[khayes]

I also ran into this today. Copying the files is non-ideal as we run dotnet restore on a dynamic build system and having to do this on every build agent is awkward.

[philcontrolf1]

@khayes Our build agent bootstrapper copies the files on - see my blog post for details.

[opless]

In the community edition the sdk is:
"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\Sdks\Microsoft.Docker.Sdk"

... and for 2.0.0-preview1-005977 on OSX you need to put the files here:
/usr/local/share/dotnet/sdk/2.0.0-preview1-005977/Sdks/

[rainersigwald]

Since these extensions are installed to VS's copy of msbuild but not dotnet's, can you run the build with msbuild.exe instead of dotnet build?

[philcontrolf1]

@rainersigwald Not on my Linux CI build agents I can't :-)

[srivatsn]

Reopening this issue since it hasn't been resolved yet. I think not being able to build a solution that contains a dcproj without hacks is a significant blocker.

@dazhao-msft - any updates on dotnet/cli#6180

[billpratt]

@srivatsn thank you for re-opening

[dmitry606]

The fix (copying VS Docker SDK to dotnet sdk) does not seem to work for 2.0.0-preview2. I get this error:
C:\Program Files\dotnet\sdk\2.0.0-preview2-006497\Sdks\Microsoft.Docker.Sdk\build\Microsoft.VisualStudio.Docker.Compose.targets(80,45): error MSB4022: The result "" of evaluating the value "$(DockerBuildTasksAssembly)" of the "AssemblyFile" attribute in element is not valid. [C:\repos\WebApplication1\docker-compose.dcproj]

Any solutions for this?

[philcontrolf1]

@dmitry606 Seems to work for me:

PS D:\Source\HelloWorld> dir

Directory: D:\Source\HelloWorld

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----       01/08/2017     09:15                bin
d-----       01/08/2017     09:15                HelloWorld
d-----       01/08/2017     09:15                obj
-a----       01/08/2017     09:15            271 docker-compose.ci.build.yml
-a----       01/08/2017     09:15            723 docker-compose.dcproj
-a----       01/08/2017     09:15             14 docker-compose.override.yml
-a----       01/08/2017     09:15            370 docker-compose.vs.debug.yml
-a----       01/08/2017     09:15            264 docker-compose.vs.release.yml
-a----       01/08/2017     09:15            136 docker-compose.yml
-a----       01/08/2017     09:15           1479 HelloWorld.sln


PS D:\Source\HelloWorld> dotnet --version
2.0.0-preview2-006497
PS D:\Source\HelloWorld> dotnet build .\HelloWorld.sln
Microsoft (R) Build Engine version 15.3.388.41745 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  HelloWorld -> D:\Source\HelloWorld\HelloWorld\bin\Debug\netcoreapp1.1\HelloWorld.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:01.97
PS D:\Source\HelloWorld> dotnet run --project .\HelloWorld\HelloWorld.csproj
Hello World!

Have you got exactly the same SDK files in exactly the same location?

$ sha1sum /cygdrive/c/Program\ Files/dotnet/sdk/2.0.0-preview2-006497/Sdks/Microsoft.Docker.Sdk/Sdk/*
a30e05d5e5d250884a2e5a29f4730f95e0fb6f85 */cygdrive/c/Program Files/dotnet/sdk/2.0.0-preview2-006497/Sdks/Microsoft.Docker.Sdk/Sdk/Sdk.props
cea93148ba2a590fbda25bfba0ed561f510e6c4a */cygdrive/c/Program Files/dotnet/sdk/2.0.0-preview2-006497/Sdks/Microsoft.Docker.Sdk/Sdk/Sdk.targets

(/cygdrive/c is just C:\ in case you're not a Cygwin person)

[dmitry606]

@philcontrolf1 Yes, it's exactly the same location

`c:\repo\HelloWorld>cd C:\Program Files\dotnet\sdk\2.0.0-preview2-006497\Sdks\Microsoft.Docker.Sdk\Sdk

C:\Program Files\dotnet\sdk\2.0.0-preview2-006497\Sdks\Microsoft.Docker.Sdk\Sdk>dir

Directory of C:\Program Files\dotnet\sdk\2.0.0-preview2-006497\Sdks\Microsoft.Docker.Sdk\Sdk

07/31/2017 03:59 PM

.
07/31/2017 03:59 PM ..
07/28/2017 06:50 PM 1,927 Sdk.props
07/28/2017 06:50 PM 1,220 Sdk.targets
2 File(s) 3,147 bytes
`

[philcontrolf1]

@dmitry606 You've definitely got different files though - my Sdk.props is 1750 bytes and my Sdk.targets is 1264 bytes. Both copied from C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\Sdks\Microsoft.Docker.Sdk\Sdk with Visual Studio 15.2.26430.16, although I don't think they've changed in a while.

[dmitry606]

@philcontrolf1 Yep.. looks like that. I copied the SDK from VS 2017 Preview 2. Probably they broke the SDK in preview2. For now I decided to use Visual Studio for build, hope it will be fixed soon..

[dazhao-msft]

@dmitry606 It looks like that you copied the Microsoft.Docker.Sdk folder from VS 15.3 Preview, under which there are 3 folders: build, Sdk, tools. Please delete build and tools and only keep Sdk.

[dmitry606]

@dazhao-msft It worked! Thanks!!

[milla]

now Linux .net core 2.0 sdk doesn't support docker? I got
docker-compose.dcproj : error MSB4236: The SDK 'Microsoft.Docker.Sdk' specified could not be found.

[philcontrolf1]

@milla Have you copied the appropriate SDK as listed in this thread? If so, exactly which files did you copy, where did you copy them from, and where did you copy them to? A number of us have this working under Preview 2, so it's definitely not completely broken.

[milla]

@philcontrolf1 thanks for checking. It's confirmed that this is a mistake, I built some vsts build agents for different purposes of testing, some of them don't have docker installed. and the agent I used and occurred this problem just doesn't have. Now everything is working perfectly.

[liri2006]

Any ETA on fix?

[tyronegroves]

So no fix?

[shanselman]

ping @richlander ?

[mzunair]

I open Docker CLI as an administrator and just move to the project directory and run following set of commands and it works for me. Also make sure kitematics is running if you are on windows 8.

1. dotnet restore
2. dotnet build
3. dotnet run

[mmisztal1980]

Is this going to get fixed with .net core 2.1?

[mzunair]

Yes.For me, it's working for .net core 2.0 and 2.1

[philcontrolf1]

@zunair-ch Yes, it will work in the project directory because then all you're building is the one project. What happens if you build the solution - e.g. dotnet build ./MySolution.sln in the solution directory?

[philcontrolf1]

For all the watchers here: dotnet/cli#8416 should fix this one. I've just bumped it to see if there's anything blocking it.

[billpratt]

This a good to hear. Thanks for fixing!

[mzunair]

Thanks all.

…

On 10 Feb 2018 4:24 am, "Bill Pratt" ***@***.***> wrote: This a good to hear. Thanks for fixing! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/dotnet/cli/issues/6178#issuecomment-364598176>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ARNoP8UM8gxIaBy9-ch9WV3DH_Yt8sXvks5tTNOxgaJpZM4MsBSr> .

[TomaszG]

I'm currently facing the same problem - solution with docker-compose.dcproj (added by VS after adding enabling docker support) cannot be built or restored with dotnet CLI.
C:\Program Files\dotnet\sdk\2.1.301\NuGet.targets(114,5): error : Invalid restore input. Invalid target framework 'unsupported'. Input files: C:\project\src\docker-compose.dcproj. [C:\project\src\something.sln]

Looking at the comments above I think it should be working, however it doesn't (at least for me). Am I missing something?

Below is dotnet --info output:

.NET Core SDK (reflecting any global.json):
 Version:   2.1.301
 Commit:    59524873d6

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.16299
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.1.301\

Host (useful for support):
  Version: 2.1.1
  Commit:  6985b9f684

.NET Core SDKs installed:
  1.0.0 [C:\Program Files\dotnet\sdk]
  2.1.4 [C:\Program Files\dotnet\sdk]
  2.1.101 [C:\Program Files\dotnet\sdk]
  2.1.102 [C:\Program Files\dotnet\sdk]
  2.1.103 [C:\Program Files\dotnet\sdk]
  2.1.104 [C:\Program Files\dotnet\sdk]
  2.1.201 [C:\Program Files\dotnet\sdk]
  2.1.301 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 1.0.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 1.1.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

[gitfool]

Same problem here with SDK 2.2.101. 😞

[joergjo]

The latest Docker support for ASP.NET Core projects doesn't use .dcproj anymore. So if you recreate your project structure with the latest tooling, you won't run into this issue anymore.

[gitfool]

@joergjo I tried; that only works if a single Dockerfile suffices. If you need to use docker-compose, after creating a new project, right-click Add Orchestrator Support and it adds a dcproj file to the sln and refers to it in the WebApplication csproj:

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>netcoreapp2.2</TargetFramework>
    <AspNetCoreHostingModel>InProcess</AspNetCoreHostingModel>
    <DockerDefaultTargetOS>Linux</DockerDefaultTargetOS>
    <UserSecretsId>f3532c1f-c8ec-4713-8e86-0fc6ebc385c8</UserSecretsId>
    <DockerComposeProjectPath>..\docker-compose.dcproj</DockerComposeProjectPath>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.App" />
    <PackageReference Include="Microsoft.AspNetCore.Razor.Design" Version="2.2.0" PrivateAssets="All" />
    <PackageReference Include="Microsoft.VisualStudio.Azure.Containers.Tools.Targets" Version="1.0.2105168" />
  </ItemGroup>
</Project>

... and the command-line build fails again:

Microsoft (R) Build Engine version 15.9.20+g88f5fadfbe for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

C:\Program Files\dotnet\sdk\2.2.101\NuGet.targets(114,5): error : Invalid restore input. Invalid target framework 'unsupported'. Input files: D:\Devel\Repos\WebApplication1\docker-compose.dcproj. [D:\Devel\Repos\WebApplication1\WebApplication1.sln]

Build FAILED.

There is a newer version available on NuGet, compared to what was generated by the right-click experience, of Microsoft.VisualStudio.Azure.Containers.Tools.Targets, but that doesn't help.

I don't know the details, but it seems to me that we're still blocked on the Microsoft.Docker.Sdk being open sourced before we can use the dotnet CLI to build these solutions on Linux.

(Note: Although I did the above quick & dirty build on Windows, my requirements are to be able to also do the above build on Linux for CI/CD on a build server, ultimately in a Docker container, and with that in mind I think I see another problem that Microsoft.VisualStudio.Azure.Containers.Tools.Targets currently only targets .NET Framework 4.6.1, so presumably it also needs to target .NET Standard 2.0 for my dream scenario to work.)

[joergjo]

I hadn't even noticed that feature!

I always use the "basic" Docker support (i.e. no .dcproj), and add docker-compose files manually as required.

If I want to debug a containerized app with its dependencies running in Docker as well (which rarely happens), I start all dependencies using docker-compose before debugging the actual app, and expose their ports on the host. In this setup, the application can resolve its dependencies using host.docker.internal as host name. It's requires a few more manual steps and works only for a debugging a single container, but it saves me all the .dcproj headache you've mentioned.

[vchirikov]

3.0.100-preview8-013656 still broken.
@livarcocc can you reopen this?

[livarcocc]

@vchirikov how is it broken for you?

[vchirikov]

@livarcocc you can read about it here microsoft/DockerTools#209

[livarcocc]

Looking at the issue you referenced, the CLI team will not enable building this dcprojs in the command line. Though, there is now an SDK that should at least skip doing anything for it.

What seems like a bug to me is the failure to use dotnet sln add for dcprojs. If that's the case, I would suggest filling a brand new issue just for dotnet sln add.

[damilosz]

@livarcocc Which SDK should skip dcprojs while building? I've just tested with 3.1.100-preview1-014459 and I'm unable to run dotnet build or dotnet test on solution containing dcproj file. The error is: C:\Program Files\dotnet\sdk\3.1.100-preview1-014459\NuGet.targets(123,5): error : Invalid restore input. Invalid target framework 'unsupported'.

And I have a workaround. Adding <TargetFramework>netcoreapp2.0</TargetFramework> to the PropertyGroup node in the dcproj file fixes the issue.

[livarcocc]

Can you please file a separate and new issue for this? So that we can start fresh?!

@dsplaisted wonder if this is due to the framework composition work we did. Could you take a look?

[damilosz]

Unfortunately I can't reproduce it with solution created from scratch with VS2017, VS2019 and different project types. Issue occurs only in my old and big repository, which I can't share.

Maybe there is a corner case that I have run into, but I can't really tell exactly what it is.

[vchirikov]

@livarcocc repro here
microsoft/DockerTools#209 (comment)

[SeWieland]

How is this issue closed? dotnet restore is still broken for me with a dcproj file in the solution.
Why are these projects not simply ignored?

Just for any others needing a workaround in their linux CI agents:
Use something like dotnet sln remove $(dotnet sln list | grep .dcproj) and you're good

[dazinator]

@SeWieland - I also have the same problem. As this one is closed I have opened issue #35134