Visual Studio Online - Cake Task failure not failing build

[devlead]

@Volak commented on Mon May 29 2017

What You Are Seeing?

I'm using cake and in the process of moving to VSTS for builds - and it seems the cake task from the market place: https://marketplace.visualstudio.com/items?itemName=cake-build.cake
Is not passing errors to the build? Even when cake throws a build exception the build succeeds - as noted in the following shots

What is Expected?

When cake throws an exception the build should fail

---

@devlead commented on Mon May 29 2017

Looks like you're missing MS build web targets Sdk on build agent.

Either it needs to be installed or use something like this nuget package
https://www.nuget.org/packages/MSBuild.Microsoft.VisualStudio.Web.targets/14.0.0.3

---

@Volak commented on Mon May 29 2017

Thank you for the tip - you are right about the error I'm getting during the build.

But that's just for example - the problem is VSTS thinks the build succeeded even though cake failed 😞

[devlead]

@Volak I just created a new VSTS project&repo, then created a minimal cake script like this

Information("Hello world");

the log looked like this

2017-05-29T10:59:51.8896940Z ##[section]Starting: Cake 
2017-05-29T10:59:51.9006935Z ==============================================================================
2017-05-29T10:59:51.9006935Z Task         : Cake
2017-05-29T10:59:51.9006935Z Description  : Build with Cake
2017-05-29T10:59:51.9006935Z Version      : 0.3.0
2017-05-29T10:59:51.9006935Z Author       : Patrik Svensson
2017-05-29T10:59:51.9006935Z Help         : [More Information about Cake](http://cakebuild.net)
2017-05-29T10:59:51.9006935Z ==============================================================================
2017-05-29T10:59:51.9206934Z Preparing task execution handler.
2017-05-29T10:59:54.9497130Z Executing the powershell script: d:\a\_tasks\Cake_b88ea9a0-7d6e-11e5-b5de-d57d652482f7\0.3.0\Cake.ps1
2017-05-29T10:59:57.5781996Z Installing Cake...
2017-05-29T10:59:57.5781996Z 
2017-05-29T10:59:57.5781996Z 
2017-05-29T10:59:58.7482572Z Feeds used:
2017-05-29T10:59:58.7482572Z   https://api.nuget.org/v3/index.json
2017-05-29T10:59:58.7482572Z 
2017-05-29T10:59:59.1352591Z   GET https://api.nuget.org/v3/registration1-gz/cake/index.json
2017-05-29T10:59:59.3222784Z   OK https://api.nuget.org/v3/registration1-gz/cake/index.json 192ms
2017-05-29T10:59:59.3872602Z 
2017-05-29T10:59:59.3872602Z 
2017-05-29T10:59:59.3892600Z Attempting to gather dependency information for package 'Cake.0.19.5' with respect to project 'd:\a\1\s\tools', targeting 'Any,Version=v0.0'
2017-05-29T10:59:59.5372622Z Gathering dependency information took 42.67 ms
2017-05-29T10:59:59.5372622Z Attempting to resolve dependencies for package 'Cake.0.19.5' with DependencyBehavior 'Lowest'
2017-05-29T10:59:59.5382886Z Resolving dependency information took 0 ms
2017-05-29T10:59:59.5382886Z Resolving actions to install package 'Cake.0.19.5'
2017-05-29T10:59:59.5382886Z Resolved actions to install package 'Cake.0.19.5'
2017-05-29T10:59:59.5382886Z Retrieving package 'Cake 0.19.5' from 'nuget.org'.
2017-05-29T10:59:59.5642621Z   GET https://api.nuget.org/packages/cake.0.19.5.nupkg
2017-05-29T10:59:59.6022615Z   OK https://api.nuget.org/packages/cake.0.19.5.nupkg 26ms
2017-05-29T10:59:59.6022615Z Installing Cake 0.19.5.
2017-05-29T10:59:59.8542648Z Adding package 'Cake.0.19.5' to folder 'd:\a\1\s\tools'
2017-05-29T10:59:59.8742645Z Added package 'Cake.0.19.5' to folder 'd:\a\1\s\tools'
2017-05-29T10:59:59.8742645Z Successfully installed 'Cake 0.19.5' to d:\a\1\s\tools
2017-05-29T10:59:59.8742645Z Executing nuget actions took 352.12 ms
2017-05-29T10:59:59.8852644Z Executing build script...
2017-05-29T10:59:59.8892643Z 
2017-05-29T10:59:59.8892643Z 
2017-05-29T10:59:59.9112644Z d:\a\1\s\tools\Cake\Cake.exe "d:\a\1\s\build.cake" -target="Default" -verbosity="Verbose" --paths_tools="d:\a\1\s\tools" 
2017-05-29T11:00:00.1272661Z Analyzing build script...
2017-05-29T11:00:00.1462656Z Processing build script...
2017-05-29T11:00:00.1462656Z Downloading and installing Roslyn...
2017-05-29T11:00:00.2872673Z Installing packages (using https://packages.nuget.org/api/v2)...
2017-05-29T11:00:02.0992788Z Copying files...
2017-05-29T11:00:02.0992788Z Copying Roslyn.Compilers.CSharp.dll...
2017-05-29T11:00:02.1112784Z Copying Roslyn.Compilers.dll...
2017-05-29T11:00:02.1112784Z Deleting installation directory...
2017-05-29T11:00:02.3462798Z Compiling build script...
2017-05-29T11:00:04.6112945Z Hello world
2017-05-29T11:00:04.6762954Z ##[section]Finishing: Cake 

and as expected the build succeeded

then added a throw to the script like this

Information("Hello world");
throw new Exception("Fail the build!");

the log then looked like this

2017-05-29T11:02:37.9647038Z ##[section]Starting: Cake 
2017-05-29T11:02:37.9803290Z ==============================================================================
2017-05-29T11:02:37.9803290Z Task         : Cake
2017-05-29T11:02:37.9803290Z Description  : Build with Cake
2017-05-29T11:02:37.9803290Z Version      : 0.3.0
2017-05-29T11:02:37.9803290Z Author       : Patrik Svensson
2017-05-29T11:02:37.9803290Z Help         : [More Information about Cake](http://cakebuild.net)
2017-05-29T11:02:37.9803290Z ==============================================================================
2017-05-29T11:02:38.0115793Z Preparing task execution handler.
2017-05-29T11:02:40.9311042Z Executing the powershell script: d:\a\_tasks\Cake_b88ea9a0-7d6e-11e5-b5de-d57d652482f7\0.3.0\Cake.ps1
2017-05-29T11:02:41.7593469Z Installing Cake...
2017-05-29T11:02:41.7749687Z 
2017-05-29T11:02:41.7749687Z 
2017-05-29T11:02:42.5255224Z Feeds used:
2017-05-29T11:02:42.5255224Z   https://api.nuget.org/v3/index.json
2017-05-29T11:02:42.5255224Z 
2017-05-29T11:02:42.9630230Z   GET https://api.nuget.org/v3/registration1-gz/cake/index.json
2017-05-29T11:02:43.0567722Z   OK https://api.nuget.org/v3/registration1-gz/cake/index.json 100ms
2017-05-29T11:02:43.1351093Z 
2017-05-29T11:02:43.1973976Z 
2017-05-29T11:02:43.1973976Z Attempting to gather dependency information for package 'Cake.0.19.5' with respect to project 'd:\a\1\s\tools', targeting 'Any,Version=v0.0'
2017-05-29T11:02:43.1973976Z Gathering dependency information took 48.59 ms
2017-05-29T11:02:43.2288051Z Attempting to resolve dependencies for package 'Cake.0.19.5' with DependencyBehavior 'Lowest'
2017-05-29T11:02:43.3224086Z Resolving dependency information took 0 ms
2017-05-29T11:02:43.3224086Z Resolving actions to install package 'Cake.0.19.5'
2017-05-29T11:02:43.3224086Z Resolved actions to install package 'Cake.0.19.5'
2017-05-29T11:02:43.3380224Z Retrieving package 'Cake 0.19.5' from 'nuget.org'.
2017-05-29T11:02:43.3692725Z   GET https://api.nuget.org/packages/cake.0.19.5.nupkg
2017-05-29T11:02:43.4005220Z   OK https://api.nuget.org/packages/cake.0.19.5.nupkg 21ms
2017-05-29T11:02:43.4005220Z Installing Cake 0.19.5.
2017-05-29T11:02:43.6348971Z Adding package 'Cake.0.19.5' to folder 'd:\a\1\s\tools'
2017-05-29T11:02:43.6505230Z Added package 'Cake.0.19.5' to folder 'd:\a\1\s\tools'
2017-05-29T11:02:43.6661480Z Successfully installed 'Cake 0.19.5' to d:\a\1\s\tools
2017-05-29T11:02:43.6661480Z Executing nuget actions took 324.12 ms
2017-05-29T11:02:43.6661480Z Executing build script...
2017-05-29T11:02:43.6661480Z 
2017-05-29T11:02:43.6661480Z 
2017-05-29T11:02:43.6973972Z d:\a\1\s\tools\Cake\Cake.exe "d:\a\1\s\build.cake" -target="Default" -verbosity="Verbose" --paths_tools="d:\a\1\s\tools" 
2017-05-29T11:02:44.0255220Z Analyzing build script...
2017-05-29T11:02:44.0411804Z Processing build script...
2017-05-29T11:02:44.0411804Z Downloading and installing Roslyn...
2017-05-29T11:02:44.0880246Z Installing packages (using https://packages.nuget.org/api/v2)...
2017-05-29T11:02:45.9613888Z Copying files...
2017-05-29T11:02:45.9613888Z Copying Roslyn.Compilers.CSharp.dll...
2017-05-29T11:02:45.9770142Z Copying Roslyn.Compilers.dll...
2017-05-29T11:02:45.9770142Z Deleting installation directory...
2017-05-29T11:02:46.2113881Z Compiling build script...
2017-05-29T11:02:48.7702797Z Hello world
-2017-05-29T11:02:48.7702797Z ##[error]Error: Fail the build!
-2017-05-29T11:02:48.8015294Z ##[error]System.Exception: Unexpected exit code 1 returned from tool Cake.exe
-2017-05-29T11:02:48.8015294Z    at Microsoft.TeamFoundation.DistributedTask.Task.Internal.InvokeToolCmdlet.ProcessRecord()
-2017-05-29T11:02:48.8015294Z    at System.Management.Automation.CommandProcessor.ProcessRecord()
-2017-05-29T11:02:48.8171549Z ##[error]PowerShell script completed with 1 errors.
2017-05-29T11:02:48.8171549Z ##[section]Finishing: Cake 

and the build failed

So not able to repro, and without more logs or an example Cake file makes it hard to debug.
Would it be possible for you to create a minimal repro that you could share?

Also could you provide info about agent version, if it's an hosted agent or not, and which version of the cake VSTS extension you're using.

[charlessolar]

Thanks for taking a look
Here's my example: https://github.com/volak/cake.test

Without the VSTS specific command stuff the build fails. But once I added the extra logging in teardown the build succeeds.
Maybe this is expected behavior I'm not sure, I don't see why commands.WriteError or checking !Success would eliminate the exception itself

[devlead]

Should there be a failed scenario here?
https://github.com/volak/cake.test/blob/master/build.cake#L79-L82

[devlead]

Also perhaps what you're trying to achieve might already be avail in @agc93 Cake BuildSystems Module

[charlessolar]

That's interesting! I guess you might be right sending success there overrides the thrown exception in VSTS.
However - ITaskTeardownContext doesn't seem to contain any way to tell if the task failed... maybe I missed it?

Edit: Seems like he has the same problem https://github.com/agc93/Cake.BuildSystems.Module/blob/develop/src/Cake.TFBuild.Module/TFBuildEngine.cs#L63
but maybe his reporting is better on failure Ill try this out if I can figure out how to get it installed without checking in the tools directory

[devlead]

Haven't tried so unsure. Could be that you have to complete any non completed tasks here

[gep13]

@Volak said...
but maybe his reporting is better on failure Ill try this out if I can figure out how to get it installed without checking in the tools directory

You shouldn't need to check in the tools directory, but you will need to check in the packages.config files, so that the required components can be restored during initialization.

[charlessolar]

@devlead you are 2 for 2 - adding the failed task reporting fixed it.

@gep13 thanks for chiming in - ill definitely check out the project its just installed in a weird way I don't quiet understand after an all nighter. Im spoiled with #addins and #tools.

[gep13]

@Volak said...
ill definitely check out the project its just installed in a weird way I don't quiet understand after an all nighter. Im spoiled with #addins and #tools.

Ha ha, totally understand!

Due to the fact that Modules are altering the internals of Cake (by overriding/extending built in functionality) they need to be loaded very early in the execution of Cake. Using the packages.config file in Modules folder will mean that the assemblies are fetched prior to cake actually executing, therefore they are there early enough to be loaded into the cake execution.

[charlessolar]

@gep13 probably not a big deal - but build.sh is checking for packages.config under modules not Modules as noted in the readme.
Im just going to rename my folder since case only matters on linux but its a small gotcha

Edit
And I could be wrong but its not looking like the cake build from the marketplace is checking the modules directory either (powershell script works though)

Edit 2
Scratch that first, linux builds didnt use the module unless it was under Modules so its the bootstrap shell script that has a bug

[gep13]

@Volak if you could raise an issue in the respective repository, that would be appreciated.

[charlessolar]

Both issues are created - feel free to close this one btw you solved it

[gep13]

Perfect, thank you.

[gep13]

For reference....

cake-build/resources#31
#40