New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial CI with Azure Pipelines #29

Merged
merged 14 commits into from Sep 28, 2018

Conversation

Projects
None yet
4 participants
@andrewlock
Owner

andrewlock commented Sep 19, 2018

No description provided.

@kaylangan

This comment has been minimized.

Show comment
Hide comment
@kaylangan

kaylangan Sep 21, 2018

I'm a Program Manager for Azure Pipelines. Let me know if you have any questions or suggestions. I'm happy to help!

kaylangan commented Sep 21, 2018

I'm a Program Manager for Azure Pipelines. Let me know if you have any questions or suggestions. I'm happy to help!

@andrewlock

This comment has been minimized.

Show comment
Hide comment
@andrewlock

andrewlock Sep 22, 2018

Owner

Hey @kaylangan, thanks for the help. I think the difficulty I'm having getting the Linux/mac builds to work is to get the multiple SDKs installed. I've tried adding multiple instances of the DotNetCoreInstaller@0 but I'm not sure that works correctly!

The various samples and test projects in this project require 2.0, 1.1, and 1.0. I probably could/should consolidate these somewhat, but I'm keen to get it building as-is before making other changes.

The current problem is that the tests expect runtime version 1.1.2, but only 1.1.4 is installed, and I can't get 1.1.2 to install. I think this is more of an issue with .NET Core versioning than azure pipelines though.

Owner

andrewlock commented Sep 22, 2018

Hey @kaylangan, thanks for the help. I think the difficulty I'm having getting the Linux/mac builds to work is to get the multiple SDKs installed. I've tried adding multiple instances of the DotNetCoreInstaller@0 but I'm not sure that works correctly!

The various samples and test projects in this project require 2.0, 1.1, and 1.0. I probably could/should consolidate these somewhat, but I'm keen to get it building as-is before making other changes.

The current problem is that the tests expect runtime version 1.1.2, but only 1.1.4 is installed, and I can't get 1.1.2 to install. I think this is more of an issue with .NET Core versioning than azure pipelines though.

@vtbassmatt

This comment has been minimized.

Show comment
Hide comment
@vtbassmatt

vtbassmatt Sep 24, 2018

Hey @andrewlock, I'm on Kayla's team and she asked me to take a look here. The software on each of our images is listed here. It looks like we don't offer 1.1.2 on the Ubuntu or Mac images, though we do on Windows. I can see about getting that version added...

Your bigger problem is that your tool installers are clobbering each other. The list of steps is processed sequentially. So for instance in the Linux job, you're asking for 1.0.4, then immediately switching to 1.1.4, then immediately switching to 2.1.401.

Instead, you probably want that in a matrix so it runs once for each .NET Core version:

- job: Linux
  pool:
    vmImage: 'ubuntu-16.04'
  strategy:
    maxParallel: 3
    matrix:
      DotNet10:
        dotnet_version: '1.0.4'
      DotNet11:
        dotnet_version: '1.1.4'
      DotNet20:
        dotnet_version: '2.0.401'
  steps:
  - task: DotNetCoreInstaller@0
    inputs:
      version: $( dotnet_version )
  - bash: ./build.sh
    env:
      COREHOST_TRACE: 0
      DOTNET_SKIP_FIRST_TIME_EXPERIENCE: 1
      DOTNET_CLI_TELEMETRY_OPTOUT: 1

And similar for the other platforms. (Once you get that going, we could also look at factoring your job out into a template so you don't have to duplicate the steps on each platform... but one thing at a time.)

vtbassmatt commented Sep 24, 2018

Hey @andrewlock, I'm on Kayla's team and she asked me to take a look here. The software on each of our images is listed here. It looks like we don't offer 1.1.2 on the Ubuntu or Mac images, though we do on Windows. I can see about getting that version added...

Your bigger problem is that your tool installers are clobbering each other. The list of steps is processed sequentially. So for instance in the Linux job, you're asking for 1.0.4, then immediately switching to 1.1.4, then immediately switching to 2.1.401.

Instead, you probably want that in a matrix so it runs once for each .NET Core version:

- job: Linux
  pool:
    vmImage: 'ubuntu-16.04'
  strategy:
    maxParallel: 3
    matrix:
      DotNet10:
        dotnet_version: '1.0.4'
      DotNet11:
        dotnet_version: '1.1.4'
      DotNet20:
        dotnet_version: '2.0.401'
  steps:
  - task: DotNetCoreInstaller@0
    inputs:
      version: $( dotnet_version )
  - bash: ./build.sh
    env:
      COREHOST_TRACE: 0
      DOTNET_SKIP_FIRST_TIME_EXPERIENCE: 1
      DOTNET_CLI_TELEMETRY_OPTOUT: 1

And similar for the other platforms. (Once you get that going, we could also look at factoring your job out into a template so you don't have to duplicate the steps on each platform... but one thing at a time.)

@andrewlock

This comment has been minimized.

Show comment
Hide comment
@andrewlock

andrewlock Sep 26, 2018

Owner

Hi @vtbassmatt,
Thanks for the clarification. Looking at that list of software it seems like I shouldn't need to install any more .NET Core SDKs, there's already a tonne installed though there's a lot of duplicates in this link.

And also, to be clear, I'm not trying to get a "matrix so it runs once for each .NET Core version". If you see the Windows build (that succeeded) its just a single build, it just runs the tests once on each target framework as part of that.

I'll try tweaking some things.

Owner

andrewlock commented Sep 26, 2018

Hi @vtbassmatt,
Thanks for the clarification. Looking at that list of software it seems like I shouldn't need to install any more .NET Core SDKs, there's already a tonne installed though there's a lot of duplicates in this link.

And also, to be clear, I'm not trying to get a "matrix so it runs once for each .NET Core version". If you see the Windows build (that succeeded) its just a single build, it just runs the tests once on each target framework as part of that.

I'll try tweaking some things.

@andrewlock

This comment has been minimized.

Show comment
Hide comment
@andrewlock

andrewlock Sep 26, 2018

Owner

Also, I'm not sure "Your bigger problem is that your tool installers are clobbering each other" holds true given that the build passes on Windows 🙂

Owner

andrewlock commented Sep 26, 2018

Also, I'm not sure "Your bigger problem is that your tool installers are clobbering each other" holds true given that the build passes on Windows 🙂

@andrewlock

This comment has been minimized.

Show comment
Hide comment
@andrewlock

andrewlock Sep 26, 2018

Owner

In fact, removing the installer steps completely fixes the mac build, and the Windows build still succeeds. Linux is still broken though 🤔

Owner

andrewlock commented Sep 26, 2018

In fact, removing the installer steps completely fixes the mac build, and the Windows build still succeeds. Linux is still broken though 🤔

@vtbassmatt

This comment has been minimized.

Show comment
Hide comment
@vtbassmatt

vtbassmatt Sep 27, 2018

I'm definitely missing something. Looking at this build which corresponds to this version of your Azure Pipelines config, I don't see where the Windows tests are running against multiple versions of .NET Core. I don't know Cake, though, so I'm probably misinterpreting the output. So, I may be on the wrong track here since I'm flying a bit blind.

Back to the current version: the Linux failure suggests we don't have a version of .NET Core ready to go on that image. That's what the DotNetCoreInstaller is for. Something in the code or tests is trying to force .NET Core 1.1.2, which we don't have in the tools cache. I would have thought that 1.1.4 would be an effective substitute since it's a point release.

vtbassmatt commented Sep 27, 2018

I'm definitely missing something. Looking at this build which corresponds to this version of your Azure Pipelines config, I don't see where the Windows tests are running against multiple versions of .NET Core. I don't know Cake, though, so I'm probably misinterpreting the output. So, I may be on the wrong track here since I'm flying a bit blind.

Back to the current version: the Linux failure suggests we don't have a version of .NET Core ready to go on that image. That's what the DotNetCoreInstaller is for. Something in the code or tests is trying to force .NET Core 1.1.2, which we don't have in the tools cache. I would have thought that 1.1.4 would be an effective substitute since it's a point release.

@andrewlock

This comment has been minimized.

Show comment
Hide comment
@andrewlock

andrewlock Sep 27, 2018

Owner

So in that build you linked to, if you scroll down (a long way) in the powershell task log, you'll see this:

2018-09-20T19:45:45.1279737Z Testing project D:/a/1/s/test/NetEscapades.Configuration.Yaml.Tests/NetEscapades.Configuration.Yaml.Tests.csproj
2018-09-20T19:45:45.8247248Z Test run for D:\a\1\s\test\NetEscapades.Configuration.Yaml.Tests\bin\Release\net452\NetEscapades.Configuration.Yaml.Tests.dll(.NETFramework,Version=v4.5.2)
2018-09-20T19:45:45.8837680Z Microsoft (R) Test Execution Command Line Tool Version 15.8.0
2018-09-20T19:45:45.8838953Z Copyright (c) Microsoft Corporation.  All rights reserved.
2018-09-20T19:45:45.8839077Z 
2018-09-20T19:45:45.9484113Z Starting test execution, please wait...
2018-09-20T19:45:47.2068445Z 
2018-09-20T19:45:47.2073791Z Total tests: 24. Passed: 24. Failed: 0. Skipped: 0.
2018-09-20T19:45:47.2074225Z Test Run Successful.
2018-09-20T19:45:47.2075245Z Test execution time: 1.2101 Seconds
2018-09-20T19:45:47.2644640Z Test run for D:\a\1\s\test\NetEscapades.Configuration.Yaml.Tests\bin\Release\netcoreapp2.0\NetEscapades.Configuration.Yaml.Tests.dll(.NETCoreApp,Version=v2.0)
2018-09-20T19:45:47.3226811Z Microsoft (R) Test Execution Command Line Tool Version 15.8.0
2018-09-20T19:45:47.3227333Z Copyright (c) Microsoft Corporation.  All rights reserved.
2018-09-20T19:45:47.3227546Z 
2018-09-20T19:45:47.3908942Z Starting test execution, please wait...
2018-09-20T19:45:50.0118444Z 
2018-09-20T19:45:50.0124611Z Total tests: 24. Passed: 24. Failed: 0. Skipped: 0.
2018-09-20T19:45:50.0124889Z Test Run Successful.
2018-09-20T19:45:50.0125702Z Test execution time: 2.5683 Seconds
2018-09-20T19:45:50.0736372Z Test run for D:\a\1\s\test\NetEscapades.Configuration.Yaml.Tests\bin\Release\netcoreapp1.1\NetEscapades.Configuration.Yaml.Tests.dll(.NETCoreApp,Version=v1.1)
2018-09-20T19:45:50.1336019Z Microsoft (R) Test Execution Command Line Tool Version 15.8.0
2018-09-20T19:45:50.1336569Z Copyright (c) Microsoft Corporation.  All rights reserved.
2018-09-20T19:45:50.1336717Z 
2018-09-20T19:45:50.1947645Z Starting test execution, please wait...
2018-09-20T19:45:53.1239581Z 
2018-09-20T19:45:53.1245340Z Total tests: 24. Passed: 24. Failed: 0. Skipped: 0.
2018-09-20T19:45:53.1245646Z Test Run Successful.
2018-09-20T19:45:53.1249242Z Test execution time: 2.8773 Seconds

That shows it running the same project against .NETFramework,Version=v4.5.2, NETCoreApp,Version=v2.0, and NETCoreApp,Version=v1.1.
Cake's basically just doing a dotnet test under the hood, and that's a multi- targeted project.

The weird thing is I don't know why it needs 1.1.2 at the moment. According to the install list you posted previously, neither the Mac or Windows machines have 1.1.2 installed either, yet they are building fine in the latest iteration.

So it's just linux. It's as though the other .net runtimes haven't been installed even though they should be according to the list of installed software. I'll add a task to run dotnet --info to see what's actually on there at some point soon, because it's all a bit odd.

Owner

andrewlock commented Sep 27, 2018

So in that build you linked to, if you scroll down (a long way) in the powershell task log, you'll see this:

2018-09-20T19:45:45.1279737Z Testing project D:/a/1/s/test/NetEscapades.Configuration.Yaml.Tests/NetEscapades.Configuration.Yaml.Tests.csproj
2018-09-20T19:45:45.8247248Z Test run for D:\a\1\s\test\NetEscapades.Configuration.Yaml.Tests\bin\Release\net452\NetEscapades.Configuration.Yaml.Tests.dll(.NETFramework,Version=v4.5.2)
2018-09-20T19:45:45.8837680Z Microsoft (R) Test Execution Command Line Tool Version 15.8.0
2018-09-20T19:45:45.8838953Z Copyright (c) Microsoft Corporation.  All rights reserved.
2018-09-20T19:45:45.8839077Z 
2018-09-20T19:45:45.9484113Z Starting test execution, please wait...
2018-09-20T19:45:47.2068445Z 
2018-09-20T19:45:47.2073791Z Total tests: 24. Passed: 24. Failed: 0. Skipped: 0.
2018-09-20T19:45:47.2074225Z Test Run Successful.
2018-09-20T19:45:47.2075245Z Test execution time: 1.2101 Seconds
2018-09-20T19:45:47.2644640Z Test run for D:\a\1\s\test\NetEscapades.Configuration.Yaml.Tests\bin\Release\netcoreapp2.0\NetEscapades.Configuration.Yaml.Tests.dll(.NETCoreApp,Version=v2.0)
2018-09-20T19:45:47.3226811Z Microsoft (R) Test Execution Command Line Tool Version 15.8.0
2018-09-20T19:45:47.3227333Z Copyright (c) Microsoft Corporation.  All rights reserved.
2018-09-20T19:45:47.3227546Z 
2018-09-20T19:45:47.3908942Z Starting test execution, please wait...
2018-09-20T19:45:50.0118444Z 
2018-09-20T19:45:50.0124611Z Total tests: 24. Passed: 24. Failed: 0. Skipped: 0.
2018-09-20T19:45:50.0124889Z Test Run Successful.
2018-09-20T19:45:50.0125702Z Test execution time: 2.5683 Seconds
2018-09-20T19:45:50.0736372Z Test run for D:\a\1\s\test\NetEscapades.Configuration.Yaml.Tests\bin\Release\netcoreapp1.1\NetEscapades.Configuration.Yaml.Tests.dll(.NETCoreApp,Version=v1.1)
2018-09-20T19:45:50.1336019Z Microsoft (R) Test Execution Command Line Tool Version 15.8.0
2018-09-20T19:45:50.1336569Z Copyright (c) Microsoft Corporation.  All rights reserved.
2018-09-20T19:45:50.1336717Z 
2018-09-20T19:45:50.1947645Z Starting test execution, please wait...
2018-09-20T19:45:53.1239581Z 
2018-09-20T19:45:53.1245340Z Total tests: 24. Passed: 24. Failed: 0. Skipped: 0.
2018-09-20T19:45:53.1245646Z Test Run Successful.
2018-09-20T19:45:53.1249242Z Test execution time: 2.8773 Seconds

That shows it running the same project against .NETFramework,Version=v4.5.2, NETCoreApp,Version=v2.0, and NETCoreApp,Version=v1.1.
Cake's basically just doing a dotnet test under the hood, and that's a multi- targeted project.

The weird thing is I don't know why it needs 1.1.2 at the moment. According to the install list you posted previously, neither the Mac or Windows machines have 1.1.2 installed either, yet they are building fine in the latest iteration.

So it's just linux. It's as though the other .net runtimes haven't been installed even though they should be according to the list of installed software. I'll add a task to run dotnet --info to see what's actually on there at some point soon, because it's all a bit odd.

@andrewlock

This comment has been minimized.

Show comment
Hide comment
@andrewlock

andrewlock Sep 27, 2018

Owner

@vtbassmatt I just added a call to dotnet --info to the test run. It confirms that the list of software installed is incorrect. Or that the .NET SDKs have been installed incorrectly. Only the 2.1.401 SDK (2.1.3 runtime) is available:

dotnet --info
2018-09-27T15:17:29.2357113Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/e8a76760-d87f-4530-9c87-66bf30815062.sh
2018-09-27T15:17:36.2690537Z .NET Core SDK (reflecting any global.json):
2018-09-27T15:17:36.2702134Z  Version:   2.1.401
2018-09-27T15:17:36.2716058Z  Commit:    91b1c13032
2018-09-27T15:17:36.2721591Z 
2018-09-27T15:17:36.2733082Z Runtime Environment:
2018-09-27T15:17:36.2745351Z  OS Name:     ubuntu
2018-09-27T15:17:36.2758376Z  OS Version:  16.04
2018-09-27T15:17:36.2770283Z  OS Platform: Linux
2018-09-27T15:17:36.6309387Z  RID:         ubuntu.16.04-x64
2018-09-27T15:17:36.6447295Z  Base Path:   /usr/share/dotnet/sdk/2.1.401/
2018-09-27T15:17:36.6464558Z 
2018-09-27T15:17:36.6478058Z Host (useful for support):
2018-09-27T15:17:36.6490366Z   Version: 2.1.3
2018-09-27T15:17:36.6505758Z   Commit:  124038c13e
2018-09-27T15:17:36.6512221Z 
2018-09-27T15:17:36.6525764Z .NET Core SDKs installed:
2018-09-27T15:17:36.6538805Z   2.1.401 [/usr/share/dotnet/sdk]
2018-09-27T15:17:36.6545580Z 
2018-09-27T15:17:36.6557977Z .NET Core runtimes installed:
2018-09-27T15:17:36.6571627Z   Microsoft.AspNetCore.All 2.1.3 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
2018-09-27T15:17:36.6584647Z   Microsoft.AspNetCore.App 2.1.3 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
2018-09-27T15:17:36.6597187Z   Microsoft.NETCore.App 2.1.3 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
2018-09-27T15:17:36.6602945Z 
2018-09-27T15:17:36.6617187Z To install additional .NET Core runtimes or SDKs:
2018-09-27T15:17:36.6631588Z   https://aka.ms/dotnet-download

They're meant to side-by-side installations, that's one of the selling points of .NET Core, so they're definitely not installed on the agent.

Edit: Very strangely, I just ran it again, and this time it shows all the installed .NET Core runtimes and SDKs! and lo and behold the build succeeds.

This looks to me like a definite bug on the Azure pipelines side - the agent I got in the last run was definitely different to the one before!

Owner

andrewlock commented Sep 27, 2018

@vtbassmatt I just added a call to dotnet --info to the test run. It confirms that the list of software installed is incorrect. Or that the .NET SDKs have been installed incorrectly. Only the 2.1.401 SDK (2.1.3 runtime) is available:

dotnet --info
2018-09-27T15:17:29.2357113Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/e8a76760-d87f-4530-9c87-66bf30815062.sh
2018-09-27T15:17:36.2690537Z .NET Core SDK (reflecting any global.json):
2018-09-27T15:17:36.2702134Z  Version:   2.1.401
2018-09-27T15:17:36.2716058Z  Commit:    91b1c13032
2018-09-27T15:17:36.2721591Z 
2018-09-27T15:17:36.2733082Z Runtime Environment:
2018-09-27T15:17:36.2745351Z  OS Name:     ubuntu
2018-09-27T15:17:36.2758376Z  OS Version:  16.04
2018-09-27T15:17:36.2770283Z  OS Platform: Linux
2018-09-27T15:17:36.6309387Z  RID:         ubuntu.16.04-x64
2018-09-27T15:17:36.6447295Z  Base Path:   /usr/share/dotnet/sdk/2.1.401/
2018-09-27T15:17:36.6464558Z 
2018-09-27T15:17:36.6478058Z Host (useful for support):
2018-09-27T15:17:36.6490366Z   Version: 2.1.3
2018-09-27T15:17:36.6505758Z   Commit:  124038c13e
2018-09-27T15:17:36.6512221Z 
2018-09-27T15:17:36.6525764Z .NET Core SDKs installed:
2018-09-27T15:17:36.6538805Z   2.1.401 [/usr/share/dotnet/sdk]
2018-09-27T15:17:36.6545580Z 
2018-09-27T15:17:36.6557977Z .NET Core runtimes installed:
2018-09-27T15:17:36.6571627Z   Microsoft.AspNetCore.All 2.1.3 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
2018-09-27T15:17:36.6584647Z   Microsoft.AspNetCore.App 2.1.3 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
2018-09-27T15:17:36.6597187Z   Microsoft.NETCore.App 2.1.3 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
2018-09-27T15:17:36.6602945Z 
2018-09-27T15:17:36.6617187Z To install additional .NET Core runtimes or SDKs:
2018-09-27T15:17:36.6631588Z   https://aka.ms/dotnet-download

They're meant to side-by-side installations, that's one of the selling points of .NET Core, so they're definitely not installed on the agent.

Edit: Very strangely, I just ran it again, and this time it shows all the installed .NET Core runtimes and SDKs! and lo and behold the build succeeds.

This looks to me like a definite bug on the Azure pipelines side - the agent I got in the last run was definitely different to the one before!

@andrewlock

This comment has been minimized.

Show comment
Hide comment
@andrewlock

andrewlock Sep 27, 2018

Owner

And now it's failed again. Def looks like there's something wrong with some of your Linux agents

Owner

andrewlock commented Sep 27, 2018

And now it's failed again. Def looks like there's something wrong with some of your Linux agents

@vtbassmatt

This comment has been minimized.

Show comment
Hide comment
@vtbassmatt

vtbassmatt Sep 27, 2018

@dakale are you the right person to investigate this? It's strange that in consecutive runs, we'd see such divergent agent behavior.

vtbassmatt commented Sep 27, 2018

@dakale are you the right person to investigate this? It's strange that in consecutive runs, we'd see such divergent agent behavior.

@dakale

This comment has been minimized.

Show comment
Hide comment
@dakale

dakale Sep 27, 2018

It looks like there are some mismatches in image versions on the Ubuntu pools. Some machines have an older image, which dont have all the SDKs installed. Ill fix today

dakale commented Sep 27, 2018

It looks like there are some mismatches in image versions on the Ubuntu pools. Some machines have an older image, which dont have all the SDKs installed. Ill fix today

@andrewlock

This comment has been minimized.

Show comment
Hide comment
@andrewlock

andrewlock Sep 27, 2018

Owner

Great, that sounds like it explains it! thanks both 🙂

Owner

andrewlock commented Sep 27, 2018

Great, that sounds like it explains it! thanks both 🙂

@andrewlock andrewlock merged commit 235ea83 into master Sep 28, 2018

2 checks passed

andrewlock.NetEscapades.Configuration #20180928.4 succeeded
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details

@andrewlock andrewlock deleted the add-azure-pipelines branch Sep 28, 2018

@andrewlock

This comment has been minimized.

Show comment
Hide comment
@andrewlock

andrewlock Sep 28, 2018

Owner

All looks to be sorted now I think, so merged this now. Thanks.

Owner

andrewlock commented Sep 28, 2018

All looks to be sorted now I think, so merged this now. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment