Switch to Azure Pipelines #259

LegNeato · Oct 1, 2018

AppVeyor is super flaky.

It looks like https://azure.microsoft.com/en-us/services/devops/pipelines/ supports all platforms and would be really fast:

"Get cloud-hosted pipelines for Linux, macOS, and Windows with unlimited minutes and 10 free parallel jobs for open source"

We could switch to one CI instead of maintaining multiple.

theduke · Oct 2, 2018

Yeah that also is the intention behind #213.
Caveat is that Cirrus does not support Rust on Windows easily so I gave up there.
If Azure works well I'm all for it.

10 parallel runners sounds nice.

kaylangan · Oct 2, 2018

I'm a Program Manager on Azure Pipelines. Let me know if you have any questions or suggestions. I'm happy to help!

LegNeato · Oct 2, 2018

@kaylangan great, thanks! Do you know of any Rust projects currently using Azure Pipelines we could take a look at?

LegNeato · Oct 3, 2018

@kaylangan I want to have am N*M matrix of jobs, similar to what we have in Travis now (rust version * platform). It doesn't appear we can do that easily?

When I do something like:

strategy:
  matrix:
    stable:
      rustup_toolchain: stable
    beta:
      rustup_toolchain: beta
    nightly:
      rustup_install: nightly
    minimum_supported_version_plus_one:
      rustup_toolchain: 1.23.0
    minimum_supported_version:
      rustup_toolchain: 1.22.0

jobs:
- template: _build/job.yml 
  parameters:
    name: macOS
    pool:
      vmImage: 'macOS-10.13'

- template: _build/job.yml
  parameters:
    name: Linux
    pool:
      vmImage: 'Ubuntu-16.04'

- template: _build/job.yml
  parameters:
    name: Windows
    pool:
      vmImage: 'vs2017-win2016'

I get an error azure-pipelines.yml (Line: 14, Col: 1): 'jobs' is not allowed. 'strategy' is already defined and is mutually exclusive.

kaylangan · Oct 3, 2018

@LegNeato The parcel-bindler/parcel project might be a good example for how to set this up. They have an azure-pipelines.yml file that lists the jobs across the three OS's. For each of those jobs, they run the azure-pipelines-template.yml template. In the template, they've specified the matrix strategy, and (bonus!) install Rust.

LegNeato · Oct 8, 2018

@kaylangan Thanks, very helpful!

Though for some reason I can't get it to work on windows. I'll keep poking around...

LegNeato · Oct 8, 2018

A downside is crates.io does not support azure pipelines for the badge key:

https://github.com/rust-lang/crates.io/blob/33cf64951b3399093049f5331dc43fec24008bcb/src/models/badge.rs#L30

LegNeato · Oct 8, 2018

Bug with display when using azure pipelines:

("-1" minute)

LegNeato · Oct 8, 2018

I've landed this and made it required to merge PRs. I haven't removed any of the other CI systems yet though.

sagiegurari · Oct 9, 2018

@LegNeato that's awesome! few things...

coverage is not executed. i have a condition yhat by default in a ci flow its only available for travis and that can be changed easily but needs to be tested (never ran kocov on azure)
codecov do not seem to support azure yet so it fails to upload reports without providing a token
definitely going to use your experience to add to cargo-make documentation on how to support azure pipelines

@kaylangan any chance you can add friendly mobile ui view support for azure pipelines? I'm trying to see the build log via chrome mobile and it's driving me crazy (focus is jumping alot)
thabks!!!

LegNeato · Oct 9, 2018

@sagiegurari good to know! I'm leaving travis as-is until we can be at feature parity. I'm likely going to patch crates.io. Also, azure pipelines seems to support templates in their own repo, so I am going to add some that people can point to rather than copying and pasting!

LegNeato · Oct 9, 2018

@kaylangan Another feature request: jobs seem to be triggered in alphabetical order, which means our windows jobs aren't run until the other ones complete due to resource usage. It would be better to randomize the order in which they start.

kaylangan · Oct 9, 2018

Bug with display when using azure pipelines:

("-1" minute)

@LegNeato thanks for the report--I've filed the bug on our side.

any chance you can add friendly mobile ui view support for azure pipelines? I'm trying to see the build log via chrome mobile and it's driving me crazy (focus is jumping alot)
thabks!!!

@sagiegurari We've heard asks for mobile views. It's not in the short-term (3-6 month plan), but is on our backlog. It sounds like at a bare minimum, you'd want a mobile view of the logs, yeah?

Another feature request: jobs seem to be triggered in alphabetical order, which means our windows jobs aren't run until the other ones complete due to resource usage. It would be better to randomize the order in which they start.

@LegNeato I'm not sure I fully understand the request. Even if the job order were randomized, the build wouldn't return status until all of the jobs finished. What's the benefit of randomizing the order?

LegNeato · Oct 10, 2018

@kaylangan sweet, thanks for filing! Re: order, every build triggers 10+ jobs due to the matrix. Because "Windows" is alphabetically last, it doesn't run until there is space in the queue. Usually jobs fail by platform, so we don't see if Windows fails until Mac or Linux jobs completely finish.

kaylangan · Oct 11, 2018

@LegNeato I gotcha. We'll need to think a little more about this one. I'm not sure randomization is the solution, but maybe an option to specify order or something like that. Thanks for the feedback!

LegNeato · Oct 17, 2018

We should look into the new github stuff to see if there are any benefits of switching yet again.

theduke · Dec 19, 2018

So the only thing not working on Azure now is code coverage, right?

LegNeato · Dec 19, 2018

Correct

theduke · Dec 19, 2018

@LegNeato also what do you think about running cargo fmt --check on CI so we don't have any "Formatting..." commits?

Might be a bit annoying for occasional contributors though.
(right now cargo fmt produces a pretty big diff again :/ )

sagiegurari · Dec 19, 2018

in the next cargo-make release (its currently only in the dev branch) adding the CARGO_MAKE_RUN_CHECK_FORMAT env var to your makefile would trigger fmt check as part of the ci flow

LegNeato · Dec 19, 2018

I'm all for autoformatting / enforcing style.

theduke · Dec 19, 2018

@sagiegurari that's convenient, but I chose to add the check as an extra job on the CI.

That way you can see with one glance that the build failed due to formatting: https://dev.azure.com/graphql-rust/GraphQL%20Rust/_build/results?buildId=149

jgillich · Feb 2, 2019

https://internals.rust-lang.org/t/which-ci-platform-should-rust-use/9322

Juniper just got a shoutout as one of the Rust projects using Pipelines. One of you might want to join the discussion and share your experience with using pipelines.

LegNeato added enhancement tooling labels Oct 1, 2018

LegNeato referenced this issue Oct 8, 2018
Merged
Set up CI with Azure Pipelines #263

LegNeato self-assigned this Oct 8, 2018

LegNeato added a commit to LegNeato/juniper that referenced this issue Oct 9, 2018

Remove Appveyor
Now that we have azure pipelines set up, there is no need to use appveyor. Part of graphql-rust#259

0def014

LegNeato added a commit to LegNeato/juniper that referenced this issue Oct 9, 2018

Remove Appveyor
Now that we have azure pipelines set up, there is no need to use appveyor. Part of graphql-rust#259

7abc87e

LegNeato referenced this issue Oct 9, 2018
Merged
Remove Appveyor #264

azure-pipelines bot referenced this issue Nov 21, 2018
Merged
Set up CI with Azure Pipelines #16

graphql-rust/juniper

Join GitHub today

Switch to Azure Pipelines #259

Comments

LegNeato commented Oct 1, 2018

LegNeato added enhancement tooling labels Oct 1, 2018

This comment has been minimized.

theduke commented Oct 2, 2018 • edited

This comment has been minimized.

kaylangan commented Oct 2, 2018

This comment has been minimized.

LegNeato commented Oct 2, 2018

This comment has been minimized.

LegNeato commented Oct 3, 2018

This comment has been minimized.

kaylangan commented Oct 3, 2018

This comment has been minimized.

LegNeato commented Oct 8, 2018 • edited

LegNeato referenced this issue Oct 8, 2018

Set up CI with Azure Pipelines #263

This comment has been minimized.

LegNeato commented Oct 8, 2018

This comment has been minimized.

LegNeato commented Oct 8, 2018

LegNeato self-assigned this Oct 8, 2018

This comment has been minimized.

LegNeato commented Oct 8, 2018

LegNeato added a commit to LegNeato/juniper that referenced this issue Oct 9, 2018

LegNeato added a commit to LegNeato/juniper that referenced this issue Oct 9, 2018

LegNeato referenced this issue Oct 9, 2018

Remove Appveyor #264

LegNeato added a commit that referenced this issue Oct 9, 2018

This comment has been minimized.

sagiegurari commented Oct 9, 2018

This comment has been minimized.

LegNeato commented Oct 9, 2018

This comment has been minimized.

LegNeato commented Oct 9, 2018

This comment has been minimized.

kaylangan commented Oct 9, 2018

This comment has been minimized.

LegNeato commented Oct 10, 2018

This comment has been minimized.

kaylangan commented Oct 11, 2018

This comment has been minimized.

LegNeato commented Oct 17, 2018

azure-pipelines bot referenced this issue Nov 21, 2018

Set up CI with Azure Pipelines #16

This comment has been minimized.

theduke commented Dec 19, 2018

This comment has been minimized.

LegNeato commented Dec 19, 2018

This comment has been minimized.

theduke commented Dec 19, 2018 • edited

This comment has been minimized.

sagiegurari commented Dec 19, 2018

This comment has been minimized.

LegNeato commented Dec 19, 2018

This comment has been minimized.

theduke commented Dec 19, 2018

This comment has been minimized.

jgillich commented Feb 2, 2019 • edited

theduke commented Oct 2, 2018 •

edited

LegNeato commented Oct 8, 2018 •

edited

theduke commented Dec 19, 2018 •

edited

jgillich commented Feb 2, 2019 •

edited