Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG] Configures Azure Pipelines #13206

Merged
merged 97 commits into from Feb 21, 2019

Conversation

Projects
None yet
6 participants
@thomasjpfan
Copy link
Member

commented Feb 21, 2019

Reference Issues/PRs

Resolves #12986

What does this implement/fix? Explain your changes.

Adds configuration for azure pipelines for linux, windows, and osx. Here is what it looks like: https://dev.azure.com/thomasjpfan/scikit-learn/_build/results?buildId=84

junitxml is configured, so we do not need to scroll down to see errors anymore, the traceback and errors are shown in the UI: https://dev.azure.com/thomasjpfan/scikit-learn/_build/results?buildId=84&view=ms.vss-test-web.build-test-results-tab

Any other comments?

The linux and osx tests are the same as the tests we run on travis CI.

For windows the difference between appveyor and azure is:

  1. The dependencies are installed through conda.
  2. Does not upload windows builds to rackspace. (This could be added if we want it)
  3. Does not support 32 bit python. Currently, the config uses the conda already installed in the VM, thus not needing to download or install miniconda. Do we want to continue to test python 32bit on windows?

I have not found a nice way to add CODECOV_TOKEN to the environment besides library variable groups. @mickeygousset What would you recommend if we need to set private env variables?

Edit: To enable codecov, follow the advice from #13206 (comment) and a new secret variable called CODECOV_TOKEN.

CC @amueller

thomasjpfan added some commits Dec 19, 2018

@ogrisel

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

I also get a 401 on https://dev.azure.com/scikit-learn .

I also tried to "create" the scikit-learn organization on azure pipelines after that but it says that it's already taken:

image

who has created the scikit-learn organization?

@amueller

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

@ogrisel that's the question ;) @mickeygousset says he might have time to look into it later today or tomorrow.

@ogrisel

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

Alright. Prior to merging to master we probably need to get that resolved to avoid having failure notifications that we cannot fix in PRs.

@ogrisel

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

Edit: Although it is nicer to keep everything on azure because of the junitxml support.

I agree. I used azure pipelines for a repo I used to teach a class and i really liked it. I just had issues when installing stuff with conda but maybe those have been fixed :)

@amueller

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

the scikit-learn org apparently has been created a year ago. MS is trying to help us recover it. I suggested to use my account (t3kcit) as a backup. We could also create an sklearn org if that name is available still.

@ogrisel

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

+1 for you to try to create the sklearn org.

@amueller

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

via email:

It appears someone else already has “sklearn” 😊
I submitted a request to open up the “scikit-learn” org name. This can take a day or two. In the meantime, you can create a new org (the name isn’t super important since it can be renamed to “scikit-learn” when this name opens up) and get your pipeline working in it. This will be the org that we also increase the parallel job limit in …
Will

@amueller

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

@ogrisel

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

Can you make it public? I get a 401.

@amueller

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

I thought I did. what's your azure ID/email?

@ogrisel

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

@ogrisel

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

401 Not Authorized

Sorry, but Olivier Grisel olivier.grisel@ensta.org (ensta.org) is not authorized to access this page

@amueller

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

@amueller

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

@amueller

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

should we merge and try?

@ogrisel

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

It works. Is there a way to make it public so that people do not have to create a MS account just to see the logs?

@ogrisel

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

actually false alarm it is public. I just got a weird redirect to a password confirmation page the first time, probably because of my active session.

But if I completely log out I can still see the scikit-learn project on azure pipeline.

@ogrisel

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

I let you invite @thomasjpfan and a bunch scikit-learn core devs to admin this project.

@amueller

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

I already invited thomasjpfan ;) We need the other dev's azure IDs.

@ogrisel

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

And feel free to merge to run the tests.

@amueller amueller merged commit 2a2dc8c into scikit-learn:master Feb 21, 2019

11 checks passed

LGTM analysis: C/C++ No code changes detected
Details
LGTM analysis: JavaScript No code changes detected
Details
LGTM analysis: Python No code changes detected
Details
ci/circleci: deploy Your tests passed on CircleCI!
Details
ci/circleci: doc Your tests passed on CircleCI!
Details
ci/circleci: doc-min-dependencies Your tests passed on CircleCI!
Details
ci/circleci: lint Your tests passed on CircleCI!
Details
codecov/patch Coverage not affected when comparing 6649059...df75a17
Details
codecov/project 92.48% (-0.01%) compared to 6649059
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@amueller

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

running now :)

@ogrisel

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

The failure is random. I have seen elsewhere.

@ogrisel

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

@adrinjalali

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

we didn't intend it to be the case, but those doctests are kinda working as good tests sometimes. I wonder if we need to improve our tests somehow.

P.S. Azure Devops ID: adrin,jalali@gmail.com

@mickeygousset

This comment has been minimized.

Copy link

commented Feb 22, 2019

Please let me know if you have any other questions or blocking issues and I'll work with you to get them resolved.

xhlulu added a commit to xhlulu/scikit-learn that referenced this pull request Apr 28, 2019

[MRG] Configures Azure Pipelines (scikit-learn#13206)
* BLD: Enable azure pipelines

* Set up CI with Azure Pipelines

* BLD: Azure fix

* BLD: Azure fix

* TST Test variables

* BLD Install scikit-learn

* BLD Install scikit-learn

* BLD Fix

* BLD Test global variables

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Fix

* BLD Test

* BLD Test

* BLD Test

* BLD Test

* BLD Test

* BLD Test

* CLN Moves matrix to root

* CLN Adds coverage

* BLD Windows

* BLD Windows

* BLD Windows

* BLD Windows

* BLD Windows

* BLD Windows

* BLD Windows

* BLD Windows

* BLD Windows

* CLN Uses system conda

* CLN Uses system conda

* CLN Fixes spelling

* CLN Fixes spelling

* Trigger CI

* BLD Adds Codecov into script

* TST Test codecov

* TST Test codecov

* TST Test agent workdir

* REV Test all of sklearn

* REV Includes all tests

* TST Uses CODECOV_TOKEN

* Trigger CI

* CLN Nicer test run title name

xhlulu added a commit to xhlulu/scikit-learn that referenced this pull request Apr 28, 2019

xhlulu added a commit to xhlulu/scikit-learn that referenced this pull request Apr 28, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.