Enable CI builds on Visual Studio Team Services

[zooba]

BPO	33522
Nosy	@brettcannon, @gpshead, @pitrou, @zooba, @matrixise, @ammaraskar
PRs	bpo-33522: Enable CI builds on Visual Studio Team Services #6865 [3.7] bpo-33522: Enable CI builds on Visual Studio Team Services (GH-6865) #6923 [3.6] bpo-33522: Enable CI builds on Visual Studio Team Services (GH-6865) #6925 [3.7] bpo-33522: Enable CI builds on Visual Studio Team Services (GH-6865) #6926

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/zooba'
closed_at = <Date 2018-05-18.15:25:21.803>
created_at = <Date 2018-05-15.16:15:54.319>
labels = ['3.7', '3.8', 'build', 'tests']
title = 'Enable CI builds on Visual Studio Team Services'
updated_at = <Date 2018-05-18.20:16:31.409>
user = 'https://github.com/zooba'

bugs.python.org fields:

activity = <Date 2018-05-18.20:16:31.409>
actor = 'steve.dower'
assignee = 'steve.dower'
closed = True
closed_date = <Date 2018-05-18.15:25:21.803>
closer = 'gregory.p.smith'
components = ['Build', 'Tests']
creation = <Date 2018-05-15.16:15:54.319>
creator = 'steve.dower'
dependencies = []
files = []
hgrepos = []
issue_num = 33522
keywords = ['patch']
message_count = 14.0
messages = ['316677', '316732', '316817', '316873', '316924', '316929', '316950', '317003', '317006', '317019', '317029', '317030', '317034', '317058']
nosy_count = 6.0
nosy_names = ['brett.cannon', 'gregory.p.smith', 'pitrou', 'steve.dower', 'matrixise', 'ammar2']
pr_nums = ['6865', '6923', '6925', '6926']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue33522'
versions = ['Python 2.7', 'Python 3.6', 'Python 3.7', 'Python 3.8']

[zooba]

We have been provided with https://python.visualstudio.com/cpython and a measure of build machines to help us run continuous integration builds. The benefits are more reliable builds (especially Mac and Windows) and higher parallelism than our current services.

My PR adds the build definition files and fixes some test bugs to ensure we can successfully test on this platform. There are a few more manual steps to make it work against our repo, which we will do immediately after merging the changes.

[pitrou]

Can you explain what Visual Studio Team Services is?

[brettcannon]

https://www.visualstudio.com/team-services/

It's Microsoft's GitHub hosting, issue tracking, and CI/CD platform.

[zooba]

New changeset e5f41d2 by Steve Dower in branch 'master':
bpo-33522: Enable CI builds on Visual Studio Team Services (bpo-6865)
e5f41d2

[zooba]

New changeset 8965d75 by Steve Dower (Miss Islington (bot)) in branch '3.7':
bpo-33522: Enable CI builds on Visual Studio Team Services (GH-6865) (GH-6926)
8965d75

[zooba]

New changeset 0d8f83f by Steve Dower in branch '3.6':
bpo-33522: Enable CI builds on Visual Studio Team Services (GH-6865) (GH-6925)
0d8f83f

[zooba]

This is implemented and merged now. I'll leave it to core-workflow to decide whether and when to make the builds required and/or reduce use of Travis/AppVeyor.

[gpshead]

Your PRs made modifications to code and did not include a useful commit message or appropriate NEWS entries for the code modifications.

Further, VSTS is unreliable. core-workflow made a premature decision to turn off Travis and AppVeyor and make VSTS blocking.

[ammaraskar]

core-workflow made a premature decision to turn off Travis and AppVeyor and make VSTS blocking.

It looks like AppVeyor and Travis are still running as of the latest PR: #6965

Annoyingly, I don't think there's a way to make certain checks like VSTS be optional and not show up with a red cross on Github.

[matrixise]

Hi Steve

I just saw VSTS in action, it's a great job, thank you.

Compared to Travis-CI:

• is there more computers for the tests on VSTS?

• with time, what will be the reference if there is a 'green' build? TravisCI or VSTS?

Thank you again for the job.

[zooba]

• is there more computers for the tests on VSTS?

I don't know what Travis is giving us, but Microsoft has provided 20 concurrent builds. They also seem to be more powerful machines, though some caching features are missing and many builds seem to take about the same time.

• with time, what will be the reference if there is a 'green' build? TravisCI or VSTS?

This is a question for the core-workflow list. I'm deliberately staying out of this part of the discussion.

[zooba]

I don't think there's a way to make certain checks like VSTS be optional and not show up with a red cross on Github

Most of the failures I've seen have been reproducible, so I suspect they're due to race conditions that rarely occurred on Travis. Unfortunately, we are no longer watching the build bots as closely as we used to, so many platform-specific or configuration-specific failures are missed :(

[gpshead]

I've emailed support@github on this one. Their UI is horrible, "show all checks" does not show them all. It shows five at most.

It is an invisible scrolling region. We're forced to manually mouse over and scroll down in that region in order to confirm that the trustworthy checks have run.

[zooba]

GitHub has broken CSS everywhere right now :) Their oauth page doesn't show scroll bars either, which means you can't get to the "authorise" button if you're in more than a couple of organisations.