Remove AppVeyor

[chrahunt]

With Azure Pipelines we get 30 parallel jobs. Currently for a single PR we're using a maximum of 7 concurrently at a time. In practice it will be less because Linux and macOS finish pretty quickly.

With AppVeyor we get 1 parallel job between pip and setuptools.

By switching all of our AppVeyor tests to Azure Pipelines, this brings our maximum concurrent jobs per PR to 9 on Azure Pipelines and 0 on AppVeyor. This means that we can have CI progressing on a minimum of 3 PRs at a time vs previously all PRs were queuing on AppVeyor.

Not to mention we now only need to focus on 3 CI providers (Travis, Azure Pipelines, GitHub) instead of 4.

[pradyunsg]

I like this idea -- it makes sense to drop AppVeyor.

The small number of AppVeyor jobs is most evident when multiple PRs are made in a short period of time, and hampers the workflow when folks are most active.

[chrahunt]

How do we want to do this? Disable the required AppVeyor check and be mindful of any existing PRs? I think we have few enough now that it is not a big risk.

[pradyunsg]

Sounds about right to me. :)

[xavfernandez]

I'm a little torn since Appveyor seems to have a slightly different setup than Azure (I think we've already had some PR passing on Azure but not Appveyor or the opposite) which might catch (or cause ;) ) some issues that would go undetected on Azure.
And in my opinion every additionnal job is good to have.

But I also completely agree that Appveyor is often the bottleneck of our CI when multiple PR are updated concurrently, we could maybe reduce the Appveyor load to a single CPython version ?

PS: But apparently since I worked around the discrepancy between Azure & Appveyor my argument is kinda moot so I'll follow your decision ^^

[chrahunt]

If we have any regret it is straightforward to revert back, but I highly suspect that we won't. :)

[chrahunt]

I unrequired the check, and should be able to disable it once this is merged.

[pradyunsg]

Whoooops. I missed @xavfernandez's comment above (it wasn't visible when I opened this issue from the notification, since I'd been scrolled down automatically).

[xavfernandez]

No issue, let's call it fate ;)
And like Chris said it's unlikely we'll regret this bottleneck :)

[pradyunsg]

I just went ahead and dropped the push event from our AppVeyor webhook -- that should result in a green tick for master when we merge our next PR.

[chrahunt]

I unchecked "active" for the webhook, since otherwise it was still running (and failing) on PRs, e.g. on push to #7354 after release. Feel free to change it back if there is a more graceful way to handle it that we feel like investing in.

[xavfernandez]

We might want to completely delete the webhook & uninstall the github app (if it does not impact setuptols) in a near future.