Backport PR #6229 on branch v0.19.x (Remove redundant testing on Appveyor)

[grlee77]

Description

Backport of #6229

Checklist

• Docstrings for all functions
• Gallery example in ./doc/examples (new features only)
• Benchmark in ./benchmarks, if your changes aren't covered by an
  existing benchmark
• Unit tests
• Clean style in the spirit of PEP8
• Descriptive commit messages (see below)

For reviewers

• Check that the PR title is short, concise, and will make sense 1 year
  later.
• Check that new functions are imported in corresponding __init__.py.
• Check that new features, API changes, and deprecations are mentioned in
  doc/release/release_dev.rst.
• There is a bot to help automate backporting a PR to an older branch. For
  example, to backport to v0.19.x after merging, add the following in a PR
  comment: @meeseeksdev backport to v0.19.x
• To run benchmarks on a PR, add the run-benchmark label. To rerun, the label
  can be removed and then added again. The benchmark output can be checked in
  the "Actions" tab.