Try using 8786 throughout all of CLI tests

[mrocklin]

If we fail, let's fail hard.

[github-actions]

Unit Test Results

       18 files  +       6         18 suites  +6   9h 14m 18s ⏱️ + 3h 7m 19s
  2 697 tests +       3    2 612 ✔️ +     15       81 💤  -   11  4 ❌  - 1 
24 113 runs  +8 009  22 852 ✔️ +7 606  1 257 💤 +405  4 ❌  - 2 

For more details on these failures, see this check.

Results for commit a45169e. ± Comparison against base commit 073ccbd.

[fjetter]

What's the benefit of doing this?

[mrocklin]

My sense is that we're not doing this so that we don't have tests conflicting with each other. But really, tests should be cleaning up anyway. Rather than try to avoid this problem, let's become even more sensitive to it so that we can identify what's wrong and fix it.

[fjetter]

There is some value in using distinct ports as well, though. I like running my tests locally using pytest-xdist. If we fix all ports within this file, we can no longer run them in parallel.
We're not doing this in CI and I rarely run CLI tests locally so I'm OK with this