[Enhancement] Support automatically running tests on file save

[yozlet]

Continuous test runners, such as pytest-watch, exist to give instant feedback through the coding process. Quoting from the above:

Whether or not you use the test-driven development method, running tests continuously is far more productive than waiting until you're finished programming to test your code. Additionally, manually running py.test each time you want to see if any tests were broken has more wait-time and cognitive overhead than merely listening for a notification.

It'd be great to have some form of continuous test running in vscode-python's testing features. At its most basic level, it could be a boolean setting which is tested on file save and triggers the existing "Run all tests" command, and that would be a good start. However, "Run all tests" can be pretty slow, so methods of making that instant feedback more instant would be helpful. Some possibilities:

• A "quick mode" for "Run all tests", using a different command/args combo setting which may do something like
  • execute a subset of tests
  • execute the test runner in "differential" mode (e.g. using something like testmon)
  • trigger test execution in a background daemon which can report a quick pass/fail (i.e. don't try and parse the full output, just tell me if I need to do a normal "Run all tests" to investigate)
• Since test discovery can take several seconds on its own: only perform test discovery on the file(s) being saved, and update the existing known set accordingly (this should probably be its own issue)
• Having an existing continuous runner (such as pytest-watch) going in the background and parsing the output
• Add "continuous mode" to vscode-python's existing pytest adapter

[luabud]

Thanks for the suggestion! We're not going to do this for now but if anyone else would find this issue useful, please let us know.