Fix test integration with IDEs

[ssbarnea]

This should fix debugging tests in both vscode and pycharm. We need to be sure that we disable pytest-cov inside pytest.ini and enable it only within tox.ini file.

This ensures that we can:

• run/debug with pytest from console (outside tox)
• run/debug from vscode
• run/debug from pycharm
• capture coverage on CI
• capture coverage with simple tox -e py (no posargs)

Related: #1912 #1909

[webknjaz]

The base args must be in pytest.ini. Maintaining them only in tox will create an inconsistent experience. tox.ini is only there for conditional overrides.

[ssbarnea]

The base args must be in pytest.ini. Maintaining them only in tox will create an inconsistent experience. tox.ini is only there for conditional overrides.

If you have an alternative that is fixing all the points noted in the description, I would be happy to see it. So far I was not the only contributor that found these issues inconvenient and I do not want to force anyone willing to contribute to alter his environment to make it work, especially as that might be even impossible without altering tracked files.

[cidrblock]

Anything we can do to ensure the first time test running/debugging experience is smooth I'll +1

[ssbarnea]

I that brings any regressions, we will fix them in follow-ups.

@cognifloyd Please test and report if this is improving the experience for you.

[cognifloyd]

Nice. Works great in pycharm.
I setup my args in pycharm's settings: -ra -l --tb=short --strict-markers
(ie "Additional Arguments" in the run templates for "autodetect" and "pytest")

[ssbarnea]

Nice. Works great in pycharm. I setup my args in pycharm's settings: -ra -l --tb=short --strict-markers (ie "Additional Arguments" in the run templates for "autodetect" and "pytest")

Probably we can put back some arguments in pytest.ini if they do not break other tools.