tox.ini template is needlessly slow at linting with flake8

[aspiers]

IIUC src/pyscaffold/templates/tox_ini.template causes tox -e flake8 to do an install even though flake8 is linting the source files directly. If I'm correct, this should be fixed by adding skip_install = True to the [testenv:flake8] section.

Also note that this should work even if [testenv] has usedevelop = True: tox-dev/tox#572

See also #161.

[FlorianWilhelm]

Thanks @aspiers for pointing that out. We are always looking into speeding things up.
@abravalheri: Do you also think we should add skip_install = True permanently to PyScaffold [testenv:flake8] section and the template or could there be negative side effects when we later want to run the tests in parallel?

[abravalheri]

Well I don't see any trouble, but the best would be doing some tests first.

Right now, I have managed to isolate all the tests in test_install using pytest-virtualenv (abravalheri/pyscaffold@issue-119), but we still have some problems with "thread-safety" related to the access to the file system in other tests (test_create_project_with_cookiecutter, test_cli_with_cookiecutter, test_pretend_chdir) related to access to the file system and caplog... (Unfortunately I could not notice it locally, so I guess this just occurs when you have a big amount of CPU cores available).

As soon as I get the test suite running with xdist, I could try try this...

[abravalheri]

I did an experiment using the following tox.ini on top of #174 and everything seems to be fine.

[tox]
minversion = 1.8
envlist = pytest,flake8

[testenv:pytest]
passenv =
    HOME
commands =
    py.test {posargs}
deps =
    pytest
    pytest-cov
    django
    cookiecutter
    pytest-virtualenv
    pytest-xdist
# We keep pytest-xdist in the test dependencies, so the developer can easily
# opt-in for distributed tests by adding, for example, the `-n 15` arguments in
# the command-line.

[testenv:flake8]
skip_install = true
deps =
  flake8
commands =
  flake8 src/pyscaffold tests/

$ tox -- -n 10
...
pytest: commands succeeded
flake8: commands succeeded
  congratulations :)

[FlorianWilhelm]

Okay, looks like skip_install = true should then be added to PyScaffold and the template. How about the usedevelop = True flag? Should we only rather add this as a comment to the template?

[abravalheri]

I agree that since PyScaffold primary purpose is to create package distributions, we should try to keep the tests as realistic as possible, and therefore generate and install fresh packages for the test venv as it would be done in production.

The usedevelop = True comment, together with a warning about how this can lead to fase-negative tests is a good compromise as you suggested in #161.

[abravalheri]

Hi @FlorianWilhelm , personally, I believe we should add it just in the template.
For PyScaffold itself, I think running pre-commit run -a in the way it is configured now is more powerful, since other checks also take place not only flake8.

[FlorianWilhelm]

@abravalheri, I added it to the template and made a PR. I think we can close this now.

[aspiers]

Cool, thanks!