Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop after the first installdeps hook succeeds #450

Merged
merged 3 commits into from Mar 27, 2017
Merged

Stop after the first installdeps hook succeeds #450

merged 3 commits into from Mar 27, 2017

Conversation

@asottile
Copy link
Member

@asottile asottile commented Feb 4, 2017

By changing this hook to firstresult=True, a plugin may override this step and avoid running the base step

I'd like to write a plugin which overwrites the install_deps command, I've got it mostly working except when I run it it triggers dependency install twice (the second time is a noop, but a slow noop).

tox2.ini

[tox]
envlist = py27     
tox_pip_extensions_ext_venv_update = true

[testenv]
deps =
    pre-commit
    setuptools
commands =
    pip freeze --all

Current behaviour

$ tox -v -r -e py27 -c tox2.ini
using tox.ini: /home/asottile/workspace/tox-pip-extensions/tox2.ini
using tox-2.6.0 from /home/asottile/workspace/tox-pip-extensions/venv/local/lib/python2.7/site-packages/tox/__init__.pyc
GLOB sdist-make: /home/asottile/workspace/tox-pip-extensions/setup.py
  /home/asottile/workspace/tox-pip-extensions$ /home/asottile/workspace/tox-pip-extensions/venv/bin/python /home/asottile/workspace/tox-pip-extensions/setup.py sdist --formats=zip --dist-dir /home/asottile/workspace/tox-pip-extensions/.tox/dist >/home/asottile/workspace/tox-pip-extensions/.tox/log/tox-0.log
py27 recreate: /home/asottile/workspace/tox-pip-extensions/.tox/py27
  /home/asottile/workspace/tox-pip-extensions/.tox$ /home/asottile/workspace/tox-pip-extensions/venv/bin/python -m virtualenv --python /home/asottile/workspace/tox-pip-extensions/venv/bin/python2.7 py27 >/home/asottile/workspace/tox-pip-extensions/.tox/py27/log/py27-0.log
py27 bootstrap: venv-update>=2
  /home/asottile/workspace/tox-pip-extensions$ /home/asottile/workspace/tox-pip-extensions/.tox/py27/bin/pip install venv-update>=2 >/home/asottile/workspace/tox-pip-extensions/.tox/py27/log/py27-1.log
py27 installdeps: pre-commit, setuptools
  /home/asottile/workspace/tox-pip-extensions$ /home/asottile/workspace/tox-pip-extensions/.tox/py27/bin/pip-faster install pre-commit setuptools --prune >/home/asottile/workspace/tox-pip-extensions/.tox/py27/log/py27-2.log
py27 installdeps: pre-commit, setuptools
  /home/asottile/workspace/tox-pip-extensions$ /home/asottile/workspace/tox-pip-extensions/.tox/py27/bin/pip-faster install pre-commit setuptools >/home/asottile/workspace/tox-pip-extensions/.tox/py27/log/py27-3.log
py27 inst: /home/asottile/workspace/tox-pip-extensions/.tox/dist/tox-pip-extensions-0.0.0.zip
  /home/asottile/workspace/tox-pip-extensions$ /home/asottile/workspace/tox-pip-extensions/.tox/py27/bin/pip-faster install /home/asottile/workspace/tox-pip-extensions/.tox/dist/tox-pip-extensions-0.0.0.zip >/home/asottile/workspace/tox-pip-extensions/.tox/py27/log/py27-4.log
  /home/asottile/workspace/tox-pip-extensions$ /home/asottile/workspace/tox-pip-extensions/.tox/py27/bin/pip freeze >/home/asottile/workspace/tox-pip-extensions/.tox/py27/log/py27-5.log
py27 installed: appdirs==1.4.0,aspy.yaml==0.2.2,cached-property==1.3.0,functools32==3.2.3.post2,jsonschema==2.5.1,nodeenv==1.1.2,packaging==16.8,pluggy==0.4.0,pre-commit==0.12.2,py==1.4.32,pyparsing==2.1.10,PyYAML==3.12,six==1.10.0,tox==2.6.0,tox-pip-extensions==0.0.0,venv-update==2.0.0,virtualenv==15.1.0
py27 runtests: PYTHONHASHSEED='1109614621'
py27 runtests: commands[0] | pip freeze --all
  /home/asottile/workspace/tox-pip-extensions$ /home/asottile/workspace/tox-pip-extensions/.tox/py27/bin/pip freeze --all 
appdirs==1.4.0
aspy.yaml==0.2.2
cached-property==1.3.0
functools32==3.2.3.post2
jsonschema==2.5.1
nodeenv==1.1.2
packaging==16.8
pip==9.0.1
pluggy==0.4.0
pre-commit==0.12.2
py==1.4.32
pyparsing==2.1.10
PyYAML==3.12
setuptools==34.1.1
six==1.10.0
tox==2.6.0
tox-pip-extensions==0.0.0
venv-update==2.0.0
virtualenv==15.1.0
wheel==0.29.0
______________________________________________________________________________________ summary ______________________________________________________________________________________
  py27: commands succeeded
  congratulations :)

The important bit here being:

py27 installdeps: pre-commit, setuptools
  /home/asottile/workspace/tox-pip-extensions$ /home/asottile/workspace/tox-pip-extensions/.tox/py27/bin/pip-faster install pre-commit setuptools --prune >/home/asottile/workspace/tox-pip-extensions/.tox/py27/log/py27-2.log
py27 installdeps: pre-commit, setuptools
  /home/asottile/workspace/tox-pip-extensions$ /home/asottile/workspace/tox-pip-extensions/.tox/py27/bin/pip-faster install pre-commit setuptools >/home/asottile/workspace/tox-pip-extensions/.tox/py27/log/py27-3.log

A -vv into that (you'll notice the second install is unnecessary).

py27 installdeps: pre-commit, setuptools
setting PATH=/home/asottile/workspace/tox-pip-extensions/.tox/py27/bin:/home/asottile/workspace/tox-pip-extensions/venv/bin:/home/asottile/bin:/home/asottile/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
  /home/asottile/workspace/tox-pip-extensions$ /home/asottile/workspace/tox-pip-extensions/.tox/py27/bin/pip-faster install pre-commit setuptools --prune 
Collecting pre-commit
  slow: full search for unpinned requirement pre-commit
  Using cached pre_commit-0.12.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools in ./.tox/py27/lib/python2.7/site-packages
Collecting nodeenv>=0.11.1 (from pre-commit)
  slow: full search for unpinned requirement nodeenv>=0.11.1 (from pre-commit)
Collecting virtualenv (from pre-commit)
  slow: full search for unpinned requirement virtualenv (from pre-commit)
  Using cached virtualenv-15.1.0-py2.py3-none-any.whl
Collecting pyyaml (from pre-commit)
  slow: full search for unpinned requirement pyyaml (from pre-commit)
Collecting jsonschema (from pre-commit)
  slow: full search for unpinned requirement jsonschema (from pre-commit)
  Using cached jsonschema-2.5.1-py2.py3-none-any.whl
Collecting aspy.yaml (from pre-commit)
  slow: full search for unpinned requirement aspy.yaml (from pre-commit)
  Using cached aspy.yaml-0.2.2-py2.py3-none-any.whl
Collecting cached-property (from pre-commit)
  slow: full search for unpinned requirement cached-property (from pre-commit)
  Using cached cached_property-1.3.0-py2.py3-none-any.whl
Requirement already satisfied: six>=1.6.0 in ./.tox/py27/lib/python2.7/site-packages (from setuptools)
Requirement already satisfied: appdirs>=1.4.0 in ./.tox/py27/lib/python2.7/site-packages (from setuptools)
Requirement already satisfied: packaging>=16.8 in ./.tox/py27/lib/python2.7/site-packages (from setuptools)
Collecting functools32; python_version == "2.7" (from jsonschema->pre-commit)
  slow: full search for unpinned requirement functools32; python_version == "2.7" (from jsonschema->pre-commit)
Requirement already satisfied: pyparsing in ./.tox/py27/lib/python2.7/site-packages (from packaging>=16.8->setuptools)
Installing collected packages: nodeenv, virtualenv, pyyaml, functools32, jsonschema, aspy.yaml, cached-property, pre-commit
Successfully installed aspy.yaml-0.2.2 cached-property-1.3.0 functools32-3.2.3.post2 jsonschema-2.5.1 nodeenv-1.1.2 pre-commit-0.12.2 pyyaml-3.12 virtualenv-15.1.0
py27 installdeps: pre-commit, setuptools
setting PATH=/home/asottile/workspace/tox-pip-extensions/.tox/py27/bin:/home/asottile/workspace/tox-pip-extensions/venv/bin:/home/asottile/bin:/home/asottile/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
  /home/asottile/workspace/tox-pip-extensions$ /home/asottile/workspace/tox-pip-extensions/.tox/py27/bin/pip-faster install pre-commit setuptools 
Requirement already satisfied: pre-commit in ./.tox/py27/lib/python2.7/site-packages
Requirement already satisfied: setuptools in ./.tox/py27/lib/python2.7/site-packages
Requirement already satisfied: nodeenv>=0.11.1 in ./.tox/py27/lib/python2.7/site-packages (from pre-commit)
Requirement already satisfied: virtualenv in ./.tox/py27/lib/python2.7/site-packages (from pre-commit)
Requirement already satisfied: pyyaml in ./.tox/py27/lib/python2.7/site-packages (from pre-commit)
Requirement already satisfied: jsonschema in ./.tox/py27/lib/python2.7/site-packages (from pre-commit)
Requirement already satisfied: aspy.yaml in ./.tox/py27/lib/python2.7/site-packages (from pre-commit)
Requirement already satisfied: cached-property in ./.tox/py27/lib/python2.7/site-packages (from pre-commit)
Requirement already satisfied: six>=1.6.0 in ./.tox/py27/lib/python2.7/site-packages (from setuptools)
Requirement already satisfied: appdirs>=1.4.0 in ./.tox/py27/lib/python2.7/site-packages (from setuptools)
Requirement already satisfied: packaging>=16.8 in ./.tox/py27/lib/python2.7/site-packages (from setuptools)
Requirement already satisfied: functools32; python_version == "2.7" in ./.tox/py27/lib/python2.7/site-packages (from jsonschema->pre-commit)
Requirement already satisfied: pyparsing in ./.tox/py27/lib/python2.7/site-packages (from packaging>=16.8->setuptools)

With my patch:

$ tox -v -r -e py27 -c tox2.ini
using tox.ini: /home/asottile/workspace/tox-pip-extensions/tox2.ini
using tox-2.6.1.dev1 from /home/asottile/workspace/tox/tox/__init__.pyc
GLOB sdist-make: /home/asottile/workspace/tox-pip-extensions/setup.py
  /home/asottile/workspace/tox-pip-extensions$ /home/asottile/workspace/tox-pip-extensions/venv/bin/python /home/asottile/workspace/tox-pip-extensions/setup.py sdist --formats=zip --dist-dir /home/asottile/workspace/tox-pip-extensions/.tox/dist >/home/asottile/workspace/tox-pip-extensions/.tox/log/tox-0.log
py27 recreate: /home/asottile/workspace/tox-pip-extensions/.tox/py27
  /home/asottile/workspace/tox-pip-extensions/.tox$ /home/asottile/workspace/tox-pip-extensions/venv/bin/python -m virtualenv --python /home/asottile/workspace/tox-pip-extensions/venv/bin/python2.7 py27 >/home/asottile/workspace/tox-pip-extensions/.tox/py27/log/py27-0.log
py27 bootstrap: venv-update>=2
  /home/asottile/workspace/tox-pip-extensions$ /home/asottile/workspace/tox-pip-extensions/.tox/py27/bin/pip install venv-update>=2 >/home/asottile/workspace/tox-pip-extensions/.tox/py27/log/py27-1.log
py27 installdeps: pre-commit, setuptools
  /home/asottile/workspace/tox-pip-extensions$ /home/asottile/workspace/tox-pip-extensions/.tox/py27/bin/pip-faster install pre-commit setuptools --prune >/home/asottile/workspace/tox-pip-extensions/.tox/py27/log/py27-2.log
py27 inst: /home/asottile/workspace/tox-pip-extensions/.tox/dist/tox-pip-extensions-0.0.0.zip
  /home/asottile/workspace/tox-pip-extensions$ /home/asottile/workspace/tox-pip-extensions/.tox/py27/bin/pip-faster install /home/asottile/workspace/tox-pip-extensions/.tox/dist/tox-pip-extensions-0.0.0.zip >/home/asottile/workspace/tox-pip-extensions/.tox/py27/log/py27-3.log
  /home/asottile/workspace/tox-pip-extensions$ /home/asottile/workspace/tox-pip-extensions/.tox/py27/bin/pip freeze >/home/asottile/workspace/tox-pip-extensions/.tox/py27/log/py27-4.log
py27 installed: appdirs==1.4.0,aspy.yaml==0.2.2,cached-property==1.3.0,functools32==3.2.3.post2,jsonschema==2.5.1,nodeenv==1.1.2,packaging==16.8,pluggy==0.4.0,pre-commit==0.12.2,py==1.4.32,pyparsing==2.1.10,PyYAML==3.12,six==1.10.0,tox==2.6.0,tox-pip-extensions==0.0.0,venv-update==2.0.0,virtualenv==15.1.0
py27 runtests: PYTHONHASHSEED='1881738361'
py27 runtests: commands[0] | pip freeze --all
  /home/asottile/workspace/tox-pip-extensions$ /home/asottile/workspace/tox-pip-extensions/.tox/py27/bin/pip freeze --all 
appdirs==1.4.0
aspy.yaml==0.2.2
cached-property==1.3.0
functools32==3.2.3.post2
jsonschema==2.5.1
nodeenv==1.1.2
packaging==16.8
pip==9.0.1
pluggy==0.4.0
pre-commit==0.12.2
py==1.4.32
pyparsing==2.1.10
PyYAML==3.12
setuptools==34.1.1
six==1.10.0
tox==2.6.0
tox-pip-extensions==0.0.0
venv-update==2.0.0
virtualenv==15.1.0
wheel==0.29.0
______________________________________________________________________________________ summary ______________________________________________________________________________________
  py27: commands succeeded
  congratulations :)

Hook implementation

A bit of a WIP but it mostly does what I want:

@hookimpl
def tox_configure(config):
    cfg = six.moves.configparser.ConfigParser()
    cfg.read(config.option.configfile)
    configured = tuple(sorted(
        k[len(TOX_KEY):]
        for k, v in cfg.items('tox')
        if k.startswith(TOX_KEY) and _assert_true_value(k, v)
    ))
    config.pip_extensions = configured


@hookimpl(tryfirst=True)
def tox_testenv_install_deps(venv, action):
    config = venv.session.config
    extensions = config.pip_extensions

    # If there's nothing special for us to do, defer to other plugins
    if not extensions:
        return None

    def _install(step, deps):
        if deps:
            action.setactivity(step, ', '.join(map(str, deps)))
            venv._install(deps, action=action)

    # Install the bootstrap dependencies
    bootstrap = config.toxinidir.join('requirements-bootstrap.txt')
    if bootstrap.exists():
        bootstrap_deps = ['-r{}'.format(bootstrap)]
    else:
        bootstrap_deps = [INSTALL_DEPS[ext] for ext in extensions]
    _install('bootstrap', bootstrap_deps)

    # Run the deps install step with the custom deps install command
    venv.envconfig.install_command[:] = INSTALL_DEPS_CMD[extensions]
    _install('installdeps', venv._getresolvedeps())

    # Run the rest of the setup with the custom install command
    venv.envconfig.install_command[:] = INSTALL_CMD[extensions]

    # Indicate to the plugin framework that we have handled installation
    return True
@asottile
Copy link
Member Author

@asottile asottile commented Feb 8, 2017

@obestwalter any feedback on this PR? If this looks good I'd like to also do the same for the virtualenv creation (which is less of a noop and more of a problem :D).

@obestwalter
Copy link
Member

@obestwalter obestwalter commented Feb 10, 2017

Hi @asottile

Disclaimer: I have not worked my way very far into the internals of tox/pluggy yet. I also have not much experience in estimating the potential backwards compatibility breakhages changes like yours can introduce. The little time I was able to spend with tox until now was gathering experience with how other contributors and users use and understand tox. So atm I am mainly an OSS gardener who is trying to tidy up a bit and keeps the weeds back but does not yet understand much about the deeper organisation :)

One thing I am pretty sure about: we need a good reason to change the definition of a hook as this is part of the API and other implementers rely on its current behaviour. How many those are I don't know. I also don't know if maybe all implementers of that hook have the same problem like you and you would solve everybodies problems with this change.

I had a closer look now anyway and one thing is that this hook is marked experimental, but because of the use of the venv and session objects - see changelog entry:

introduce experimental tox_testenv_create(venv, action) and tox_testenv_install_deps(venv, action) hooks to allow plugins to do additional work on creation or installing deps. These hooks are experimental mainly because of the involved "venv" and session objects whose current public API is not fully guranteed.

Now for working towards a decision if we want to change this - I only have questions:

Is there another way to reach your goal or is there a strong case for really changing that hook for all users? I have no idea, because I started exactly now to actually think about hooks in tox and their implications.

Why has no other tox hook firstresult=True? Is this maybe by design as it would limit the scope in which the hooks could be used?

Why was this hook not marked with firstresult=True to start with if this is the right thing to do? Is there maybe a valid use case why this hook might be called more than once? @brmzkw sent the original PR, maybe he can chip in about this? Otherwise @hpk42 might be able to say if this change is good and safe.

So ... more questions than answers I am afraid. I definitely don't feel qualified to make an informed decision about the validity of this change.

@obestwalter
Copy link
Member

@obestwalter obestwalter commented Feb 10, 2017

Correction: tox_get_python_executable has firstresult=True set ... so this wouldn't be the only hook doing this.

@asottile
Copy link
Member Author

@asottile asottile commented Feb 12, 2017

I typed out a really long response to this but I guess I never posted it, here's an attempt to remember what I wrote the first time :)

I did some searching around for implementers of this hook:

  • I couldn't find a single implementer of tox_testenv_create (searched through all of github code search and 3-4 pages of google results).
  • For tox_testenv_install_deps, I found a single plugin: tox-debian-plugin. Unlike my use case, this plugin doesn't want to override the existing tox installation, but instead wants to augment it by installing debian dependencies before python dependencies. With the change I'm proposing here, their plugin will continue to work as expected because their plugin returns None (a lack of return in python implicitly returns None. pluggy will continue to run other implementations if a None value is returned.

I tried implementing my hook in another way, but the only possible way I could was to monkeypatch as a side-effect of plugin importing, which seems to defeat the purpose of a plugin system entirely :)

I think the argument for tox_testenv_create is a much stronger one as it isn't currently possible to implement a plugin which sets up the virtualenv without tox's implementation clobbering that. The argument for tox_testenv_install_deps is similar (and would be stronger if I were using something not-pip-compliant to install dependencies).

Definitely would love to get input from @hpk42 and @brmzkw!

(I think this typed up version was better than the first time I typed it up!)

@brmzkw
Copy link

@brmzkw brmzkw commented Feb 13, 2017

From what I understand, I made the original PR. Can you give me a link to it? I can't find it, and I can't remember having done anything on tox lately. Seeing some code or discussion might refresh my memory and I might be able to give more insights.

Thanks,

@obestwalter
Copy link
Member

@obestwalter obestwalter commented Feb 13, 2017

Ups - sory Julien, I misread the diff in the Changelog adding this - your name was mentioned in the line above ...

@asottile
Copy link
Member Author

@asottile asottile commented Feb 21, 2017

Any updates on this?

@asottile
Copy link
Member Author

@asottile asottile commented Feb 28, 2017

Bump: 1 more week @hpk42 @obestwalter <3

@obestwalter
Copy link
Member

@obestwalter obestwalter commented Mar 6, 2017

@asottile o.k. I let your enthusiasm infect me and will have a go at determining wether merging this will cause the end of the world as we know it for existing users. Expect a reply in the next few days :)

@asottile
Copy link
Member Author

@asottile asottile commented Mar 6, 2017

<3, I've also opened #469 such that both can be considered at the same time (if agreeable)

@obestwalter
Copy link
Member

@obestwalter obestwalter commented Mar 10, 2017

O.k. I had another look and I would be o.k. to merge this and #469 after the next minor release, which will happen soonish.

I say after, because I don't want this to block the next release of tox and detox and I still think that @hpk42 should greenlight this, as I don't know the reasoning behind why this hook is not marked that way already and he has way more experience with potentially breaking changes like this.

@asottile
Copy link
Member Author

@asottile asottile commented Mar 20, 2017

@hpk42 thoughts? We'd like to start being able to take advantage of this :)

@fschulze
Copy link
Contributor

@fschulze fschulze commented Mar 25, 2017

So far the result of tox_testenv_install_deps isn't used, so I think it's valid to change the behaviour to stop when something is returned. It should just be documented properly. Maybe tox itself should check the return value and do something with it. Maybe we could define it to be a string which is printed, then the plugin can return something like a status message. In that case tox should say that some plugins may have been skipped. Unfortunately we can't check which ones with the current pluggy API.

@asottile
Copy link
Member Author

@asottile asottile commented Mar 25, 2017

@obestwalter I've tried and failed to contact @hpk42 (I imagine on holiday or such?) both here and irc :(

Given there hasn't been any objection to this (and the related change) in the last almost-two months, is this mergeable? I've been waiting quite patiently without action items but this is getting to be quite a long wait time for an "active" project (and a very simple PR).

I'd really love to start being able to use these two hooks which are unusable in the current state :)

@obestwalter
Copy link
Member

@obestwalter obestwalter commented Mar 25, 2017

Thanks @fschulze - so let's just go ahead with this. @asottile can you work in the suggestions of @fschulze? If you get this done in the next days I don't really see why we shouldn't sneek it into 2.7 then.

@obestwalter
Copy link
Member

@obestwalter obestwalter commented Mar 25, 2017

@asottile thanks for your patience. This project is definitely active without quotes but your PR proposes a change to an existing API (albeit marked experimental), so this is not to be taken lightly. I wanted to give some time for feedback from the old hands here. The way it looks atm, you can expect your PRs to be in the 2.7.0 release next week :)

@asottile
Copy link
Member Author

@asottile asottile commented Mar 25, 2017

Didn't mean to call the project inactive :D, I've definitely observed lots of activity and progress!

=====

Sweet, for now I'm going to leave out printing / additional behaviour / etc. I think that can be done as a followup -- if a hook wants to print, it can already do so now and if one wants to know whether the original hook ran, the -vvv output of tox includes sufficiently identifiable output from the builtin implementations. I hope that's reasonable?

I will spruce up the docs though, they definitely could do with some additional prose for these two hooks.

@asottile
Copy link
Member Author

@asottile asottile commented Mar 26, 2017

I've spruced up the docstrings of the hookspecs for these two hooks. Let me know what you think :)

@obestwalter obestwalter merged commit eb33549 into tox-dev:master Mar 27, 2017
2 checks passed
@asottile asottile deleted the firstresult_for_install_deps branch Mar 28, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants