Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'develop-inst-nodeps' activity always runs #909

Closed
stephenfin opened this issue Jul 13, 2018 · 2 comments
Closed

'develop-inst-nodeps' activity always runs #909

stephenfin opened this issue Jul 13, 2018 · 2 comments

Comments

@stephenfin
Copy link
Contributor

@stephenfin stephenfin commented Jul 13, 2018

When a project is using [tox] skipsdist = True and [testenv] usedevelop = True, tox will attempt to determine when it needs to reinstall the package (the develop-inst-nodeps activity) and when it does not (the develop-inst-noop target). It does this by comparing the last modified timestamps of the setup.py or setup.cfg files with that of the .egg-info directory.

tox/src/tox/venv.py

Lines 253 to 257 in b1e526d

if not self._needs_reinstall(setupdir, action):
action.setactivity("develop-inst-noop", setupdir)
return
action.setactivity("develop-inst-nodeps", setupdir)
extraopts = ["--no-deps"]

Unfortunately, this is a flawed check as, even after running the develop-inst-nodeps activity (which executes a variant of pip install -e $setupdir --no-deps) the timestamp of the .egg-info directory is not updated. This is a particularly irritating issue for larger projects when the install step can take many seconds or even minutes.

The ideal fix would be in setuptools and a patch to this effect has been submitted. However, it will take time for this to filter down to tox and, IMO, we should fix this ourselves in the interim.

@obestwalter
Copy link
Member

@obestwalter obestwalter commented Jul 13, 2018

Hi @stephenfin, I agree. This is a long standing issue and it would be great if someone would find the time to fix this properly. There is whole project collecting problems in that area, if you are interested: https://github.com/tox-dev/tox/projects/9

GitHub
tox - Command line driven CI frontend and development task automation tool

stephenfin added a commit to stephenfin/tox that referenced this issue Jul 13, 2018
When a project is using 'skipsdist = True' and 'usedevelop = True', tox
will attempt to determine when it needs to reinstall the package (the
'develop-inst-nodeps' activity) and when it does not (the
'develop-inst-noop' target). It does this by comparing the last modified
timestamps of the 'setup.py' or 'setup.cfg' files with that of the
'.egg-info' directory.

Unfortunately, this is a flawed check as, even after running the
'develop-inst-nodeps' activity (which executes a variant of
'pip install -e $setupdir --no-deps') the timestamp of the '.egg-info'
directory is not updated. This is a particularly irritating issue for
larger projects when the install step can take many seconds or even
minutes.

The ideal fix would be in setuptools. However, while we wait on this, we
can fix it ourselves by simply 'touch'ing the '.egg-info' directory
after a reinstall. We do this.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Closes: tox-dev#909
stephenfin added a commit to stephenfin/tox that referenced this issue Jul 13, 2018
When a project is using 'skipsdist = True' and 'usedevelop = True', tox
will attempt to determine when it needs to reinstall the package (the
'develop-inst-nodeps' activity) and when it does not (the
'develop-inst-noop' target). It does this by comparing the last modified
timestamps of the 'setup.py' or 'setup.cfg' files with that of the
'.egg-info' directory.

Unfortunately, this is a flawed check as, even after running the
'develop-inst-nodeps' activity (which executes a variant of
'pip install -e $setupdir --no-deps') the timestamp of the '.egg-info'
directory is not updated. This is a particularly irritating issue for
larger projects when the install step can take many seconds or even
minutes.

The ideal fix would be in setuptools. However, while we wait on this, we
can fix it ourselves by simply 'touch'ing the '.egg-info' directory
after a reinstall. We do this.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Closes: tox-dev#909
gaborbernat added a commit to stephenfin/tox that referenced this issue Jul 17, 2018
When a project is using 'skipsdist = True' and 'usedevelop = True', tox
will attempt to determine when it needs to reinstall the package (the
'develop-inst-nodeps' activity) and when it does not (the
'develop-inst-noop' target). It does this by comparing the last modified
timestamps of the 'setup.py' or 'setup.cfg' files with that of the
'.egg-info' directory.

Unfortunately, this is a flawed check as, even after running the
'develop-inst-nodeps' activity (which executes a variant of
'pip install -e $setupdir --no-deps') the timestamp of the '.egg-info'
directory is not updated. This is a particularly irritating issue for
larger projects when the install step can take many seconds or even
minutes.

The ideal fix would be in setuptools. However, while we wait on this, we
can fix it ourselves by simply 'touch'ing the '.egg-info' directory
after a reinstall. We do this.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Closes: tox-dev#909
gaborbernat added a commit to stephenfin/tox that referenced this issue Jul 17, 2018
When a project is using 'skipsdist = True' and 'usedevelop = True', tox
will attempt to determine when it needs to reinstall the package (the
'develop-inst-nodeps' activity) and when it does not (the
'develop-inst-noop' target). It does this by comparing the last modified
timestamps of the 'setup.py' or 'setup.cfg' files with that of the
'.egg-info' directory.

Unfortunately, this is a flawed check as, even after running the
'develop-inst-nodeps' activity (which executes a variant of
'pip install -e $setupdir --no-deps') the timestamp of the '.egg-info'
directory is not updated. This is a particularly irritating issue for
larger projects when the install step can take many seconds or even
minutes.

The ideal fix would be in setuptools. However, while we wait on this, we
can fix it ourselves by simply 'touch'ing the '.egg-info' directory
after a reinstall. We do this.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Closes: tox-dev#909
gaborbernat added a commit that referenced this issue Jul 17, 2018
When a project is using `skipsdist = True` and `usedevelop = True`, tox will attempt to determine when it needs to reinstall the package (the `develop-inst-nodeps` activity) and when it does not (the `develop-inst-noop` target). It does this by comparing the last modified timestamps of the 'setup.py' or 'setup.cfg' files with that of the `.egg-info` directory.

Unfortunately, this is a flawed check as, even after running the `develop-inst-nodeps` activity (which executes a variant of `pip install -e $setupdir --no-deps`) the timestamp of the `.egg-info` directory is not updated. This is a particularly irritating issue for larger projects when the install step can take many seconds or even minutes.

The ideal fix would be in setuptools. However, while we wait on this, we can fix it ourselves by simply `touch`ing the `.egg-info` directory after a reinstall. We do this.
@gaborbernat
Copy link
Member

@gaborbernat gaborbernat commented Jul 17, 2018

Fixed via #910

asottile added a commit that referenced this issue Jul 17, 2018
When a project is using `skipsdist = True` and `usedevelop = True`, tox will attempt to determine when it needs to reinstall the package (the `develop-inst-nodeps` activity) and when it does not (the `develop-inst-noop` target). It does this by comparing the last modified timestamps of the 'setup.py' or 'setup.cfg' files with that of the `.egg-info` directory.

Unfortunately, this is a flawed check as, even after running the `develop-inst-nodeps` activity (which executes a variant of `pip install -e $setupdir --no-deps`) the timestamp of the `.egg-info` directory is not updated. This is a particularly irritating issue for larger projects when the install step can take many seconds or even minutes.

The ideal fix would be in setuptools. However, while we wait on this, we can fix it ourselves by simply `touch`ing the `.egg-info` directory after a reinstall. We do this.
guykisel added a commit to guykisel/inline-plz that referenced this issue Sep 24, 2018
This PR updates [tox](https://pypi.org/project/tox) from **3.1.2** to **3.4.0**.



<details>
  <summary>Changelog</summary>
  
  
   ### 3.4.0
   ```
   ------------------

Bugfixes
^^^^^^^^

- add ``--exists-action w`` to default pip flags to handle better VCS dependencies (`pip documentation on this &lt;https://pip.pypa.io/en/latest/reference/pip/exists-action-option&gt;`_) - by :user:`gaborbernat` (`503 &lt;https://github.com/tox-dev/tox/issues/503&gt;`_)
- instead of assuming the Python version from the base python name ask the interpreter to reveal the version for the ``ignore_basepython_conflict`` flag - by :user:`gaborbernat` (`908 &lt;https://github.com/tox-dev/tox/issues/908&gt;`_)
- PEP-517 packaging fails with sdist already exists, fixed via ensuring the dist folder is empty before invoking the backend and `pypa/setuptools 1481 &lt;https://github.com/pypa/setuptools/pull/1481&gt;`_ - by :user:`gaborbernat` (`1003 &lt;https://github.com/tox-dev/tox/issues/1003&gt;`_)


Features
^^^^^^^^

- add ``commands_pre`` and ``commands_post`` that run before and after running
  the ``commands`` (setup runs always, commands only if setup suceeds, teardown always - all
  run until the first failing command)  - by :user:`gaborbernat` (`167 &lt;https://github.com/tox-dev/tox/issues/167&gt;`_)
- ``pyproject.toml`` config support initially by just inline the tox.ini under ``tool.tox.legacy_tox_ini`` key; config source priority order is ``pyproject.toml``, ``tox.ini`` and then ``setup.cfg`` - by :user:`gaborbernat` (`814 &lt;https://github.com/tox-dev/tox/issues/814&gt;`_)
- use the os environment variable ``TOX_SKIP_ENV`` to filter out tox environment names from the run list (set by ``envlist``)  - by :user:`gaborbernat` (`824 &lt;https://github.com/tox-dev/tox/issues/824&gt;`_)
- always set ``PIP_USER=0`` (do not install into the user site package, but inside the virtual environment created) and ``PIP_NO_DEPS=0`` (installing without dependencies can cause broken package installations) inside tox - by :user:`gaborbernat` (`838 &lt;https://github.com/tox-dev/tox/issues/838&gt;`_)
- tox will inject some environment variables that to indicate a command is running within tox: ``TOX_WORK_DIR`` env var is set to the tox work directory,
  ``TOX_ENV_NAME`` is set to the current running tox environment name, ``TOX_ENV_DIR`` is set to the current tox environments working dir - by :user:`gaborbernat` (`847 &lt;https://github.com/tox-dev/tox/issues/847&gt;`_)
- While running tox invokes various commands (such as building the package, pip installing dependencies and so on), these were printed in case they failed as Python arrays. Changed the representation to a shell command, allowing the users to quickly replicate/debug the failure on their own - by :user:`gaborbernat` (`851 &lt;https://github.com/tox-dev/tox/issues/851&gt;`_)
- skip missing interpreters value from the config file can now be overridden via the ``--skip-missing-interpreters`` cli flag - by :user:`gaborbernat` (`903 &lt;https://github.com/tox-dev/tox/issues/903&gt;`_)
- keep additional environments config order when listing them - by :user:`gaborbernat` (`921 &lt;https://github.com/tox-dev/tox/issues/921&gt;`_)
- allow injecting config value inside the ini file dependent of the fact that we&#39;re connected to an interactive shell or not  - by :user:`gaborbernat` (`947 &lt;https://github.com/tox-dev/tox/issues/947&gt;`_)
- do not build sdist if skip install is specified for the envs to be run - by :user:`gaborbernat` (`974 &lt;https://github.com/tox-dev/tox/issues/974&gt;`_)
- when verbosity level increases above two start passing through verbosity flags to pip - by :user:`gaborbernat` (`982 &lt;https://github.com/tox-dev/tox/issues/982&gt;`_)
- when discovering the interpreter to use check if the tox host Python matches and use that if so - by :user:`gaborbernat` (`994 &lt;https://github.com/tox-dev/tox/issues/994&gt;`_)
- ``-vv`` will print out why a virtual environment is re-created whenever this operation is triggered - by :user:`gaborbernat` (`1004 &lt;https://github.com/tox-dev/tox/issues/1004&gt;`_)


Documentation
^^^^^^^^^^^^^

- clarify that ``python`` and ``pip`` refer to the virtual environments executable - by :user:`gaborbernat` (`305 &lt;https://github.com/tox-dev/tox/issues/305&gt;`_)
- add Sphinx and mkdocs example of generating documentation via tox - by :user:`gaborbernat` (`374 &lt;https://github.com/tox-dev/tox/issues/374&gt;`_)
- specify that ``setup.cfg`` tox configuration needs to be inside the ``tox:tox`` namespace - by :user:`gaborbernat` (`545 &lt;https://github.com/tox-dev/tox/issues/545&gt;`_)
   ```
   
  
  
   ### 3.3.0
   ```
   ------------------

Bugfixes
^^^^^^^^

- fix ``TOX_LIMITED_SHEBANG`` when running under python3 - by :user:`asottile` (`931 &lt;https://github.com/tox-dev/tox/issues/931&gt;`_)


Features
^^^^^^^^

- `PEP-517 &lt;https://www.python.org/dev/peps/pep-0517/&gt;`_ source distribution support (create a
  ``.package`` virtual environment to perform build operations inside) by :user:`gaborbernat` (`573 &lt;https://github.com/tox-dev/tox/issues/573&gt;`_)
- `flit &lt;https://flit.readthedocs.io&gt;`_ support via implementing ``PEP-517`` by :user:`gaborbernat` (`820 &lt;https://github.com/tox-dev/tox/issues/820&gt;`_)
- packaging now is exposed as a hook via ``tox_package(session, venv)`` - by :user:`gaborbernat` (`951 &lt;https://github.com/tox-dev/tox/issues/951&gt;`_)


Miscellaneous
^^^^^^^^^^^^^

- Updated the VSTS build YAML to use the latest jobs and pools syntax - by :user:`davidstaheli` (`955 &lt;https://github.com/tox-dev/tox/issues/955&gt;`_)
   ```
   
  
  
   ### 3.2.1
   ```
   ------------------

Bugfixes
^^^^^^^^

- ``--parallel--safe-build`` no longer cleans up its folders (``distdir``, ``distshare``, ``log``). - by :user:`gaborbernat` (`849 &lt;https://github.com/tox-dev/tox/issues/849&gt;`_)
   ```
   
  
  
   ### 3.2.0
   ```
   ------------------

Features
^^^^^^^^

- Switch pip invocations to use the module ``-m pip`` instead of direct invocation. This could help
  avoid some of the shebang limitations.  - by :user:`gaborbernat` (`935 &lt;https://github.com/tox-dev/tox/issues/935&gt;`_)
- Ability to specify package requirements for the tox run via the ``tox.ini`` (``tox`` section under key ``requires`` - PEP-508 style): can be used to specify both plugin requirements or build dependencies. - by :user:`gaborbernat` (`783 &lt;https://github.com/tox-dev/tox/issues/783&gt;`_)
- Allow to run multiple tox instances in parallel by providing the
  ``--parallel--safe-build`` flag. - by :user:`gaborbernat` (`849 &lt;https://github.com/tox-dev/tox/issues/849&gt;`_)
   ```
   
  
  
   ### 3.1.3
   ```
   ------------------

Bugfixes
^^^^^^^^

- A caching issue that caused the ``develop-inst-nodeps`` action, which
  reinstalls the package under test, to always run has been resolved. The
  ``develop-inst-noop`` action, which, as the name suggests, is a no-op, will now
  run unless there are changes to ``setup.py`` or ``setup.cfg`` files that have
  not been reflected - by stephenfin (`909 &lt;https://github.com/tox-dev/tox/issues/909&gt;`_)


Features
^^^^^^^^

- Python version testenvs are now automatically detected instead of comparing
  against a hard-coded list of supported versions.  This enables ``py38`` and
  eventually ``py39`` / ``py40`` / etc. to work without requiring an upgrade to
  ``tox``.  As such, the following public constants are now deprecated
  (and scheduled for removal in ``tox`` 4.0: ``CPYTHON_VERSION_TUPLES``,
  ``PYPY_VERSION_TUPLES``, ``OTHER_PYTHON_INTERPRETERS``, and ``DEFAULT_FACTORS`` -
  by :user:`asottile` (`914 &lt;https://github.com/tox-dev/tox/issues/914&gt;`_)


Documentation
^^^^^^^^^^^^^

- Add a system overview section on the index page that explains briefly how tox works -
  by :user:`gaborbernat`. (`867 &lt;https://github.com/tox-dev/tox/issues/867&gt;`_)
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/tox
  - Changelog: https://pyup.io/changelogs/tox/
  - Docs: https://tox.readthedocs.org/
</details>
@tox-dev tox-dev locked and limited conversation to collaborators Jan 14, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants