python setup.py test does not install packages in tests_require

[hamogu]

I found this issue in CCDproc, but I report it here because it seems to be an issue of astropy-helpers.

ccdproc requires the module pytest-capturelog and specifies that in its tests_require in setup.py.
( https://github.com/astropy/ccdproc/blob/master/setup.py#L98 )
I don't have pytest-capturelog installed on my machine and the corresponding tests fail with

        fixture 'caplog' not found                                               
        available fixtures: tmpdir_factory, pytestconfig, ccd_data, cache, recwarn, monkeypatch, triage_setup, record_xml_property, capfd, capsys, tmpdir         
        use 'py.test --fixtures [testpath]' for help on them.                    

Theses failures go away after manually installing pytest-capturelog.
Thus, it seems that either python setup.py test does not install the required dependencies or it installs them in a location where they are not found.
(My understanding is that tests_require dependencies are installed in a temporary directory only, but I might be wrong.)

There are two obvious workarounds: Installs test dependencies by hand or put them into install_requires even though they are only needed for testing.
(Note that this does not show up as an issue in the travis, because this dependency is installed by hand here https://github.com/astropy/ccdproc/blob/master/.travis.yml#L101 )

cc: @mwcraig

[embray]

To add a little more background, setuptools comes with its own test command, which knows, to a limited extent, how to locate and run tests that use the standard library unittest module. But it's typical to override this command to support other test frameworks (like py.test).

It's only setuptools' built-in test command that does anything explicitly with the test_requires setup argument. I see no reason this couldn't be supported in the test command in astropy-helpers though. The code for that is here, and I see no reason the same code couldn't be used in the astropy-helpers test command either. This will save all downloaded packages into a .eggs directory in the source checkout. The one wrinkle is that astropy-helpers' test command starts a subprocess to actually run the tests, so you have to ensure that any eggs downloaded for test_requires (and install_requires for that matter) are activated in the subprocess as well.

[eteq]

👍 to making use of this in some way in the test command. Although I think someone running the tests is unlikely to expect anything to be installed, so I would think it might be better to print out a message or not run the tests or something rather than auto-downloading.

[cmccully]

I would also like to see this functionality implemented. I would like to see the astropy_helpers module have the same behavior for install_requires and tests_require as setuptools. I use the astropy package-template, and have spent several hours searching for issues with the tests_require on the setuptools side. I hadn't realized that astropy_helpers was overriding that command. So for me, there was significantly more confusion because the package was not explicitly downloading packages rather than @eteq 's concern.

This also makes travis testing more straightforward using the current version of the package template. The default commands in the travis.yml file are to run test and egg-info. If setup.py test ignores the tests_require (and install_requires for that matter), the build will fail. This at least needs better documentation.

[cmccully]

I just implemented pull request astropy/astropy#5092 that would implement this feature. Please let me know what you think @eteq and @embray.

[astrofrog]

This has since been implemented in astropy core, so closing!

[tashrifbillah]

What could be the reason that setuptools still do not install stuff from tests_require? This is my setup.py

[embray]

@tashrifbillah I don't know why you're asking this here--this is not a general Python helpdesk. In case it helps, tests_require has been deprecated for a long time and isn't often used or supported anymore. You should simply not be using it.