New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test suite fails for the 14.0 release #360

Closed
bb-migration opened this Issue Mar 7, 2015 · 9 comments

Comments

Projects
None yet
1 participant
@bb-migration

bb-migration commented Mar 7, 2015

Originally reported by: ralphbean (Bitbucket: ralphbean, GitHub: ralphbean)


As mentioned at the bottom of #359, the 14.0 release is exhibiting test suite failures https://kojipkgs.fedoraproject.org//work/tasks/2236/9162236/build.log


@bb-migration

This comment has been minimized.

bb-migration commented Mar 8, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


I believe I understand the nature of these failures. In all three cases, the errors are triggered in easy_install.finalize_options when the system site packages isn't writable. These tests don't fail on my workstation, because I run as Administrator on a Windows box. They don't fail on Travis CI for similar reasons.

The first failure is TestUserInstallTest.test_user_install_not_implied_user_site_enabled. This test formerly tested that user install was enabled, and thus would not affect system site packages.

The second failure is TestUserInstallTest.test_user_install_not_implied_user_site_disabled. This test formerly did not invoke finalize_options but only invoked initialize_options, and checked the setting that was set during that operation. This test was poorly-written, because it was enforcing a poor behavior where excess logic was present in the initialize_options.

The third failure is TestUserInstallTest.test_setup_requires. This test formerly installed a test package to the user site packages. With the changes pertinent to 14.0, this test package now gets installed to the system site packages, or fails.

This last issue should be resolved by digging into the test and its origin and determining if the test is still relevant in light of the 14.0 release.

The first two failures probably need to be addressed by factoring out the write test from the finalize_options. I'm not sure what the spec is for putting behavior in finalize_options versus run.

In all cases, this won't be an easy fix, and I don't know when I'll get to it. @bubla, can you help with the failing tests?

For what it's worth, except for the behavior noted in the 14.0 release notes, the failing tests can be disregarded.

What's the impact of the failing tests with kojipkgs?

@bb-migration

This comment has been minimized.

bb-migration commented Mar 9, 2015

Original comment by ralphbean (Bitbucket: ralphbean, GitHub: ralphbean):


So long as they can be safely disregarded, the impact for us is not all that high. We would have to simply disable our run of the tests in our packaging process, but importantly, remember to re-enable it for future releases. It's just a helpful check for us to make sure we're not accidentally packaging something that is certain to not work for end users.

Impact level: manageable nuisance.

Thanks for the thorough reply. Should we leave the ticket open until its fully resolved? I can help test future iterations if you like.

@bb-migration

This comment has been minimized.

bb-migration commented Mar 9, 2015

Original comment by bubla (Bitbucket: bubla, GitHub: bubla):


I would like to help, but this is more about the internals of the setuptools test suites, which seem to be non-trivial.
AFAIK, the failing tests are rather regression tests than unit tests. I have a C/C++ background and I prefer this type of tests based on shell (and make) rather on the programming language. Shell makes it somehow less at cost of test richness and the the cross-platform aspect (on Windows, one would have to install msys).

@bb-migration

This comment has been minimized.

bb-migration commented Mar 14, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


It's more than just a nuisance. The test suite shouldn't be attempting (successfully or not) to write to system site packages just to run tests. I'll explore some options.

@bb-migration

This comment has been minimized.

bb-migration commented Mar 15, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


The third failure is related to #19, but the fix will be different.

@bb-migration

This comment has been minimized.

bb-migration commented Mar 15, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


Bypass the checking of site-packages when asserting not user install in site-packages. Fixes #360.

@bb-migration

This comment has been minimized.

bb-migration commented Mar 15, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


Remove regression test for Distribute issue 318. The continuing relevance of this test is questionnaible, given that user installs of setuptools no longer imply user installs of other packages. Ref #360.

@bb-migration

This comment has been minimized.

bb-migration commented Mar 15, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


Released as 14.1.1. Please report if this build doesn't fix the test failures.

@bb-migration

This comment has been minimized.

bb-migration commented Mar 15, 2015

Original comment by ralphbean (Bitbucket: ralphbean, GitHub: ralphbean):


All fixed on my end: https://kojipkgs.fedoraproject.org//work/tasks/6189/9236189/build.log

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment