Running tests fails with newer packaging version (pulled in by setuptools)

[dvzrv]

When rebuilding 3.4.1 for Arch Linux I attempted running the tests by installing to a temporary directory and running the tests as referenced in the documentation.

That looks roughly like this:

python setup.py build

local test_dir="test_dir"
local site_packages=$(python -c "import site; print(site.getsitepackages()[0])")
python setup.py install --optimize=1 --root="${test_dir}"

PATH="${test_dir}/usr/bin:${PATH}" PYTHONPATH="${test_dir}/${site_packages}:${PYTHONPATH}" python -m unittest calmjs.tests.make_suite

Unfortunately this seems to fail on DeprecationWarning: Creating a LegacyVersion has been deprecated and will be removed in the next major release, which is logged by packaging's version.py.

These messages are interfering with:

• test_spec_toolchain_advice_apply (test_runtime.ToolchainRuntimeTestCase)
• test_spec_optional_advice (test_runtime.ToolchainRuntimeTestCase)
• test_spec_deferred_addition (test_runtime.ToolchainRuntimeTestCase)
• test_setup_egg_info (test_dist.DistIntegrationTestCase)
• test_build_calmjs_artifact (test_dist.DistIntegrationTestCase)

The entire build and test output can be found here:
calmjs-3.4.1-build-test.log

[dvzrv]

Applying -Wignore to the python call helps in getting around all the runtime errors.
However, I am left with the errors for

• test_setup_egg_info (test_dist.DistIntegrationTestCase)
• test_build_calmjs_artifact (test_dist.DistIntegrationTestCase)

As those are not relevant for packaging (in my case), I guess I will remove them before running the test suite.

[metatoaster]

Can you please include which specific versions of the dependencies that is causing this behavior? I don't have Arch and I can't confirm the codepath that triggers that particular construction of LegacyVersion.

My hunch is that somehow there is a mismatch of pip, setuptools and/or packaging, For an example how this can happen, see pypa/pip#9540.

Alternatively if you can provide a full traceback that lead to the error (e.g. run the test using PYTHONWARNINGS=error) might produce one.

As for the failed test_dist cases, that's probably something to be investigated (due to changes with latest versions of setuptools when producing output, most likely).

[metatoaster]

However, I am left with the errors for

* test_setup_egg_info (test_dist.DistIntegrationTestCase)

* test_build_calmjs_artifact (test_dist.DistIntegrationTestCase)

As for these failing tests, it's most certainly due to the test installation method resulting in the associated entry points not being installed in the place where distutils/setuptools knows how to pick them up.

[dvzrv]

Can you please include which specific versions of the dependencies that is causing this behavior? I don't have Arch and I can't confirm the codepath that triggers that particular construction of LegacyVersion.

They are included in the log output (all installed package versions used in the nspawn container).

Alternatively if you can provide a full traceback that lead to the error (e.g. run the test using PYTHONWARNINGS=error) might produce one.

I can!

Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/packaging/version.py", line 57, in parse
    return Version(version)
  File "/usr/lib/python3.9/site-packages/packaging/version.py", line 298, in __init__
    raise InvalidVersion("Invalid version: '{0}'".format(version))
packaging.version.InvalidVersion: Invalid version: '/build/python'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.9/unittest/__main__.py", line 18, in <module>
    main(module=None)
  File "/usr/lib/python3.9/unittest/main.py", line 100, in __init__
    self.parseArgs(argv)
  File "/usr/lib/python3.9/unittest/main.py", line 147, in parseArgs
    self.createTests()
  File "/usr/lib/python3.9/unittest/main.py", line 158, in createTests
    self.test = self.testLoader.loadTestsFromNames(self.testNames,
  File "/usr/lib/python3.9/unittest/loader.py", line 220, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/usr/lib/python3.9/unittest/loader.py", line 220, in <listcomp>
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/usr/lib/python3.9/unittest/loader.py", line 205, in loadTestsFromName
    test = obj()
  File "/build/python-calmjs/src/python-calmjs-3.4.1/test_dir/usr/lib/python3.9/site-packages/calmjs/tests/__init__.py", line 7, in make_suite
    import calmjs.registry
  File "/build/python-calmjs/src/python-calmjs-3.4.1/test_dir/usr/lib/python3.9/site-packages/calmjs/registry.py", line 25, in <module>
    from pkg_resources import working_set
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3243, in <module>
    def _initialize_master_working_set():
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3226, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3255, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 559, in _build_master
    ws = cls()
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 552, in __init__
    self.add_entry(entry)
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 608, in add_entry
    for dist in find_distributions(entry, True):
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2059, in find_on_path
    path_item_entries = _by_version_descending(filtered)
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2029, in _by_version_descending
    return sorted(names, key=_by_version, reverse=True)
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2027, in _by_version
    return [packaging.version.parse(part) for part in parts]
  File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2027, in <listcomp>
    return [packaging.version.parse(part) for part in parts]
  File "/usr/lib/python3.9/site-packages/packaging/version.py", line 59, in parse
    return LegacyVersion(version)
  File "/usr/lib/python3.9/site-packages/packaging/version.py", line 127, in __init__
    warnings.warn(
DeprecationWarning: Creating a LegacyVersion has been deprecated and will be removed in the next major release

As for these failing tests, it's most certainly due to the test installation method resulting in the associated entry points not being installed in the place where distutils/setuptools knows how to pick them up.

That is certainly a possibility. I am not sure how else I can prepare the test run though to have setuptools pick it up (I'm providing PATH and PYTHONPATH to the - temporarily - installed content). Would you have any suggestions? :)

[metatoaster]

The traceback shows that the issue was caused by from pkg_resources import working_set, which means there are some version string being picked up by Python and causing the error, it has nothing to do with calmjs.

That is certainly a possibility. I am not sure how else I can prepare the test run though to have setuptools pick it up (I'm providing PATH and PYTHONPATH to the - temporarily - installed content). Would you have any suggestions? :)

This is not really practical. setuptools is quite specific about how it looks things up by default, and given calmjs is a package literally designed for Python level package management, it makes very little sense to do a non-standard installation as that's a guarantee to gum things up.

That said, it does appear that running python using your test method does not result in the site-packages/calmjs-3.5.0.dev0-py3.9-nspkg.pth being accessed, thus ultimately resulting in the entry point not being loaded.

The test cases provided by the package are intended to only pass if the resulting installation of the package is done through a standard method, which means all the required files are presented in the correct location.

Edit: That said, I think I can add a workaround via an environment variable to disable integration tests.

[dvzrv]

The test cases provided by the package are intended to only pass if the resulting installation of the package is done through a standard method, which means all the required files are presented in the correct location.

I understand. Unfortunately, our packaging infrastructure does not allow installation to system-wide locations and afterwards running the tests.
When we're installing to "${pkgdir}" (the base directory which will serve as / for the files of the package) in the packaging context, this is done in the context of a fakeroot environment, which ensures that we can create files "as root". After creation of the package the files are installed by the user on their system (to the system-wide location).

Edit: That said, I think I can add a workaround via an environment variable to disable integration tests.

That would be much appreciated! Thank you! :)

[metatoaster]

I still cannot reproduce the exact issue despite trying my best to replicate the setup. My feeling is that there may be some other packages causing and interfering with the handling of metadata (hinted by those DeprecationWarning generated inside packaging/version.py simply by importing from pkg_resources). In any case, to assist with your specific issues (and also ensuring that this package will continue to work in Python 3.10), the latest release calmjs-3.4.2 (now up on PyPI) includes the CALMJS_SKIP_INTEGRATION environment variable for skipping those integration tests that fail using that alternative method.

I would strongly suggest trying to install the package for real using pacman after the fact and see if the issue persists, if so it may indicate some issues with the Python package metadata environment on ArchLinux.