Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop dependency on Setuptools #4443

Merged
merged 12 commits into from
Mar 13, 2023
Merged

Drop dependency on Setuptools #4443

merged 12 commits into from
Mar 13, 2023

Conversation

jakirkham
Copy link
Member

@jakirkham jakirkham commented Apr 26, 2022

Instead of relying on setuptools to get functionality from pkg_resources (which is slowly being refactored away), leverage packaging for the functionality used by Conda-Build.

Edit: Looked to see whether Conda needed a similar change. Appears it only indirectly uses pkg_resources via a vendored copy of tqdm. Updating the vendored copy of tqdm would fix that issue ( conda/conda#11432 ).

Fixes #4733

@anaconda-issue-bot anaconda-issue-bot added the cla-signed [bot] added once the contributor has signed the CLA label Apr 26, 2022
@jezdez jezdez force-pushed the use_packaging branch 2 times, most recently from 74ec13d to 539d0f7 Compare May 9, 2022 19:08
@jakirkham
Copy link
Member Author

Updated with latest master. Please let me know if there's anything else needed here 🙂

@jezdez
Copy link
Member

jezdez commented Aug 1, 2022

One thing to note is that pkg_resources.parse_version falls back to the LegacyVersion which is a pre-PEP440 compatible version, and as such could still be used in one or more packages out there. In other words, this has some potential to be a breaking change for a few projects and needs to be communicated as such.

FWIW I'm in favor of this move since it's way over time to standardize on PEP440 and following.

@chenghlee Have you heard anything about how compatible we are with PEP 440 nowadays?

@jezdez jezdez requested a review from a team August 1, 2022 14:30
@kenodegard kenodegard added this to the 3.22.0 milestone Aug 1, 2022
@jakirkham
Copy link
Member Author

Open to other solutions if there is a preferred alternative

@jezdez
Copy link
Member

jezdez commented Aug 2, 2022

I've looked into more of the actual failing tests, and I think we shouldn't rush this for 3.22.0.

It's worth considering all the implications of the old LegacyVersion being used for the various non-PEP 440 conforming versions though, which happen to exist for conda packages that ship non-Python software, for example.
I do think we should support and use PEP 440 as the only standard for Python packages, but at the same time use a more relaxed version specifier for non-Python packages.

@jezdez jezdez removed this from the 3.22.0 milestone Aug 2, 2022
Instead of relying on `setuptools` to get functionality from
`pkg_resources` (which is slowly being refactored away), leverage
`packaging` for the functionality used by Conda-Build.
@jezdez
Copy link
Member

jezdez commented Jan 20, 2023

I've backported LegacyVersion to get this unblocked and get rid of the dependency on setuptools. When we'll send skeleton into the retirement, we can remove it again.

@jezdez
Copy link
Member

jezdez commented Jan 20, 2023

@jakirkham I took the liberty to amend your PR, could you take a look if that's along the lines what you had in mind?

@jezdez jezdez added this to the 3.24.0 milestone Jan 20, 2023
@jakirkham
Copy link
Member Author

Thanks Jannis! Doing a bit of vendoring to protect ourselves makes sense to me 🙂

@jezdez jezdez requested a review from jaimergp January 31, 2023 14:27
@jakirkham
Copy link
Member Author

jakirkham commented Feb 1, 2023

Am seeing this test failure in some jobs (for example). Not sure what the expect behavior was before, but that version number doesn't look ok.

It is worth noting that SymPy is on GitHub these days. So a test that was presumably for Bitbucket/Mercurial may not be as relevant (particularly since Bitbucket dropped Mercurial support).

Maybe something using Heptapod would be better (if Mercurial testing is desired).

___________________________ test_sympy[with version] ___________________________
[gw0] linux -- Python 3.8.16 /usr/share/miniconda/envs/test/bin/python
Traceback (most recent call last):
  File "/home/runner/work/conda-build/conda-build/src/tests/test_api_skeleton.py", line 189, in test_sympy
    api.skeletonize(
  File "/home/runner/work/conda-build/conda-build/src/conda_build/api.py", line 270, in skeletonize
    skeleton_return = module.skeletonize(packages, output_dir=output_dir, version=version,
  File "/home/runner/work/conda-build/conda-build/src/conda_build/skeletons/pypi.py", line 280, in skeletonize
    versions = sorted(pypi_data["releases"].keys(), key=Version)
  File "/usr/share/miniconda/envs/test/lib/python3.8/site-packages/packaging/version.py", line 197, in __init__
    raise InvalidVersion(f"Invalid version: '{version}'")
packaging.version.InvalidVersion: Invalid version: '0.5.13-hg'

@jakirkham
Copy link
Member Author

Any thoughts on what we should do here?

@jezdez
Copy link
Member

jezdez commented Mar 8, 2023

@jakirkham @jaimergp I've updated it to use the legacy version parser everywhere, I think that should be okay, assuming the tests pass

@jakirkham jakirkham closed this Mar 8, 2023
@jakirkham jakirkham reopened this Mar 8, 2023
@jakirkham
Copy link
Member Author

Thanks Jannis! 🙏

Seeing another failure just on Unix Python 3.7 CI jobs (for example). Though not sure why it is coming up here.

_________________ test_render_with_python_arg_reduces_subspace _________________
[gw0] linux -- Python 3.7.16 /usr/share/miniconda/envs/test/bin/python
Traceback (most recent call last):
  File "/usr/share/miniconda/envs/test/lib/python3.7/shutil.py", line 566, in move
    os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/share/miniconda/envs/test/conda-bld/work' -> '/usr/share/miniconda/envs/test/conda-bld/test_subspace_selection_cli_1678300329405/work'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/conda-build/conda-build/tests/cli/test_main_render.py", line 135, in test_render_with_python_arg_reduces_subspace
    main_render.execute(args)
  File "/home/runner/work/conda-build/conda-build/conda_build/cli/main_render.py", line 194, in execute
    variants=args.variants)
  File "/home/runner/work/conda-build/conda-build/conda_build/api.py", line 41, in render
    permit_unsatisfiable_variants=permit_unsatisfiable_variants)
  File "/home/runner/work/conda-build/conda-build/conda_build/render.py", line 834, in render_recipe
    m.config.compute_build_id(m.name(), m.version(), reset=reset_build_id)
  File "/home/runner/work/conda-build/conda-build/conda_build/config.py", line 615, in compute_build_id
    shutil.move(old_dir, work_dir)
  File "/usr/share/miniconda/envs/test/lib/python3.7/shutil.py", line 580, in move
    copy_function(src, real_dst)
  File "/usr/share/miniconda/envs/test/lib/python3.7/shutil.py", line 266, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/share/miniconda/envs/test/lib/python3.7/shutil.py", line 120, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/share/miniconda/envs/test/conda-bld/work'

@jezdez
Copy link
Member

jezdez commented Mar 9, 2023

Given that 3.7 isn't maintained by conda-forge and defaults anymore, I'm ready to take the chance and remove it from the test matrix. Do you have concerns about this?

@jezdez
Copy link
Member

jezdez commented Mar 9, 2023

I've filed #4796 and #4797 to unblock us here.

@jakirkham
Copy link
Member Author

None. Have also been dropping Python 3.7 from other projects I work on

@jezdez jezdez merged commit bd49d73 into conda:main Mar 13, 2023
@jakirkham jakirkham deleted the use_packaging branch March 13, 2023 17:37
@jakirkham
Copy link
Member Author

Thanks Jannis! 🙏

@beckermr
Copy link
Contributor

@dholth
Copy link
Contributor

dholth commented Apr 10, 2023

It looks like the new file is https://github.com/pypa/packaging/blob/main/src/packaging/version.py with underscores in front of each function. (It would be cleaner to import a non-underscored name from that file)

@jakirkham
Copy link
Member Author

Maybe this should be raised as a new issue?

AIUI the public APIs we looked at weren't sufficient for reproducing the behavior we needed. Do you know a better public API to replace these with?

@dholth
Copy link
Contributor

dholth commented Apr 10, 2023

I'm surprised that parse from packaging's version.py has been renamed to _parse in our version, but it seems like there are other differences including us having _LegacyVersion.

@jakirkham
Copy link
Member Author

IIUC _LegacyVersion was just a vendored copy of this class (since it is being phased out and we still need it atm), but maybe I'm missing something

@github-actions github-actions bot added the locked [bot] locked due to inactivity label Apr 10, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 10, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
cla-signed [bot] added once the contributor has signed the CLA locked [bot] locked due to inactivity
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

setuptools 66 breaks conda-build
7 participants