Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: --about to list the python versions supported by the connector #1628

Closed
pnadolny13 opened this issue Apr 20, 2023 · 6 comments · Fixed by #1800
Closed

feat: --about to list the python versions supported by the connector #1628

pnadolny13 opened this issue Apr 20, 2023 · 6 comments · Fixed by #1800
Assignees
Labels
kind/Feature New feature or request valuestream/SDK

Comments

@pnadolny13
Copy link
Contributor

It would be great if SDK plugins would advertise their supported python versions. This way we can scrape them for:

  • listing on the hub
  • analyzing the ecosystem i.e. identifying taps that should be upgraded to support newer python versions or remove support for deprecated versions

I'm not exactly sure what format this should be in but something like whats in the pypoetry.toml would probably work python = "<3.12,>=3.7.1"

cc @WillDaSilva

@edgarrmondragon
Copy link
Collaborator

edgarrmondragon commented Apr 20, 2023

If you don't need the parsed version range, then this a relatively low effort as we're already using importlib.metadata to extract the version information, and we could use it to extract the Requires-Python key from the package metadata.

Also, fwiw in the script where you install the package to run --about, you could in theory get the Python version information even for taps/targets not built with the SDK.

@tayloramurphy
Copy link
Collaborator

This is related to:

We'll need to align the Hub with the SDK on this but I'm very supportive of making this happen!

@WillDaSilva
Copy link
Member

Getting the Requires-Python string from the package metadata should work well. It'd be nice if we could normalize it (e.g. transforming ~= specifiers into a >= and < pair), but I'm not aware of any simple way of doing that. We won't be able to get a "fully" normalized form no matter what we do since the field can be very complex, and we don't know what all of the allowed values are (e.g. what versions of Python might exist in the future).

@edgarrmondragon @pnadolny13 In light of the above, here are two approaches we should consider:

  1. Instead of (or in addition to) reporting the Requires-Python string as-is, we report a boolean value for each version of Python we care to check for whether the plugin supports it. This would look something like this:
    from importlib.metadata import metadata
    
    from packaging.specifiers import SpecifierSet
    from packaging.version import Version
    
    
    supported_python_versions = SpecifierSet(metadata("plugin_name")["Requires-Python"])
    for v in ("3.7", "3.8", "3.9", "3.10", "3.11", "3.12", "3.13")
        if v in supported_python_versions:
            report_supported_python_version(v)
  2. Alternatively, we don't perform the above checks within the plugin, and instead defer this processing to the Hub or whatever else is consuming this data. Ultimately whatever is processing Requires-Python will likely need to use the packaging library as shown above in order to be accurate.

@edgarrmondragon
Copy link
Collaborator

@WillDaSilva I vote for 2. It's probably better to return the raw specifier and let downstream consumers get the details they need by parsing it.

@WillDaSilva
Copy link
Member

@edgarrmondragon Agreed that that info should be provided. Thoughts on doing both? Providing the raw specifier, and also parsing some more immediately useful info out of it.

@edgarrmondragon
Copy link
Collaborator

@edgarrmondragon Agreed that that info should be provided. Thoughts on doing both? Providing the raw specifier, and also parsing some more immediately useful info out of it.

@WillDaSilva Yeah, that makes sense. Your example is a good start.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/Feature New feature or request valuestream/SDK
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants