DM-39402: Refactor python package determination to use a hierarchy #159
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The smptd package is deprecated and will be removed in python 3.12. We don't use chunk anywhere so use that for the import test instead.
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## main #159 +/- ##
==========================================
+ Coverage 93.80% 93.89% +0.08%
==========================================
Files 43 43
Lines 2775 2799 +24
==========================================
+ Hits 2603 2628 +25
+ Misses 172 171 -1
☔ View full report in Codecov by Sentry. |
ktlim
reviewed
May 31, 2023
* Skip python standard library and assign them python version. They do not have individual package versions and there is no point in asking. * Use a hierarchical approach to version determination. Assume that if a.b.c has a version that there is no need to look for a.b.c.d. Also assume any component in the hierarchy that starts with an underscore is irrelevant since that is a private import. These changes can make version determination an order of magnitude faster.
3.8 and 3.9 do not know the stdlib packages and so by default will not get version numbers for them. This means that "chunk" can not be used as a test that new packages turn up after importing. Instead we fall back to using "smtpd" which works on 3.8 because it has a version string even though most stdlib packages do not.
_version is filtered out and .version will likely never be encountered since module "x" should return a version without needing "x.version".
Instead of creating a hierarchical dict and then recursing through the dict, take the simpler approach of sorting the modules names and then comparing the previous line's version with the current line's hierarchy.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Using
importlib.metadata.versioncan be slow so using it for "a.b.c" and "a.b.c.d1" and "a.b.c.d2" when we know the version of "a.b.c" is wasteful. Use the hierarchy to minimize the number of version requests. Also skip standard library modules.Checklist
doc/changes