-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[MAINTENANCE] Add check to CloudDataContext
to ensure using latest PyPI version
#7753
Merged
cdkini
merged 18 commits into
develop
from
m/lakitu-118/warn_if_not_on_latest_version_pypi
Apr 28, 2023
Merged
Changes from 2 commits
Commits
Show all changes
18 commits
Select commit
Hold shift + click to select a range
1ff4de0
feat: add version checker
cdkini 6b8ecce
chore: make everything private
cdkini 42c0783
test: add tests
cdkini 049c9b4
chore: more defensive coding
cdkini ba98f88
chore: more defense
cdkini 89503a2
chore: more cleanup
cdkini 487433a
Merge branch 'develop' of https://github.com/great-expectations/great…
cdkini 93b1355
chore: misc cleanup
cdkini a0ca2fd
Merge branch 'develop' into m/lakitu-118/warn_if_not_on_latest_versio…
cdkini fdcda6b
Merge branch 'develop' of https://github.com/great-expectations/great…
cdkini bfdede9
chore: rename flag
cdkini 325c7b8
Merge branch 'm/lakitu-118/warn_if_not_on_latest_version_pypi' of htt…
cdkini d9d7e55
Merge develop into m/lakitu-118/warn_if_not_on_latest_version_pypi
github-actions[bot] dc66257
Merge develop into m/lakitu-118/warn_if_not_on_latest_version_pypi
github-actions[bot] 2f46be1
Merge branch 'develop' into m/lakitu-118/warn_if_not_on_latest_versio…
cdkini 5790173
Merge develop into m/lakitu-118/warn_if_not_on_latest_version_pypi
github-actions[bot] 4132afc
Merge develop into m/lakitu-118/warn_if_not_on_latest_version_pypi
github-actions[bot] 39c43c6
Merge develop into m/lakitu-118/warn_if_not_on_latest_version_pypi
github-actions[bot] File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,71 @@ | ||
from __future__ import annotations | ||
|
||
import json | ||
import logging | ||
|
||
import requests | ||
from packaging import version | ||
from typing_extensions import TypedDict | ||
|
||
logger = logging.getLogger() | ||
|
||
|
||
class _PyPIPackageInfo(TypedDict): | ||
version: str | ||
|
||
|
||
class _PyPIPackageData(TypedDict): | ||
info: _PyPIPackageInfo | ||
|
||
|
||
class _VersionChecker: | ||
|
||
_BASE_PYPI_URL = "https://pypi.org/pypi" | ||
_PYPI_GX_ENDPOINT = f"{_BASE_PYPI_URL}/great_expectations/json" | ||
|
||
def __init__(self, user_version: str) -> None: | ||
self._user_version = version.Version(user_version) | ||
|
||
def check_if_using_latest_gx(self) -> None: | ||
pypi_version = self._get_latest_version_from_pypi() | ||
if not pypi_version: | ||
logger.debug("Could not compare with latest PyPI version; skipping check.") | ||
return | ||
|
||
if self._is_using_outdated_release(pypi_version): | ||
self._warn_user(pypi_version) | ||
|
||
def _get_latest_version_from_pypi(self) -> version.Version | None: | ||
response_json: _PyPIPackageData | None = None | ||
try: | ||
response = requests.get(self._PYPI_GX_ENDPOINT) | ||
response.raise_for_status() | ||
response_json = response.json() | ||
except json.JSONDecodeError as jsonError: | ||
logger.debug(f"Failed to parse PyPI API response into JSON: {jsonError}") | ||
except requests.HTTPError as http_err: | ||
logger.debug( | ||
f"An HTTP error occurred when trying to hit PyPI API: {http_err}" | ||
) | ||
except requests.Timeout as timeout_exc: | ||
logger.debug( | ||
f"Failed to hit the PyPI API due a timeout error: {timeout_exc}" | ||
) | ||
|
||
if not response_json: | ||
return None | ||
|
||
info = response_json["info"] | ||
pkg_version = info["version"] | ||
|
||
return version.Version(pkg_version) | ||
|
||
def _is_using_outdated_release(self, pypi_version: version.Version) -> bool: | ||
return pypi_version > self._user_version | ||
|
||
def _warn_user(self, pypi_version: version.Version) -> None: | ||
logger.warning( | ||
f"You are using great_expectations version {self._user_version}; " | ||
f"however, version {pypi_version} is available.\nYou should consider " | ||
"upgrading via `pip install great_expectations --upgrade`\n." | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might be worth it to create a
responses
fixture mock for this.Or... adding a check here to see if the code is being run from pytest.
https://stackoverflow.com/a/44595269/6304433
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ooh might ask for some help here to make sure no other tests are impacted - will probably tackle tomorrow